Original: Emergence of compositional language in communication through noisy channel
Revised: Emergence of compositional language in communication through noisy channel

Original: Anonymous Authors

Original: Abstract
Revised: Abstract

Original: In this paper, we investigate how communication through a noisy channel can lead to the emergence of compositional language.
Revised: In this paper, we investigate how communication through a noisy channel can lead to the emergence of compositional language.

Original: Ournovel approach is end-to-end, allows for different inductive biases on the agents’ architecture, and trains without periodical resets of the networks’ weights.
Revised: Our approach is end-to-end, allows for different inductive biases on the agents’ architecture, and trains without periodical resets of the networks’ weights.

Original: This relaxes some of the assumptions in recently developed methods.
Revised: This relaxes some of the assumptions in recently developed methods.

Original: The impact on the structure of the resulting language is shown in the context of signaling games.
Revised: The impact on the structure of the resulting language is shown in the context of signaling games.

Original: We also develop a new metric for measuring degree of compositionality.
Revised: We also develop a new metric for measuring degree of compositionality.

Original: 1. Introduction
Revised: 1. Introduction

Original: Communication can emerge in situations which require coordination and information sharing to achieve a joint objective.
Revised: Communication can emerge in situations that require coordination and information sharing to achieve a joint objective.

Original: This is common in multi-agent systems with partial observation Foerster et al.
Revised: This is common in multi-agent systems with partial observation Foerster et al.

Original: ; Lazaridou et al.
Revised: ; Lazaridou et al.

Original: ; Jaques et al.
Revised: ; Jaques et al.

Original: (2018); Raczaszek-Leonardi et al.
Revised: (2018); R ˛aczaszek-Leonardi et al.

Original: Communication is compositional if complex signals can be represented as a combination of their constituents.
Revised: Communication is compositional if complex signals can be represented as a combination of their constituents.

Original: It is a feature of human languages, and it facilitates generalization, productivity, and knowledge sharing.
Revised: It is a feature of human languages, and it facilitates generalization, productivity, and knowledge sharing.

Original: As such, it is considered essential for general intelligence
Revised: As such, it is considered essential for general intelligence

Original: (Lake et al.
Revised: (Lake et al.

Original: Noise is ubiquitous in non-digital communication.
Revised: Noise is ubiquitous in non-digital communication.

Original: For example, in biology, it has profound importance being the driving force of evolution.
Revised: For example, in biology, it has profound importance being the driving force of evolution.

Original: It is also present in human communication (Rothwell (1999)) and is perhaps equally signiﬁcant, if, indeed, it promotes compositionality.
Revised: It is also present in human communication (Rothwell (1999)) and is perhaps equally signiﬁcant, if, indeed, it promotes compositionality.

Original: Our main contribution is showing experimentally that this is the case in a simple communication model: signaling games.
Revised: Our main contribution is showing experimentally that this is the case in a simple communication model: signaling games.

Original: We argue that this effect might follow from the fact that a compositional language is (partially) robust to the mistakes caused by a noisy channel.
Revised: We argue that this effect might follow from the fact that a compositional language is (partially) robust to the mistakes caused by a noisy channel.

Original: Signaling games (Lewis (1969); Skyrms (2010);
Revised: Signaling games (Lewis (1969); Skyrms (2010);

Original: Lazaridou et al.
Revised: Lazaridou et al.

Original: ; Fudenberg et al.
Revised: ; Fudenberg et al.

Original: (1991)) are popular model of communication.
Revised: (1991)) are popular model of communication.

Original: In such a game, there is a sender and a receiver.
Revised: In such a game, there is a sender and a receiver.

Original: The former observes a state of the world and sends a message to the latter.
Revised: The former observes a state of the world and sends a message to the latter.

Original: Upon receiving a message, the receiver performs an action, which results in a reward or a loss.
Revised: Upon receiving a message, the receiver performs an action, which results in a reward or a loss.

Original: Typically, both the agents are rewarded if the receiver can identify the state of the world communicated by the sender.
Revised: Typically, both the agents are rewarded if the receiver can identify the state of the world communicated by the sender.

Original: Compositionality rarely emerges in standard versions of this model.
Revised: Compositionality rarely emerges in standard versions of this model.

Original: While signaling games are inherently a (multi-agent) reinforcement learning setup, in the simplest case, it can be represented as a supervised learning problem.
Revised: While signaling games are inherently a (multi-agent) reinforcement learning setup, in the simplest case, it can be represented as a supervised learning problem.

Original: More precisely, it can be viewed as a classiﬁcation task, with the underlying architecture resembling a discrete auto-encoder: the encoder acts as a sender, the bottleneck corresponds to a communication channel, and a decoder plays the role of a receiver.
Revised: More precisely, it can be viewed as a classiﬁcation task, with the underlying architecture resembling a discrete auto-encoder: the encoder acts as a sender, the bottleneck corresponds to a communication channel, and a decoder plays the role of a receiver.

Original: Without additional constraints, good accuracy can be achieved with non-compositional communication protocolit is enough that the sender distinguishes the features of the observed state in his messages.
Revised: Without additional constraints, good accuracy can be achieved with non-compositional communication protocolit is enough that the sender distinguishes the features of the observed state in his messages.

Original: It is likely that such a mapping will show poor generalization to novel combinations of features (Kottur et al., 2017).
Revised: It is likely that such a mapping will show poor generalization to novel combinations of features (Kottur et al., 2017).

Original: Our experiments were conducted using a set of images with two features: shape and color (in a similar vein to Choi et al.
Revised: Our experiments were conducted using a set of images with two features: shape and color (in a similar vein to Choi et al.

Original: (2018), Bogin et al.
Revised: (2018), Bogin et al.

Original: (2018), and Korbak et al.
Revised: (2018), and Korbak et al.

Original: We obtained some promising results, which demonstrate that introducing a noisy channel to the system, indeed creates enough tension for compositionality to emerge.
Revised: We obtained some promising results, which demonstrate that introducing a noisy channel to the system, indeed creates enough tension for compositionality to emerge.

Original: Additionally, we show that the noisy channel can be modeled as a simple dense layer, which takes probabilities as input, has a ﬁxed weight matrix of a certain form, and uses a logarithmic activation function.
Revised: Additionally, we show that the noisy channel can be modeled as a simple dense layer, which takes probabilities as input, has a ﬁxed weight matrix of a certain form, and uses a logarithmic activation function.

Original: This constitutes a small change to the supervised learning pipeline, and is interesting in its own right (it can be viewed as a yet another regularizing layer).
Revised: This constitutes a small change to the supervised learning pipeline and is interesting in its own right (it can be viewed as a yet another regularizing layer).

Original: In particular, this does not require template transfer (Korbak et al.
Revised: In particular, this does not require template transfer (Korbak et al.

Original: periodical resets of the agents’ weights or memory (Li & Bowling (2019), Kottur et al.
Revised: periodical resets of the agents’ weights or memory (Li & Bowling (2019), Kottur et al.

Original: (2017), Das et al.
Revised: (2017), Das et al.

Original: The approach is also agnostic to the choice of neural network architectures for a receiver and sender.
Revised: The approach is also agnostic to the choice of neural network architectures for a receiver and sender.

Original: This makesit less cognitively demanding, when compared to other methods (e.g.  the obverter algorithm  Batali (1998), Choi et al.
Revised: This makes less assumptions about the cognitive abilities of the agents, when compared to other methods (e.g. in the obverter algorithm it is assumed that an agent can use its own responses to messages to predict other agent’s responses, see Batali (1998), Choi et al.

Original: Measuring compositionalitymeasures is another research question.
Revised: Measuring compositionality is quite challenging.

Original: To this end we introduce a new, conﬂict counting , measure and use to literature measures: topographical similarity (see e.g. Brighton & Kirby (2006);
Revised: We introduce a new, conﬂict counting , measure and use it alongside topographical similarity, a metric commonly used in the literature (see e.g. Brighton & Kirby (2006);

Original: Lazaridou et al.
Revised: Lazaridou et al.

Original: (2018) and Kriegeskorte (2008); Bouchacourt & Baroni (2018), where it is called a representational similarity ).
Revised: (2018) and Kriegeskorte (2008); Bouchacourt & Baroni (2018), where it is called a representational similarity ).

Original: We provide a discussion of conﬂict counting, highlighting some interesting phenomenons that emerge. We also provide analysis of compositionality with respect to  the  noise level.
Revised: We provide a discussion concerning these measures and use them to analyze the relationship between noise level and compositionality of emergent language.

Original: The rest of the paper is organized as follows.
Revised: The rest of the paper is organized as follows.

Original: In the next section, we discuss related works.
Revised: In the next section, we discuss related works.

Original: In Section 3, we present details of our method.
Revised: In Section 3, we present details of our method.

Original: Section 4 is devoted to experimental results.
Revised: Section 4 is devoted to experimental results.

Original: We discuss future work and conclude the paper in
Revised: We discuss future work and conclude the paper in

Original: Section 5.
Revised: Section 5.

Revised: The neural network architecture and the choice of hyperparameters can be found in Appendix A. For comparison of the aforementioned compositionality measures, see

Revised: Appendix B. Finally, theoretical results concerning optimality of compositional communication over a noisy channel were placed in Appendix C.

Original: 2. Related work
Revised: 2. Related work

Original: Signaling games (Lewis (1969); Skyrms (2010);
Revised: Signaling games (Lewis (1969); Skyrms (2010);

Original: Lazaridou et al.
Revised: Lazaridou et al.

Original: ; Fudenberg et al.
Revised: ; Fudenberg et al.

Original: (1991)) are popular models of communication and have traditionally been solved using either simple reinforcement learning (Skyrms, 2010) or evolutionary optimization (Cangelosi, 2001; Grouchy et al., 2016).
Revised: (1991)) are popular models of communication and have traditionally been solved using either simple reinforcement learning (Skyrms, 2010) or evolutionary optimization (Cangelosi, 2001; Grouchy et al., 2016).

Original: An early work on the subject utilizing neural networks is (Rumelhart et al., 1986), while the recent results taking advantage of deep learning progress Lazaridou et al.
Revised: Early work on the subject utilizing neural networks is (Rumelhart et al., 1986), while the recent results taking advantage of deep learning progress Lazaridou et al.

Original: (2018), Bouchacourt & Baroni (2018).
Revised: Bouchacourt & Baroni (2018).

Original: Kottur et al.
Revised: Kottur et al.

Original: (2017) argue that a strong inductive biases are necessary for the emergence of compositionality between communicating agents.
Revised: (2017) argue that a strong inductive bias is necessary for the emergence of compositionality between communicating agents.

Original: Das et al.
Revised: Das et al.

Original: (2017) placesa pressure on agents, to use symbols consistently across varying contexts, by a frequent reset of the agent’s memory.
Revised: (2017) places pressure on agents, to use symbols consistently across varying contexts, by a frequent reset of the agent’s memory.

Original: The topic of communication is inherently interesting in the context of mult-agent RL system, see Hernandez-Leal et al.
Revised: The topic of communication is inherently interesting in the context of a multi-agent RL system, see Hernandez-Leal et al.

Original: (2020, Table 2) for up-to-date survey information.
Revised: (2020, Table 2) for a recent survey.

Original: A psychologically driven approach was proposed by Choi et al.
Revised: A psychologically driven approach was proposed by Choi et al.

Original: (2018) and Bogin et al.
Revised: (2018) and Bogin et al.

Original: (2018), who build upon the obverter algorithm (Oliphant & Batali (1997), Batali (1998)).
Revised: (2018), who build upon the obverter algorithm (Oliphant & Batali (1997), Batali (1998)).

Original: This algorithm assumes that an agent can use its own policy as a model to predict the response of the other agent.
Revised: This algorithm assumes that an agent can use its own policy as a model to predict the response of the other agent.

Original: This approach could be connected with theory of mind (Premack &
Revised: This approach could be connected with theory of mind (Premack &

Original: Woodruff (1978)) and its variant, simulation theory (Gordon (1986), Heal (1986)).
Revised: Woodruff (1978)) and its variant, simulation theory (Gordon (1986), Heal (1986)).

Original: The obverter algorithm has several limitations (the agents must share an identical architecture and the task must be symmetric) and Korbak et al.
Revised: The obverter algorithm has several limitations (the agents must share an identical architecture and the task must be symmetric) and Korbak et al.

Original: (2019) used an alternative approach, based on the idea of template transfer (Barrett & Skyrms (2017)).
Revised: (2019) used an alternative approach, based on the idea of template transfer (Barrett & Skyrms (2017)).

Original: A different take on compositionality, creates  bias towards protocols that are easy to teach to new agents (Kirby (2001);
Revised: A different take on compositionality creates a bias towards protocols that are easy to teach to new agents (Kirby (2001);

Original: Kirby et al.
Revised: Kirby et al.

Original: (2008), Brighton (2002)).
Revised: (2008), Brighton (2002)).

Original: In the machine learning literature, this idea was explored by Li & Bowling (2019) and Cogswell et al. 
Revised: In the machine learning literature, this idea was explored by Li & Bowling (2019) and Cogswell et al. In a similar spirit, De Beule &

Original: Gradually increasing task complexity that incentivizes reusing existing patterns of communication (De Beule & Bergen, 2006).
Revised: Bergen (2006) gradually increased task complexity that incentivized the reuse of existing patterns of communication. The notion of compositional language is also related to the idea of learning disentangled representations from highdimensional inputs (see e.g. Higgins et al. (2017), Locatello et al. , Kim & Mnih (2018)).

Original: The use of noise as a regularizer is a powerful concept in deep learning, see e.g. dropout (Srivastava et al.
Revised: The use of noise as a regularizer is a powerful concept in deep learning, see e.g. dropout (Srivastava et al.

Original: or semantic hashing (Salakhutdinov & Hinton (2009)).
Revised: or semantic hashing (Salakhutdinov & Hinton (2009)).

Original: The latter was used in discrete autoencoders (Kaiser & Bengio (2018)) as a mechanism allowing backpropagation through a discrete latent.
Revised: The latter was used in discrete autoencoders (Kaiser & Bengio (2018)) as a mechanism allowing backpropagation through a discrete latent.

Original: A similar idea was also applied in the context of learning to communicate (Foerster et al.
Revised: A similar idea was also applied in the context of learning to communicate (Foerster et al.

Revised: The authors used noise as a mechanism to backpropagate through a discrete communication channel, and observed that it is essential for successful training.

Revised: They also investigated the channel’s capacity.

Original: In the context of compositionality, Li & Bowling (2019) used noise to reset the weights of the agents’ neural networks.
Revised: In the context of compositionality, Li & Bowling (2019) used noise to promote the emergence of this phenomenon, by resetting the weights of the agents’ neural networks.

Original: In this work we use the noise explicitly to corrupt the communication channel.
Revised: In this work, we use the noisy channel exclusively, as a mechanism to encourage compositionality.

Revised: The backpropagation through a discrete communication channel is done differently, by the use of a Gumbel softmax.

Original: 3. Method
Revised: 3. Method

Original: Training pipeline Consider a sender s θ , modeled as a neural network, which observes an image x from some dataset D .
Revised: Training pipeline Consider a sender s θ , modeled as a neural network, which observes an image img from some dataset D .

Original: We assume that each element of D has K independent features f 1 , . . .
Revised: We assume that each element of D has K independent features f 1 , . . .

Original: , f K (here we consider K = 2 ).
Revised: , f K (here we consider K = 2 ).

Original: The sender sends a message comprising of L symbols (here we assume L = K ).
Revised: The sender sends a message comprising of L symbols (here we assume L = K ).

Original: Basing on these symbols, the receiver has to guess the values for each of K features correctly.
Revised: Basing on these symbols, the receiver has to guess the values for each of K features.

Original: Both the features and symbols are discrete and enumerated with
Revised: Both the features and symbols are discrete and enumerated with

Original: A f = { 1 , . . .
Revised: A f = { 1 , . . .

Original: , d f } and A s = { 1 , . . .
Revised: , d f } and A s = { 1 , . . .

Original: , d s } , respectively.
Revised: , d s } , respectively.

Original: Formally, s θ ( x ) =
Revised: Formally, s θ ( img ) =

Original: ( s iθ ( x ))
Revised: ( s iθ ( img ))

Original: Ki =1 , where s iθ ( x ) =
Revised: Ki =1 , where s iθ ( img ) =

Original: ( s ij,θ ( x ))
Revised: ( s ij,θ ( img ))

Original: dj =1 ∈ P ( A s )  represents the probability distribution1 corresponding to  i th symbol.
Revised: d s j =1 ∈ P ( A s ) 1 represents the probability distribution corresponding to the i th symbol.

Original: Deﬁne a function noise : P ( A s ) → R d s as follows:
Revised: Deﬁne a function noise : P ( A s ) → R d s as follows:

Original: [Equation]
Revised: [Equation]

Original: where W ∈ R d s × d s is a ﬁxed matrix, such that Wx >and
Revised: where W ∈ R d s × d s is a ﬁxed matrix, such that Wx >and

Original: Wx ∈ P
Revised: Wx ∈ P

Original: ( A s ) , for any x ∈ P ( A s ) 2 .
Revised: ( A s ) , for any x ∈ P ( A s ) 2 .

Original: The second condition on W is satisﬁed, for instance, by a family of stochastic matrices; several examples are also given at the end of this section.
Revised: The second condition on W is satisﬁed, for instance, by a family of stochastic matrices; several examples are also given at the end of this section.

Original: Let (cid:98) s iθ ( x ) denotes the distribution ofa i th symbol which passes through the noisy channel:
Revised: Let (cid:98) s iθ ( img ) denotes the distribution of i th symbol which passes through the noisy channel:

Original: [Equation]
Revised: [Equation]

Original: Suppose further that g i
Revised: Suppose further that g i

Original: = ( g i 1 , . . .
Revised: = ( g i 1 , . . .

Original: , g id s ) is a vector of i.i.d. 
Revised: , g i d s ) is a vector of i.i.d. [Equation]

Revised: We could also deﬁne noise for all x ∈ R m , for some m , by ﬁrst applying softmax to x , and then using equation (1).

Original: Gumbel (0 , 1) random variables. Deﬁne
Revised: Gumbel (0 , 1) random variables and deﬁne the following functions:

Original: [Equation]
Revised: [Equation]

Original:  The receiver neural network is deﬁned as r ψ ( m ) =
Revised: Let [Equation] The receiver neural network is deﬁned as r ψ ( m ) =

Original: ( r iψ ( m ))
Revised: ( r iψ ( m ))

Original: Ki =1 , where r iψ ( m ) =
Revised: Ki =1 , where r iψ ( m ) =

Original: ( r ij,ψ ( m ))
Revised: ( r ij,ψ ( m ))

Original: dj =1 ∈ P
Revised: dj =1 ∈ P

Original: ( A f ) represents the probability distribution on A f , corresponding to  i th feature.
Revised: ( A f ) represents the probability distribution on A f , corresponding to the i th feature.

Original: [Equation]

Original: We could also deﬁne noise for all x ∈ R m , for some m , by ﬁrst applying softmax to x , and then using equation (1).

Original: The functions gumbel sample and gumbel softmax are deﬁned as follows:

Original: [Equation]

Original: In the Straight-Through mode (see Jang et al.
Revised: In the Straight-Through mode (see Jang et al.

Original: , r ψ takes (cid:98) m as input half  the time, and(cid:101) m , the remaining half  the time. Here 
Revised: , r ψ takes (cid:98) m as input half of the time, and the remaining half of the time, it takes (cid:101) m . Here (cid:101) m = stop _ gradient ( (cid:98) m

Original: [Equation]
Revised: [Equation]

Original: i.e. (cid:98) m 
Revised: m ) + m [Equation] gumbel _ sample

Original:  =
Revised: [Equation] ( img [Equation] A, i.e. (cid:98) m =

Original: ( (cid:98) m i )
Revised: ( (cid:98) m i )

Original: Ki =1 is a sampled noisy message.
Revised: Ki =1 is a sampled noisy message.

Original: The neural networks are trained using
Revised: The neural networks are trained using

Original: [Equation]
Revised: [Equation]

Original: The cross-entropy loss is deﬁned as
Revised: The cross-entropy loss is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: Furthermore, L KL =
Revised: Furthermore, L KL =

Original: E x ∼D (cid:104)(cid:80)
Revised: E x ∼D (cid:104)(cid:80)

Original: Ki =1 KL ( U [0 , 1] || s iθ ( x )) (cid:105) .
Revised: [Equation]

Original: and L l 2 =
Revised: and L l 2 =

Original: ||
Revised: ||

Original: θ
Revised: θ

Original: ||
Revised: ||

Original: + || ψ
Revised: + || ψ

Original: || 2 .
Revised: || 2 .

Original: Compositionality measures In this paper we use two compositionality measures: topographical similarity (see e.g. Brighton & Kirby (2006), Lazaridou et al.
Revised: Compositionality measures In this paper we, use two compositionality measures: topographical similarity (see e.g. Brighton & Kirby (2006), Lazaridou et al.

Original: Kriegeskorte (2008), Bouchacourt & Baroni (2018)), and our new conﬂict counting measure described now and deﬁned in (2). 
Revised: Kriegeskorte (2008), Bouchacourt & Baroni (2018)), and present a new, conﬂict counting measure described below and deﬁned in (2). For more discussion on the topic, see also Section 4 and Appendix B.

Original: We assumed L = K , the number of symbols generated by the sender is equal to the number of features.
Revised: We assumed L = K , the number of symbols generated by the sender is equal to the number of features.

Original: In such a setting, we expect that compositional language will use one symbol for each feature.
Revised: In such a setting, we expect that compositional language will use one symbol for each feature.

Original: To simplify we also assume that the permutation of the position of symbols to the position of features, φ : { 1 , . . .
Revised: To simplify we also assume that the permutation of the position of symbols to the position of features, φ : { 1 , . . .

Original: , L } (cid:55)→ { 1 , . . .
Revised: , L } (cid:55)→ { 1 , . . .

Original: , K } , is ﬁxed.
Revised: , K } , is ﬁxed.

Original: We say that the principal meaning of a symbol s at position
Revised: We say that the principal meaning of a symbol s at position

Original: j assuming φ is m ( s, j ; φ ) :
Revised: j assuming φ is m ( s, j ; φ ) :

Original: = arg max f count ( s, j, f ; φ ) , where 
Revised: = arg max f count ( s, j, f ; φ ) , where count ( s, j,

Original: [Equation] 
Revised: [Equation] [Equation]

Original: and ties in arg max are broken arbitrarily.
Revised: and ties in arg max are broken arbitrarily.

Original: Our metric is deﬁned as:
Revised: Our metric is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where score ( s, j ; φ ) = (cid:80) f (cid:54) = m ( s,j ; φ ) count ( s, j, f ; φ ) .
Revised: where score ( s, j ; φ ) = (cid:80) f (cid:54) = m ( s,j ; φ ) count ( s, j, f ; φ ) .

Original: Intuitively, score measures how many times  feature assigned to a symbol s at a position j diverts from its principal meaning m ( s, j ; φ ) .
Revised: Intuitively, score measures how many times the feature assigned to a symbol s at a position j diverts from its principal meaning m ( s, j ; φ ) .

Original: conflicts totals these errors and takes min over possible orderings φ .
Revised: conflicts totals these errors and takes min over possible orderings φ .

Original: Noise In this paragraph, we show how to deﬁne different noise channels using  noise layer, as deﬁned in (1).
Revised: Noise In this paragraph, we show how to deﬁne different noise channels using the noise layer, as deﬁned in (1).

Original: Consider a probability distribution p ∈ P ( d s ) .
Revised: Consider a probability distribution p ∈ P ( d s ) .

Original: Then Wp can represent a diverse set of distributions 4 .
Revised: Then Wp can represent a diverse set of distributions 3 .

Original: A uniform noise applied to p manifests itself as a mixture distribution of p and U { 1 , . . .
Revised: A uniform noise applied to p manifests itself as a mixture distribution of p and U { 1 , . . .

Original: , d s } , with  density equal to (1 − ε ) 
Revised: , d s } , with a density equal to (1 − ε ) p

Original: p + εd s , where ε is the probability of randomly scrambling the symbol.
Revised: + εd s , where ε is the probability of randomly scrambling the symbol.

Original: The logits of such a distribution can be represented as noise ( p ) , where W , deﬁned in (1), can take the following form: W ij = 1 −  (1 − 1 /d s ) and W ij = ε/d s for i (cid:54) =
Revised: The logits of such a distribution can be represented as noise ( p ) , where W , deﬁned in (1), can take the following form: W ii = 1 − ε (1 − 1 /d s ) and W ij = ε/d s for i (cid:54) =

Original: j .
Revised: j .

Original: Here, the probability of changing a symbol equals ε (1 − 1 /d s ) .
Revised: Here, the probability of changing a symbol equals ε (1 − 1 /d s ) .

Original: To make it equal to ε , we deﬁneW as
Revised: To make it equal to ε , we deﬁne

Original: W ij =
Revised: W as

Original: [Equation]
Revised: [Equation]

Original: The formula given in (3) is the one we use in this paper.
Revised: The formula given in (3) is the one we use in this paper.

Original: Similarly, a formula for a bit-ﬂipping noise can be obtained 5 .
Revised: Similarly, a formula for a bit-ﬂipping noise can be obtained 4 .

Original: 4. Experiments
Revised: 4. Experiments

Original: Experiments setup For our experiments, we chose a subset of a dataset used by Choi et al.
Revised: Experiments setup For our experiments, we chose a subset of a dataset used by Choi et al.

Original: (2018) 6 and Korbak et al.
Revised: (2018) 5 and Korbak et al.

Original: (2019), by selecting four shapes (box, cylinder, ellipsoid, sphere) in four colors (blue, cyan, gray, green), see Figure 1.
Revised: (2019), comprised of four shapes (box, cylinder, ellipsoid, sphere) in four colors (blue, cyan, gray, green), see Figure 1.

Original: Each image has dimensions of 128 x 128 x 3 and there is 100 images for each shape–color pair.
Revised: Each image has dimensions of 128 × 128 × 3 and there are 100 images for each shape–color pair.

Original: The senderconsists of a vision and encoding module.
Revised: The sender and the receiver are implemented as feed-forward neural networks.

Original: The former consists of two convolution layers (each havingﬁlter, kernel 4x4, stride 1, elu activation function). After each convolutional layer there is a 2x2 max pool layer. The output of the last max pool layer is passed through two dense layers (with 64 neurons and elu activation) and linear classiﬁer with softmax for each symbol.
Revised: For details concerning architecture and the choice of hyperparameters, see Appendix A.

Original: The receiver’s network takes one-hot encoded symbols s as input.
Revised: The compositionality of emerging languages was measured on the training set as our dataset was not big enough to

Original: Consequently, s and 1 − s are passed to a dense layers.
Revised: = Y i ⊕ j , where ⊕ is a xor operation, Y 1 = 1 − ε and Y

Original: The sum of those layers isprocessed through the elu activation function and two dense layers (as in Kaiser & Bengio (2018)). Finally, there are two linear classiﬁers - one for shape and one for color. Each dense layer in the receiver has a width of 64.
Revised: i = ε 1 ( i − 1 is a power of 2) / (cid:98) log 2 ( d ) (cid:99) .

Original: Thecompositionality of emerging languages was measured on the training set as our dataset was not big enough to ensure generalization (the reconstruction accuracy was low).
Revised: The dataset is available at https://github.com/ benbogin/obverter . ensure generalization (the reconstruction accuracy was low).

Original: We conjecture that this  orthogonal to compositionality and  plan to investigate this in  future work.
Revised: We conjecture that this is orthogonal to compositionality and we plan to investigate this in issue in the future work.

Original: Tight language: compositionality
Revised: Tight alphabet

Original: In this experiment we learned a language  as tight as possible.
Revised: In this experiment, we trained a language with the alphabet as tight as possible.

Original: Clearly, four symbols are enough to describe four shapes in four colors.
Revised: Clearly, four symbols are enough to describe four shapes in four colors.

Original: How- ever, we found out that it is quite challenging to learn in this setup.
Revised: However, we found out that it is quite challenging to learn in this setup.

Original: Consequently, we loosened the constraints on a language and allowed the sender to use ﬁve symbols.
Revised: Consequently, we loosened the constraints on the alphabet size and allowed the sender to use ﬁve symbols.

Revised: This allowed the model to learn, which could be attributed to higher model capacity or better ability to escape local minima during training.

Original: Example of a language learned by the sender is presented in Table 1.
Revised: An example of a language learned by the sender is presented in Table 1.

Original: Although, the language uses ﬁve symbols, only four  are used to describe  shape and  four to describe color.
Revised: Although the alphabet contains ﬁve symbols, only four of them are used to describe the shape and a different four to describe color.

Original: We speculate that adding one additional symbol created enough wiggle room to allow the training to escape local minima, and hence enabled the model to learn.
Revised: We did not use pre-training in this experiment to demonstrate the ability of our method to work in the end-to-end setup.

Original: SHAPE
Revised: [Table]

Original: BOX CYLINDER ELLIPSOID SPHERE COLOR
Revised: Abundant alphabet

Original: BLUE 0, 3, 1, 4, CYAN 0, 3, 1, 4, GREEN 0, 3, 1, 4, Big language: Compositionality is not what it seems In this experiment, we trained language signiﬁcantly bigger than necessary with | A s | = 7 .
Revised: In this experiment, we trained a model using alphabet signiﬁcantly bigger than necessary: | A s | = 7 .

Original: In half of the experiments, the sender learned a language that is compositional regarding to our metrics (with zero or one conﬂicts).
Revised: In half of the experiments, the sender learned a language that is compositional according to our metrics (with zero or one conﬂicts).

Original: Language that has a highest number of conﬂicts is presented in Table 2.
Revised: The language that has the highest number of conﬂicts is presented in Table 2.

Original: Despite poor values of the metrics ( 9 conﬂicts, topographical similarity of 0 . 68 ) it can be argued that the language is also compositional.
Revised: Despite poor values of the metrics ( 9 conﬂicts and topographical similarity of 0 . 68 ) it can be argued that the language is also compositional.

Original: It may be seen from the table that for box and cylinder messages starts with color followed by  shape.
Revised: It may be seen from Table 2, that in the ﬁrst two columns, the messages start with a color, followed by a shape.

Original: The other two shapes have this order  reversed.
Revised: In the remaining two columns, this order is reversed.

Original: It’s possible to achieve high accuracy (0.95) because symbols used in the ﬁrst group of shapes are almost never used in the same position in the second group (only exceptionbeing 2 used for blue box and sphere as ﬁrst symbol of message).
Revised: It is possible to achieve high accuracy (0.95) because symbols used in the ﬁrst two columns are almost never used in the same position in the remaining columns (with the exception of a symbol 2).

Original: [Table]
Revised: [Table]

Original: COLOR

Original: BLUE 2, 2, 3, GRAY 6, 6, 3, 1, GREEN 4, 4, 3, 1, Noise vs metrics We examined the relation between noise level  and compositionality of emergent language.
Revised: Noise vs metrics We examined the relationship between noise level (in the noisy channel as deﬁned in (3)) and compositionality of emergent language.

Original: For each selected noise level we run a batch of experiments withdifferent seeds, and we measured compositionality using topographical similarity and conﬂicts count.
Revised: For each selected noise level we run a batch of experiments with 10 different seeds, and we measured compositionality using topographical similarity and conﬂicts count.

Original: All experiments were performed with the same pre-trained convnets.
Revised: The experiments were performed with the same pre-trained convnets.

Original: The results are presented in Figure 2.
Revised: The results are presented in Figure 2.

Original: The main take away is thatthere is no compositionality without noise in the channel. On the other hand, too large noise prevents the model from learning.
Revised: The main takeaway is that compositionality does not emerge without, or with excessive levels, of noise, see also Appendix C for theoretical justiﬁcation.

Original: As a result, there is a sweet spot for noise, located in quite a narrow window ε ∈
Revised: As a result, there is a sweet spot for noise, located in quite a narrow window ε ∈

Original: 15] , which results inin a compositional languages.
Revised: 15] , which results in compositional languages.

Original: [Figure]
Revised: [Figure]

Original: noise probability ( )

Original: [Figure] [Figure] un [Figure]
Revised: [Table]

Original: [Figure]

Original: o

Original: [Figure]
Revised: [Figure]

Original: o g r a p

Original: h

Original: i c a l s i m il a r i t

Original: y

Original: 5. Conclusions and future work
Revised: 5. Conclusions and future work

Original: We presented a promising method to achieve emergence of compositional language in communication between two agents.
Revised: We presented a promising method to achieve emergence of compositional language in communication between two agents.

Original: The key insight was to leverage randomness by introducing a noisy communication channel, which can be implemented as a regularization layer.
Revised: The key insight was to leverage randomness by introducing a noisy communication channel, which can be implemented as a regularization layer.

Original: We tested the approach on a version of  signaling game, where a sender observes an image, communicates with the receiver, and the receiver has to guess the image’s features.
Revised: We tested the approach on a version of a signaling game, where a sender observes an image, communicates with the receiver, and the receiver has to guess the image’s features.

Original: This work can be extended in several directions.
Revised: There remain many directions for future work.

Original: One of them would be to perform a fully-ﬂedged comparison with other methods, as well as to increase the  number of features and their possible range of values (e.g. by using CLEVR dataset).
Revised: Two important ones, concern improving the results on the held-out dataset, and incorporating a reinforcement learning objective. A natural further extension would be to apply the method on more complex datasets, with greater number of features and their possible range of values (e.g. by using CLEVR dataset).

Original: This could require more expressive neural network architectures, in particular sequence-to-sequence models (e.g. LSTMs or Transformers). Furthermore, the problem can be steered more towards the Reinforcement Learning framework.
Revised: It is also interesting to study how introducing a noisy layer can improve performance of neural networks in other contexts. Acknowledgments Special thanks to Joanna R ˛aczaszek-Leonardi, Julian Zubek and Tomasz Korbak.

Original: This includes a reinforcement learning objective and a broader, multi-agent, environment.
Revised: This research was supported by the

Original: Additionally, it is interesting how the training pipeline can be simpliﬁed (e.g. byremoving the KL loss).
Revised: PL-Grid Infrastructure. We extensively used the Prometheus supercomputer, located in the Academic Computer Center Cyfronet in the AGH University of Science and Technology in Kraków, Poland. The work of Piotr Miło´s was supported by the Polish National Science Center grants UMO2017/26/E/ST6/00622.

Original: Finally, it is interesting to study how introducing a noisy layer can improve performance of  neural networks in other contexts.
Revised: We managed our experiments using https://neptune.ai. We would like to thank the Neptune team for providing us access to the team version and technical support. A. Experimental setup The architecture of a neural network used in this paper is shown in Figure 3.

Revised: It consists of three main parts: the sender, the receiver, and the noisy discrete channel between them.

Revised: The sender network consists of a vision and encoding module.

Revised: The former consists of two convolutional layers (each having 8 ﬁlters, kernel 3 × 3 , stride 1, and elu activation function).

Revised: After each convolutional layer, there is a 2 ×max pool layer with stride 2.

Revised: The output of the last max pool layer is passed through two dense layers (with 64 neurons and elu activation) and a linear classiﬁer with softmax for each symbol.

Revised: The noisy channel layer consists of a dense layer with | A s | neurons, a ﬁxed weights matrix, and a log activation function.

Revised: This is followed by a Gumbel softmax layer.

Revised: The receiver takes two one-hot encoded symbols as input and concatenates them to obtain one input vector s .

Revised: Consequently, s and 1 − s are passed to dense layers.

Revised: The sum of those layers is processed through the elu activation function and two dense layers (similarly to Kaiser & Bengio (2018)).

Revised: Finally, there are two linear classiﬁers - one for shape and one for color and softmax layer.

Revised: Each dense layer in the receiver has a width of 64.

Revised: For training, we used λ KL = 0 .

Revised: 01 , λ l 2 = 0 .

Revised: 001 , an Adam optimizer with learning rate 0 .

Revised: 001 , and a batch size of 64.

Revised: To speed up training, we used a pre-trained convnets and the ﬁrst dense layer of the sender.

Revised: During the pre-training, only part of the neural network was trained.

Revised: Linear classiﬁer with 16 classes was added after the ﬁrst dense layer of the sender.

Revised: This classiﬁer was then trained to recognize classes of images from our dataset (in one class all images present the same shape and color).

Revised: In the ﬁnal experiments, the full model was used with the pre-trained weights, which were frozen during the training.

Revised: We used pretraining for most of our experiments.

Revised: It was used in the experiment with an abundant alphabet presented in Table 2 and while exploring the relationship between compositionality and noise presented in Figure 2.

Revised: However, it’s not strictly necessary and language presented in Tablewas learned in end-to-end setup (without pre-training).

Revised: B. Metrics

Revised: Conﬂict count and topographical similarity are two measures of compositionality.

Revised: Figure 4 presents a scatter plot of them.

Revised: Despite a strong correlation, there is also a difference between the two, in the way synonyms are treated.

Revised: For example, Table 3 presents a language which has a conﬂict count equal to 0 and topographical similarity 0.87.

Revised: This language uses synonyms for the gray color (symbols 0 and 1) and for an ellipsoid shape (symbols 0 and 4).

Revised: In particular, there is no conﬂict of meaning.

Revised: This is possible because the alphabet is slightly larger than the space of described features (5 symbols and only 4 features).

Revised: Topographical similarity penalizes synonyms (because similar meanings are described by distinct symbols) while conﬂict count allows for them.

Revised: We can see from the bottom part of table 3, that the receiver learned to report color as grey when the symbol on the second position is 0 or 1, regardless of the shape.

Revised: The same happens with an ellipsoid shape, which is reported when there is 0 or 4 in the ﬁrst position, regardless of the color.

Revised: [Figure]

Revised: SHAPE

Revised: BOX

Revised: CYLINDER

Revised: ELLIPSOID

Revised: SPHERE COLOR

Revised: BLUE 2, 3, 4, 1,

Revised: CYAN

Revised: [Table]

Revised: 1 symbol1 5 symbol1 11 1concat  1 − x  + 1 1 1 color1 shape

Revised: Sender

Revised: Noisy channel

Revised: Receiver

Revised: Legend:

Revised: [Figure]

Revised: C. Optimality of compositional communication

Revised: Suppose that there are K > 1 features, and feature f k takes values in a set F k .

Revised: For simplicity, we assume that the sets F 1 , . .

Revised: , F K are mutually disjoint, and |F k | =

Revised: F = | A s | =

Revised: d s , where A s = { 1 , . . .

Revised: , d s } .

Revised: This means that d f = KF .

Revised: Denote F = (cid:81)

Revised: Ki =

Revised: 1 F i .

Revised: A language is a mapping from the feature space to the set of K -symbol messages, i.e. l : F → A Ks .

Revised: For any K dimensional vectors v , w , deﬁne ρ ( v , w )

Revised: =

Revised: (cid:80) K i =1 1 ( v i = w i ) .

Revised: Let s =

Revised: ( s 1 , . . .

Revised: , s K ) ∈

Revised: A Ks be a message.

Revised: Below we assume that a noisy message, s (cid:48) =

Revised: ( s (cid:48) 1 , . . .

Revised: , s (cid:48) K ) , that is produced by the noisy channel, has the following conditional distribution:

Revised: [Equation]

Revised: for any ˆ s ∈ A Ks , and a ﬁxed ε ∈ (0 , 1) .

Revised: Suppose that l is a one-to-one mapping, and that the sender uses l to generate messages, while the receiver uses l −to decode them.

Revised: For any given symbol s , and a corrupted message s (cid:48) , the reward they get is the amount of correctly encoded features:

Revised: [Equation]

Revised: Lemma 1.

Revised: Assume that F = d s ≥ 2 and ε < ( d − 1) /d .

Revised: Then for any distribution µ ∈ P ( A Ks ) , a compositional language is optimal, in the sense that it maximizes the expected reward

Revised: [Equation]

Revised: where the max is taken over all one-to-one mappings l .

Revised: Proof.

Revised: We start by observing that

Revised: [Equation]

Revised: Deﬁne Λ k ( s ) = { α ∈ F : ρ ( α, l − 1 ( s )) =

Revised: k } and Λ k,j ( s ) =

Revised: { α ∈ F : ρ ( α, l − 1 ( s )) =

Revised: k, ρ ( s , l ( α ))

Revised: =

Revised: j } .

Revised: Then

Revised: [Equation]

Revised: Suppose, that there exists k (cid:54) =

Revised: j , such that Λ k,j ( s ) (cid:54) = ∅ .

Revised: Since, l is a one-to-one mapping and F =

Revised: d s , there exists a sequence of distinct numbers k 1 , k 2 , . . .

Revised: , k m , where m ≤ F , such that Λ k

Revised: i ,k i +1 ( s ) (cid:54) = ∅ , for i ≤ m and k m +1 = k 1 .

Revised: To see why, assume that such a k and j exist.

Revised: Then either Λ j,k (cid:54) = ∅

Revised: and we are done ( m = 1 ), or Λ j,k = ∅

Revised: and there exists j 1 / ∈ { j, k } such that Λ j,j 1 (cid:54) = ∅ .

Revised: After repeating this argument at most F times, we will ﬁnd the desired sequence k 1 , . . .

Revised: , k m .

Revised: Let α i,i +1 ∈ Λ k

Revised: i ,k i +1 ( s ) and deﬁne ˆ l which is equal to l , except for α 1 , 2 , . . . , α m,m +1 , where ˆ l ( α i,i +1 )

Revised: =

Revised: l ( α i − 1 ,i ) , and α 0 , 1 = α m,m +1 .

Revised: Then

Revised: K (cid:88)

Revised: [Equation]

Revised: −

Revised: [Equation]

Revised: 0 , where the last line follows from the rearrangement inequality and the fact that 1 − εε ( d s − 1) > 1 (by our assumption).

Revised: This means that until there exists k (cid:54) =

Revised: j such that Λ k,j ( s ) (cid:54) = ∅ , we can improve l .

Revised: So suppose that Λ k,j ( s ) = ∅ for any k (cid:54) =

Revised: j .

Revised: Then

Revised: [Equation]

Revised: and consequently,

Revised: [Equation]

Revised: Since a compositional language satisﬁes Λ k,j ( s ) = ∅ for any k (cid:54)

Revised: =

Revised: j , it is optimal.