Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments
Authors: Ryo Ueda
Tadahiro Taniguchi
Venue: ICLR 2024 poster (2024)
Keywords: Emergent Communication
Emergent Language
Probabilistic Generative Model
Variational Autoencoder
beta-VAE
Zipf’s law of abbreviation
Harris’s articulation scheme
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We reinterpret Lewis's signaling game, a frequently used setting in emergent communication, as beta-VAE and reformulate its objective function as ELBO.

Abstract: As a sub-discipline of evolutionary and computational linguistics, emergent communication (EC) studies communication protocols, called emergent languages, arising in simulations where agents communicate. A key goal of EC is to give rise to languages that share statistical properties with natural languages. In this paper, we reinterpret Lewis's signaling game, a frequently used setting in EC, as beta-VAE and reformulate its objective function as ELBO. Consequently, we clarify the existence of prior distributions of emergent languages and show that the choice of the priors can influence their statistical properties. Specifically, we address the properties of word lengths and segmentation, known as Zipf's law of abbreviation (ZLA) and Harris's articulation scheme (HAS), respectively. It has been reported that the emergent languages do not follow them when using the conventional objective. We experimentally demonstrate that by selecting an appropriate prior distribution, more natural segments emerge, while suggesting that the conventional one prevents the languages from following ZLA and HAS.

Published as a conference paper at ICLR 2024
LEWIS'S S SIGNALING GAME AS BETA-VAE FOR NATURAL WORD LENGTHS AND SEGMENTS
Ryo Ueda
Tadahiro Taniguchi
The University of Tokyo
Kyoto University Ritsumeikan University taniguchi wttieeeeiiirreeteeeeeeeeee .ac.
ryoryouedalis, www.ss...ttttottt..oo jp
ABSTRACT
As sub-discipline of evolutionary and computational linguistics, emergent munication (EC) studies communication protocols, called emergent languages, arising in simulations where agents communicate. A key goal of EC is to give rise to languages that share statistical properties with natural languages. In this paper, we reinterpret Lewis's signaling game, frequently used setting in EC. as beta-VAE and reformulate its objective function as ELBO. Consequently. we clarify the existence of prior distributions of emergent languages and show that the choice of the priors can influence their statistical properties. Specifically, we address the properties word lengths and segmentation, known as Zipf"s law abbreviation (ZLA) and Harris's articulation scheme (HAS), respectively. It has been reported that the emergent languages do not follow them when using the conventional objective. We experimentally demonstrate that by selecting an ap- propriate prior distribution, more natural segments emerge, while suggesting that the conventional prevents the languages from following ZLA and HAS.
INTRODUCTION
Understanding how language and communication emerge is one of the ultimate goals in evolution- ary linguistics and computational linguistics. Emergent communication (EC, Lazaridou & Barom.
2020; Galke al 2022 Brandizzi.
(0233 attempts to simulate the emergence of language using techniques such as deep learning and reinforcement learning. A key challenge in EC is to elucidate how emergent communication protocols, or emergent languages, reproduce statistical properties similar natural languages, such compositionality (Kottur et aLl 2017: Chaaboum et all, 2020: Ren et all, 2020), word length distribution (Chaabouni et all 201% Rita et all, 2020),, and word seg- mentation (Resnick et all, 2020; (eda all, 2023).. In this paper, reformulate Lewis's 's signaling game (Lewis 196% Skyrms, M01I)) popular setting in EC, as beta- VAE (Kingma & Welling 2014 for the emergence of more natural word lengths and segments.
Higgins all
Lewis's signaling game and its objective: Lewis's signaling game is frequently adopted in EC (e.g., Chaabount et an 2019 Rita et all 20226). It is simple communication game involving sender Ss and receiver Rg. with only unidirectional communication allowed from S to Rg. In each play, So obtains an object x and converts it into a message m. Then, Ro interprets m to guess the original object v. The game is successful upon a correct guess. Sz and Ro are typically represented as neural networks such as RNNs (Hochreiter Schmidhuber. 1199; Kho et all, 201) and optimized for certain objective Jeons. which is in most cases defined as the maximization of the log-likelihooe of the receiver's guess (or cquivalently minimization of cross-entropy loss).
ELBO as objective: In contrast, we propose to regard the signaling game as (beta-)VAE (Kingma & Welling. 2014: Higgins et all, 2012)) and utilize ELBO (with a KL-weighing parameter (3) as the game objective. We reinterpret the game representation learning in a generative model, regarding x as an observed variable, m as latent variable, So as encoder, R4 as decoder, and an additionally introduced "language model" parior as prior latent distribution. Several reasons support our refor- mulation. First, the conventional objective Joony actually resembles ELBO in some respects. We illustrate their similarity in Figure and Table MI Jeeav has an implicit prior message distribution, while ELBO contains explicit one. Also, Jeun is often adjusted with an entropy regularizer while ELBO contains entropy maximizer. Although they are not mathematically equal, their motiva-

---

Published as a conference paper at ICLR 2024
Figure 1: Illustration of similarity between signaling games and (beta-)VAE.
Latent
Massage
Prior Distribution
Prior Distribution (Language Model)
pa (m)
Encoder Parameter
Decoder & Prior
Sender
Receiver Panametir
Encoder
Decader
Sender
Recelver
Parameter
polals)
Sylnlo)
m Roleim)
Observation
Object
VAE
Signaling Game
VAE
Signaling Game
Bayesian Network
Neural Network Architecture
Table 1: A list of objective functions in this paper: It is useful to grasp their similarity.
objective
definition
prior (implicit/explcit)
entropy-related term entropy regularizer entropy regularizer (implicit)
Jocm (+, 0)
Eq. 0
(Section
P (M)
(implicit)
Eq. 3
(Section M
Pa (M)
Jabool ④ 0; a))
Eq. T
(Section
pere (M)
(explicit)
entropy maximizer entropy maximizer
Tours( (,b 0 (8)
Eq. M (Section G)
per (M) (explicit)
tions coincide in that they keep the sender's entropy high to encourage exploration (L Levind 2018).
Second, the choice of prior message distribution can affect the statistical properties of emergent languages. Specifically, this paper addresses Zipf's 's law of abbreviation (ZLA, Zipi, 1925% 1949)
and Harris's 's articulation scheme (HAS, Harris, 1955:: Tanaka-Ishti 202II.. has been reported that emergent languages do not give rise to these properties as long the conventional objective Jconv is used (Chaabouni et all 201%; Ueda et all 2023). We suggest that such an artifact is caused by unnatural prior distributions and that it has been obscure due to their implicitness. Moreover, our ELBB-baseed formulation can be justified from the computational psycholinguistic point of view, namely surprisal theory (Hale, ΣΑлI: Levy, 2008; Smith & Levy, 201)). Interestingly, the prior distribution can be seen as a language model, and thus the corresponding term can be seen as the surprisal, cognitive processing cost, of sentences (messages). In that sense, ELBO naturally models the trade-off between the informativeness and processability of sentences.
Related work and contribution: The idea per se of viewing communication representation learning is also seen in previous work. For instance, the linguistic concept compositionality is often regarded as disentanglement (Andreas, 2019 Chaabouni et all, 2020; RRenick et all A.Aso,, Tucker et all (2022) formulated VQ- game based on Variational Information Bottleneck (VIB, Alemi et all, 2017), which known as a generalization of beta- VAE. Moreover, [Tanguchi et al. (MH) naming game in which communication was formulated
(2022) defined a Ceetropolix-Hastimne as representation learning on a generative model and emerged through MCMC. However, to the best of our knowledge, no previous work has addressed in an integrated way (1) the similarities between the game and (beta-)VAE from the generative viewpoint, (2) the potential impact of prior distribution on the properties of emergent languages, and (3) psycholinguistic reinterpretation of emergent communication. They are the main contribution of this paper. Another important point is that, since Chaaboum all (2019), most EC work on signaling games has adopted either the one-character or fixed-length message setting, avoiding the variable-length setting, though the variable-length is more natural. We revisit the variable-length setting towards a better simulation of language emergence.
BACKGROUND
Mathematical Notation: Throughout this paper, we use calligraphic letter for set (e.g., A'), an uppercase letter for a random variable of the set (e.g., X), and a lowercase letter for an element of the set (e.g., 2). We denote finite object space by X. a finite alphabet by A. finite message space by M, a probability distribution of objects by Pay (X)), a probabilistic sender agent by S X)), and probabilistic receiver agent by (X.). S& is parametrized by & and Rg by O.
2.1 LANGUAGE EMERGENCE VIA LEWIS'S ' SIGNALING GAME
To simulate language emergence, defining environment surrounding agents is necessary. One of the most standard settings in EC is Lewis's 's signaling game (Lewis. 1969: Skyrms, 2010: Rita et all 20226), though there are variations in the environment definition (Foester all 2018 Lowe all Mordatch & Abbeel, 2018; Jaques et all, 2019, inter alia). D The game involves two agents,
Strictly speaking, our subject is the reconstruction game, though the discrimination (referential) game is also frequently studied (1 (Lazandou et all 20T7; ctal, 2018; Dessi atlll 2021: Rictal 2023).
2

---

Published as a conference paper at ICLR 2024
sender S& and receiver Re. In single play, So observes an object and generates message m. Re obtains the message m and guesses the object x. The game is successful if the guess is correct. Sp and Rg are often represented as RNNs and optimized toward successful communication. During training, the sender probabilistically observes an object & ~ Par (-), generates a message m (-1r), and the receiver guesses via log Ro (x/mm)) During validation, the sender observes one by one and greedily generates m..
Objects æ can be defined in various ways, from abstract to realistic data. In this paper, an object x is assumed to be an altribute-value (att-val) object (Kottur et all 2017: Chaaboum et all 2020) which is -up-l of integers (V1..... ..,.,,. Vnn)) E X, where o, E {l..... Reva) for each i. nan is called the number of attributes and Reaal is called the number of values.
Object Space:
Message Space: In most cases, the message space M is defined as a set of sequences of a (maxi- mum) length Lmux over a finite alphabet A. The message length, denoted by |m|, can be either fixed (Chaabouni atl 202() or variable (Chaabouni ell 2019). This paper adopts the latter setting:
M = {m1 m,
m, E (feos) (1 S sisk 1), my eos, S S mmax) }
(1)
where eos E A is the und-of-seueences marker. We assume |A| 2 and thus (og((AA] 1) > O.
Game Objective: The game objective is often defined as (CChaabouni etall 2019; etla 2022h):
Joomv (Φ, 0) := En- m) [log Re m)].
For clarity, we refer to Joons as the conventional objective, in contrast to our objective Jours defined later. The gradient of Jcomn ( 0) is obtained as (Chaabouni all 20T9):
Tonn (0,0== Ex- P.. (op. (o.oloogRooo/hm) (m))
m) B(x) (o. (
where B : X +
baseline. Intuitively, Φ is updated via REINFORCE (Willams, (992) and
0 via the standard backpropagation. In practice, an entropy regularizer (Willams & Peng 1991; Mnih et all 20I6) added to VoonJcom 0):
V entreg
Aentreg
Σ im
t=1 VO. (M x mimia))),,
(3)
0..0
where Aentreg is a hyperparameter adjusting the weight. It encourages the sender agent to explore the message space during training by keeping its policy entropy high. A message-lenge penalty -alm) is sometimes added to the objective to prevent messages from becoming unnaturally long:
Toons (Φ, a)) Ea (m [log Rø(x m) a m|].
(4)
Chaabouni et all (2019) and Riaa ct (2121) pointed out that the message-length penalty is neces- sary to give rise to Zipf's s law of abbreviation (ZLA) in emergent languages.
2.2 ON THE STATISTICAL PROPERTIES OF LANGUAGES
It is a key challenge in EC to fill the gap between emergent and natural languages. Though it is not straightforward to evaluate the emergent languages that are uninterpretable for human evaluators, their natural language likeness has been assessed indirectly by focusing on their statistical properties (e.g., Chaabount et all 2019; Kharitonov el all 2020). We briefly introduce previous EC studies on Zipf's law of abbreviation and Harris's articulation scheme.
Word Lengths: Zipf's 's law of abbreviation (ZLA, Zipl, 1935% (9499) refers to the statistical property of natural language where frequently used words tend to be shorter. Chaabouni et all (2019) reported
that emergent languages do not follow ZLA by the conventional objective Jeam. (0mm..gg (Ø) 0). They defined
Pass( (X) as a power-law distribution. According to ZLA,, it was expected that [m | < [mx[2] overall for (e))) such that Pot((<())) > Pay (z)2)). Their experiment, however, showed the opposite tendency [m) (2) They also reported that the emergent languages follow ZLA when the message-length penalty (Eq, H is additionally introduced. Previous work has ascribed such anti- efficiency to the lack of desirable inductive biases, such the sender's laziness (Chaabouni all, 2019) and receiver's impatience ell 2020), and memory constraints (Ueda & Washio, 202
Word Segments: Harris's 's articulation scheme (HAS, Harsss 1955 Tanaka- 2021) is a sta- tistical property of word segmentation in natural languages. According to HAS, the boundaries of word segments in character sequence can be predicted to some extent from the statistical behavior
3

---

Published as a conference paper at ICLR 2024
(11555) that there tend to be of character n-grams. This scheme is based on the discovery of Hams word boundaries at points where the number of possible successive characters after given contexts increases in English corpora. HAS reformulates it based information theory (Cover & Thomas, 2006). Formally, let BE(s) be the branching entropy of character sequences & = (1 ... On (a; E A):
BE(s):=
E
log, P(Ant1=a| =s): (5)
where is the length of s and P(An+1 = A1: =) is defined as a simple n-gram model.
BB(s) is known to decrease monotonically on average all (990), while its increase can be of special interest. HAS states that there tend to be word boundaries at BE's increasing points. HAS can be utilized as an unsupervised word segmentation algorithm (Ina ishr 2005: [anaked Shi
Shr 2007)) whose pseudo-code is shown in Appendix Al However, whether obtained segments from emergent languages are meaningful is not obvious. For instance, no one would believe that segments obtained from a random unigram model represent meaningful words; emergent languages face similar issue as they are not human languages. We do not even have a direct method to evaluate the meaningfulness since there is no ground-truth segmentation data in emergent languages. To alleviate it, Leda ct al (21) proposed the following criteria to indirectly verify their meaningfulness:
CI. The number of boundaries per message (denoted by Thbou) should increase if nau increases.
C2. The number of distinct segments in language (denoted by Neag) should increase if rlaal increases.
C3. W-TopSim C-TopSim should hold, or Aw.c := W-TopSim C-TopSim 0 should hold.
These are rephrased to some extent. C-TopSim is the normal TopSim that regards characters a E AA as symbol units, whereas W-TopSim regards segments (chunked characters) as symbol units. In the att-val setting, meaningful segments are expected to represent attributes and values in a disentangled On the other hand, + based on the concept called double articulation by Martinei (1960,, who pointed out that, in natural language, characters within a word are hardly related to the corresponding meaning whereas the word is associated. For instance, characters a, e, 1, p are irrelevant rounded, red fruit, but word apple is relevant to it. Thus, the word-level compositionality is expected to be higher than the character- one, motivating CK With the conventional objective, however, emergent languages did not satisfy any of them, and the potential causes for this have not been addressed well in Ueda all (20233).
way, motivating C and 2
3 REDEFINE OBJECTIVE AS ELBO FROM THE GENERATIVE PERSPECTIVE
In this section, we first redefine the objective of signaling games as (beta-)VAE's (Kigmma & Welling ROE; Higgins et all 2017), i.e., the evidence lower bound (ELBO) with KL weighting hyperparameter B. Next, we show several reasons supporting our ELBO-based formulation.
Though somewhat abrupt, let us think of a receiver agent Rş generative model, joint distribu- tion over objects X and messages M:
Riciit (X, == Rx M) (M).
Intuitively, the receiver consists not only of the conventional reconstruction term Ro [M) but also of a 'language model" Pg (M). The optimal sender would be the true posterior Sg (M[ = (M) by Bayes' theorem. Let us assume, however, that the sender can only approximate via a variational posterior S& X). The sender cannot access the true posterior because it is intractable, and the sender cannot directly peer into the receiver's "brain" 0. Now one notices the similarity between the signaling game and (beta-)VAE. Their similarity is illustrated in Figure II and Table II VAE consists of an encoder, decoder, and prior latent distribution.
PIn EC, compositionality is frequently studied (Kottur et all 20T7: Chaabouni el all P020: L & Bowling, 2019 Andreas, 201; Ren eLal 2020). The standard compositionality measure is Topographic Similarity (TopSim, [Biihtto Kirby, 2IKE; Lazaridou all 201S) Let dx be a distance function in an object space and dy in message space. TopSim is defined as the Spearman correlation between dx (x. (2)) and d dm(mm") (1) 1 m²) for all combinations (1)), (2)) E X' (without repetition). Here, cach (9) is the greedy sample from (*) dx and dm are usually defined as Hamming and Levenshtein distance, respectively. 'The latent variable of the standard (beta-)VAF is real vector in Fuclidean space, whereas the latent variable of the signaling game is discrete sequence.
4

---

Published as a conference paper at ICLR 2024
Here, regard M as latent. S& (X) encoder, Ro (x(X)M) as receiver, and Pa (M) as prior distribution. Our objective is now defined as follows:
Jaurs( 0 3) Ea- (mmmSs)))) (log Ro(x mm) / (S x) P (M)),
(7)
where DKL denotes the KI divergence and |3 > 0 is hyperparameter that weights the KI. term. Although Eq. @ equulss to the precise ELBO only 3 = 1, is often set < initially and (optionally)
annealed to prevent posterior collapse (Bowman et al 2016: Alemi et all 2018 En at 2019 Klushyn et al. 2019). In what follows, we discuss reasons supporting our redefinition.
Recipe for Supporting Our ELBO-based Formulation
1.
The conventional objective actually resembles ELBO in some respects.
Section
1
The conventional objective has implicit prior distribution.
Section
The conventional objective is similar beta-VAE's
2. The choice of prior distributions affect the properties of emergent languages.
Section +
Be aware that unnatural prior distributions cause inefficient messages.
Section 4
Use learnable prior distribution instead, richer "language model."
Section
The learnable prior distribution gives rise more meaningful segments.
3. ELBO can be justified from the computational psycholinguistic viewpoint.
Section
ELBO models the trade-off between informativenes: and surprisal.
3.1 CONVENTIONAL ORJECTIVE HAS AN IMPLICIT PRIOR DISTRIBUTION
We show that the conventional signaling game has implicit prior distributions p (M) or parior (M). First, let parie (M) be uniform message distribution and pisim
D.nnir (X, M) be as follows:
pptior unil (m) = [MP
pioint O,unir m) = Ro(x m) p unit (m).
pptior
(8)
Then, the following holds (see Appendix
B. for its proof):
Tcom (Φ) 0) Va.e Exy (.mpl-).mmmmSaaa [log paivint (2, m)],
ie., maximizing Jecm ( 0) via the gradient method is the same maximizing the expectation of log ploint (1, m1. Next, the pength-penalizing. objective Taun (sb 0: a) also has a prior distribution.
(X,M) Let perior (M) and 0.a (x, .)) be as follows:
pprior (m) x exp(-alml), plein (x, (x.m) Ro (xx pn (m).
(10)
Then, the following holds (see Appendix B. for its proof):
V VoonJoom (Φ, oi) 0.0 Px=P)(() ())m)) [log Paça (z, m)],
(11)
ie., maximizing Jeom (4. 0: a) is the same as maximizing the expectation of log pjoint (æ, m). Note ploint
that Eq. is a special case of Eq. III: if O, perior (m)= parior parior(m) (m) and ploint B (x. m)=
0.a (a, m).
3.2 CONVENTIONAL OBJECTIVE IS SIMILAR TO (BETA-)VAE
The conventional objective may be similar to variational inference generative model, because it turned out have implicit prior distributions. Indeed, similar to (beta-)VAF. As Jean (Φ, 0: o:) has the implicit prior pptior (M), one notices the similarity between the signaling game and VAE, regarding M latent, (x) as encoder, (x(XM) decoder. Here, the corresponding ELBO Jetho 0; a:) can be written as follows:
Jatba (1b, 0:a) Ee-Pa))) Emoss(((a) flog Rx m)] DKL. (Sx()))) (M)) (12)
5

---

Published as a conference paper at ICLR 2024
Then, the following holds (see Appendix B-2 for its proof):
V Vebooet (Ø, a) V ae ( a; V o.o Exw Pay (HHS)) x))]
(13)
resembles Vi (Fq.
i.e, maximizing Jelbo ( 0; a+) is the same as maximizing Joom (Φb 0; a)) plus an entrpyy maximizer. Recall that previous work typically adopts entropy regularizer (Eq. B). Though they not math- ematically equal, they are similar enough in that they keep the sender's entropy high to encourage exploration (Levine, 2018). In this sense, the conventional signaling game is similar VAE. Or rather, it is similar to beta-VAy because A adjusts the entropy regularizer in practice, which roughly corresponds to that adjusts the KI. term weight.
3.3 IMPLICIT PRIOR DISTRIBUTION AND INEFFICIENCY OF EMERGENT LANGUAGE
We have thus far seen that the conventional objective has an implicit prior distribution and sim- ilar to beta- VAE. Now we need to elaborate why one should be aware of them. To this end, in this subsection, we suggest that the inefficiency of emergent languages is due to unnatural prior distributions. Chabounn et aD (2019) reported that the objective Joom (Φ. O) did not result in emer- gent languages that follow ZLA, while the length- penalizinn one Jconv (cb, O; a)) did. Recall that Teom (Φ, 0) has the implicit prior distribution p (M) and Joom (eb 0; a) has Parie (M). Then, define the corresponding distributions over message lengths K:
porin (K k) Ement
1 |mm| 1 P (m) exp(rkk),
(14)
Perior (K k) = Σ MEM 1 | Ppeioe (m) x exp((7 ai)k:,
where 7 = log(IA - 1) 0 and (-) is the indicator function. P (k)) (resp. Pa (k)) defines the probability that the length K a message sampled from pattor (M) (resp. patior (M)) k. parier (k) grows up w.r.t k. i.c., longer messages
common, and even the longest messages most likely. It is clearly unnatural as a prior distribution. Regularized by such a prior distribution, emergent languages would obviously become unnaturally long. In contrast, when a > 7, Pare (k) decays w.r.t At i.e., shorter messages are more common. Thus, unnatural inefficiency is arguably due to the implicit, inappropriate prior distribution patior
the
unif (m), while patior (m) may mitigate the problem. This conclusion contrasts with the previous work that has ascribed the issue to inductive biases, as mentioned in Section k Note that an additional interpretation rather than a rebuttal. It is likely to be complicated issue involving both the prior distribution and inductive bias.
3.4 INTRODUCE LEARNABLE PARAMETRIZED PRIOR DISTRIBUTION
In the previous subsection, saw that the choice of prior distributions influence the statistical properties of emergent languages. However, the signaling- game counterpart of the VAE's prior distribution has not been clear yet. In other words, as sub-domain of linguistics, it is important to take clear stance on which part of real-world communication the prior distribution models. This paper considers it as a receiver's 'ss neural language model (LM) parametrized by 0:
(15) It can be seen as an LM because it approximates the average behavior of sender, i.e., the KL term in Eq. M Erg (ar) (Sx((s is minimized w.r.t 0 when (prio(m))
pfrior (m)
II_ ml
prrix (m m1a-1).
(M)) Er (H(S((M) (s (1)mmm)llp [log perirr (m)],
In contrast, parior (m) seems to be pptie( (m)) is still unnatural as simulation of language since it restricts the complexity of the sender's behavior. Our modification leads to more meaningful segments (see Section E!).
[s,,(
oversimplified as an LM; though it is more natural than pirii
3.5 RELATIONSHIP TO SURPRISAL. THEORY
We formulated the game objective as ELBO and defined its prior distribution as a learnable LMM also be seen modeling of surprisal theory (Hale, 2001: Levy, 2008: /Smith & Levy, 2013) in

---

Published as a conference paper at ICLR 2024
computational psycholinguistics. The theory has recently been considered important for evaluating neural language models as cognitive models (Wilcox ef all 2020) In the theory, surprisal refers to the negative logllikelihood of sentences/message - log l (m) = Lo partor (ma and higher surprisals are considered to require greater cognitive processing costs. Intuitively,, the cost for reader/litener/reeeier to predict the next token accumulates over time. In fact, our objective Tours models naturally, as one notices from the following rewrite:
Joans( ( 0; B) Eew Pay(-) [Em= So (-1x) log Ro m) +.3 log Peri (m) +.8 3 H(S [x))] (16)
reconstruction
nogative surprisal
entropy maximizer
The first term is the reconstruction, the second is the (negative) surprisal, and the third is the entropy maximizer. Reconstruction and surprisal can be in trade-off relationship. Conveying more infor-
mation results in messages with a higher cognitive cost, while messages with a lower cognitive cost may not convey as much information.
EXPERIMENT
In this section, we validate the effects of the formulation presented in Section Specifically, we demonstrate improvements in response to the problem raised by [Deda et all (2024) that the conven- tional objective does not give rise to meaningful word segmentation emergent languages. The experimental setup is described in Section [ and the results are presented in Section !
4.1 SETUP
Object and Message Spaces: We define an object space X' as set of att-val objects. Following
Ueda et all (2023), we set (Mam Thyal) E {(2 64), (3, 16). 8), (6, 4), (12, 2),, ensuring all the object space sizes X are (Rvat)" 4096. To define a message space M, we set Timax 32, the same as Ueda et (2023), and |A| = 9, one bigger than Leda et all (2023)) so that eos A.
Agent Architectures and Optimization: The agents Sq. Ro are based on GRU (Cho all 2014), following Chaabouni al (2020); [eda et all (2102)) and have symbol embedding layers. Layer- Norm (Ba all 2016) is applied to the hidden states and embeddings for faster convergence. The agents have linear layers (with biases) to predict the next symbol; Using the symbol prediction layer, the sender generates a message while the receiver computes log penior (m). The sender embeds an object æ the initial hidden state, and the receiver outputs the object from the last hidden state. We apply Gaussian dropout (Srivasiava et all 201E)) to regularize 0, because otherwise it can be un- stable under competing pressures log Re (m m|(r) and log Re (m). We optimize ELBO in which latent variables are discrete sequences, similarly to Mmhh & Gregor (201). By the stochastic computation graph approach (Schulman et all, 2015), we obtain a surrogate objective that is the combination of the standard backpropagation w.r.t the receiver's parameter 0 and REINFORCE (Williams, (1922) w.r.t the sender's parameter <b. Consequently, we obtain similar optimization strategy to the one adopted in Chaahoumi et al (2M)); Deda ctll (2(23). 3 is initially set << 1 to avoid posterior collapse and annealed to via REWO (Klushyn et all 2019). For more detail, see Appendix [
Evaluation: We have three criteria C1. 0+ and [K to measure the meaningfulness of segments obtained by the boundary detection algorithm. The threshold parameter is set to 0. We compare objective with the following basclines: (BL1) the conventional objective Joon plus the entropy regularizer (where xenmeg=_), (BI.2) the El.BO-based objective that is almost the same as ours, except that its prior is perior (where a= log LAD) B We ran each 16 times with distinct random seeds.
'It may also be seen as the trade-off between expresiiiiylinnoomaivven and compressiiityleficcieney the language evolution field (Kirby et arl 2015: Zaslavsky et all (022)), though our ELBO-basce formulation per se is orthogonal to models involving generation change, c.g., itcrated learning (Kirby et all, 2012).
5 'We also investigated whether emergent languages show a ZLA-like tendency when using our objective. They indeed show such a tendency with moderate alphabet sizes. See Appendix IB.
"We exclude (1, 4096) hecause, in preliminary experiments, the learning convergence was much slower. It not big concern as it an exceptional configuration for which TopSim cannot be defined.
We also tried 0.25, 0.5 for the threshold parameter (see Appendix Di
When log [A], lim lim Los to pa (M)) coincides with uniform monkey typing (see Appendix B.

---

Published as a conference paper at ICLR 2024
Figure 2: Results for Thouu (CD), nseg Aw. (C.), C-TopSim (C5), and W-TopSim (C3)
shown in order from the left. The x-axis represents (nau Reya)) while the y-axis represents the val- ues of each metric. The shaded regions and error bars represent the standard error of mean. The threshold parameter is set to O. The blue plots represent the results for our ELBO-based objective Jours orange ones for (اLII) the conventional objective Jenny plus the entropy regularizer, and the grey ones for (BL2)) the ELBO-bbased objective whose prior is porior The apparent inferior per- formance of Aw.c for Tours compared the baselines might be misleading. It because Jours greatly improves both C-TomSim and W-TopSim. The larger scale of their improvements could result in seemingly worse Aw.c. but this does not necessarily indicate poorer performance.
Tlxea
Rgeg
C-TopSim
W-TopSim
25-
I0P
0.2
4.3
63
0.2
4.5
10.0
(Pat-Phaa))
Our ELBO-based objective J...
(BL1) Conventional objective J... entropy reqularizer
(BL2) ELBO-based objective prior Fx...
4.2 EXPERIMENTAL RESULT
Aw.e C-TopSim and W-Toppim (CS) are shown in order from the left. As will be explained below, it can be observed that the meaningfulness of segments in emergent language improves when using our objective Tours-
We show the results in Figure The results for 7boou
(MII)
Tleeg
First, the result for Abou in Figure Q It is evident that the line monotonically increases only for Jours. It means that, as aaa increases, Thou also increases only when using Jouss confirming that Cil is satisfied. Next, see the result for nlag in Figure D. Again, it is evident that the line monotonically decreases only for Jours. That is, as Thaa increases, Raeg also increases only when using Jouss confirming that
Criteria Cl and OZ satisfied with objective Tuurs
satisfied.
Criterion C is not met but TopSim improved: Refer to the results of Aw,c in Figure A Aw,e < regardless of the objective used. Even when using Jours C3 not satisfied. However, observe the results for C-TopSim and W-TopSim in Figure A By using Tours. both C-TopSim and W-TopSim have improved. Notably, the improvement in W-Toppim indicates that the meaningfulness of seg- ments has improved. Aw, does not become positive since C-TopSim has increased even more. We speculate that one reason for Aw, < might be due to the "spelling variation" of segments.
DISCUSSION
In Section E we proposed formulating the signaling game objective as ELBO and discussed support- ing scenarios. particular, we suggested that prior distributions can influence whether emergent languages follow ZLA. In Section HL we demonstrated the improvement of segments' meaningful- ness regarding HAS with a Iearnable prior distribution (neural language model).
Why Did Segments Become More Meaningful? Though it is not straightforward to give clear mathematical basis, we currently hypothesize that the improvement of segments' meaningfulness is due to the competing pressure of the reconstruction and the surprisal. On the one hand, due the reconstruction term, a receiver has to be "surprised," i.e., to receive symbols m, with relatively high Objects contain aan statistically independent components, i.c., attributes. The receiver must be surprised several times x Nau to reconstruct the object from sequential message. On the other hand, due to the surprisal term, the receiver does not want to be "surprised," ie., it prefers relatively low log perior (me (melmaa--)) In natural language, branching entropy (Eq. 5 decreases on average, but there are some spikes indicating boundaries. other words, the next character should often be predictable (less surprising). while boundaries appear at points where
- log pprior (m (m//m/==1))
8

---

Published as a conference paper at ICLR 2024
predictions are hard. The conventional objective Joom does not provide any incentives to make characters "often predictable." whereas the surprisal term in ELBO provides. B
Limitation and Future Work: A populated signaling game (Chaaboum of 2002 Rita et all 20223: Michel et all 2023) that involves multiple senders and receivers is important from a soci-
olinguistic perspective (Raviy all 20T9). Defining reasonable ELBO for such setting is our future work. Perhaps it would be similar to multi-view VAE (Suzuki & Matsua 20Z2). Another re- maining task is to extend our generative perspective discrimination games for which the objective is not reconstructive but contrastive, such as InfoNCE (Man den Oord et all, 2018). Furthermore, al- though we adopted GRU prior message distribution, various neural language models have been proposed as cognitive models in computational (psycho)linguistics (Dyer et all, 2016, ihhe et all, 201% Stanojevic al, 2021: Kuribayashi et all, 2022, inter alia). It should be a key direction to investigate whether such cognitively motivated models give rise to richer structures such as syntax.
RELATED WORK
EC as Representation Learning: Several studies regard EC as representation learning (Andreas, 2019: Chhaaooui et all 2020: all 2022).. For instance, compositionality and disentanglement are often seen as synonymous. TopSim is a representation similarity analysis (RSA, Knegeskorte ala 2008), as van der Wal al (2020) pointed out. Resnick eesmickkttal all (2020) formulated the objective as ELBO. However, they defined messages as fixed-length binaries and agents as non-autoregressive models, which is not applicable to our variable-length setting. Moreover, they do not discuss the prior distribution choice; they seem to have adopted Bernoulli purely for computational convenience. In contrast, indicated that their choice influences the structure of emergent languages." !
VIB: Lucker et al (2(222) defined a communication game called VQ- RIB, based on Variational Information Bottleneck (VIB, Alemi et all 2017) which is known
generalization of beta- -AE. Also, Chaabouni et all (202I) formalized a color naming game with a similar motivation. Note that messages are defined single symbols in their settings. In contrast, messages of variable length in our setting so that we can discuss the structure of emergent languages such as ZLA and HAS.
MH Naming Game: Taniguchi et all
(2 : Inukar (K))5)) and HHong al. (4022) defined
a (recursive) MIl naming game in which communication was formulated as representation learning on a generative model and emerged through MCMC instead of variational inference.
Havrylov & Titoy (2017)) formalized a referential game with pre- trained natural language model as prior distribution. Their qualitative evaluation showed that languages became more compositional. Lazarıdou et all (2020) adopted a similar approach.
Natural I Language as Prior:
Dealing with Discreteness: Though we adopted the REINFORCE-like optimization following the previous EC work (Chaabouni et all 2019; (eda et all, 2025), there are various approaches dealing with discrete variables (Rolfe, 2017: Vahdat 2018::6 at 2017: Maddison et all 2017)).
7 CONCLUSION
In this paper, EC was reinterpreted as representation learning on a generative model. We regarded Lewis's signaling game as beta-VAL and formulated its objective as ELBO. Our main contributions are (1) to show the similarities between the game and (beta-)VAE, (2) to indicate the impact of prior message distribution on the statistical properties of emergent languages, and (3) to present a com- putational psycholinguistic interpretation of emergent communication. Specifically, we addressed the issues of Zipf's law of abbreviation and Harris's articulation scheme, which had not been re- produced well with the conventional objective. Another important point is that we revisited the variable-Iengle message setting, which previous work often avoided. The remaining tasks for future work are formulating populated or discrimination game (while keeping the generative viewpoint) and introducing cognitively motivated language models as prior distributions.
9 Our current hypothesis does not fully explain the reason why the branching entropy does converge a flat one instead of a jittering one. At the moment, we conjecture that, as matter of probability; it is simply difficult to converge to a flat one among infinitely many possible bit patterns (almost all of them would jitter). II Appendix EL Table ( supplementally shows the relationship between previous EC work and this paper.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported by JSPS KAKENHI Grant Numbers JP2IH04904, JP23H04835, and JP23KJ0768. We would like to express our gratitude to the anonymous reviewers for their lively and constructive discussions.
REPRODUCIBILITY STATEMENT
The reproducibility of the experiments in this paper is ensured in the following ways:
The experimental setup is described in Section B. and Appendix E
The source code has been released upon acceptance (See Appendix ()).
REFERENCES
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Operriieeppeeeiie.nneeeee, 2017. URL netps: //oeenzeview.o nwt/ Xoyoom/2me2.c.xxxxcccc...
Alexander A. Alemi, Ben Poole, Ian Fischer, Joshua V. Dillon, Rif A. Saurous, and Kevin Mur- phy. Fixing a broken ELBO. In Jennifer G. Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference on Machine Learning. ICMI. 2018, Stockholmsmüssan, Stockholm, Sweden, July 10- 15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 159- 168. PMLR, 2018. URL
ngs ml prwss/ wB0/a aew/aaammmmmmmm 18 htmm I.
Jacob Andreas. Measuring compositionality in representation learning. In 7th International Confer- ence on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenRe- view.net, 2019. URL
ew net mett:forrum2d-d000l000ll
Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. CoRR, Abs//607.00440, 2016. URL
wtg. abs/ 1607.0000000 1637.06450,
Timothy C. Bell, John G. Cleary, and lan H. Witten. Text Compression. Prentice-Haud Hall, Inc., 1990.
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. Generating sentences from a continuous space. In Yoav Goldberg and Stefan Rie- zler (eds.), Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. CONLL 2016, Berlin, Germany, August 11-12, 2016, pp. 10-21. ACL, 2016. doi: 10.18653/v1/k16- 1002.
Nicolo' Brandizzi. Towards more human-like Al communication: A review of emergent com- munication research. CoRR, abs/2308.0254) 2023. doi: 10-485500/aXxx22238.02444 URL
https::/dol... www.l.oogg/iio 0...445550/11111
0254T]
Henry Brighton and Simon Kirby. Understanding linguistic evolution by visualizing the emergence of topographic mappings. Artif. Life, 12(2):229-22 2222, 2006. URL httts::/ddl..zz/
TT 62/art 2006. 12.2.229
Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, and Marco Baroni. Anti-efficient en- coding in emergent communication. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d' Alché-Buc, Buch Emily B. Fox, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp- 6290-6300, 2019. URL https: http:::://poccedinnss. neurunn...neeuusss.. wctpaaeer2019//aas.
31cauuca (wtllaaa1114ppeS sce www.::///.......eee.e.0.. Abstro0cbbbtttttt html.
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Ba- roni. Compositionality and generalization in emergent languages. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 4427 4442. As- sociation for Computational Linguistics, 2020. URL AtEps: /oo. org/ 10 18653/v1/
12020 acl -ma in 4077
10

---

Published as a conference paper at ICLR 2024
Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, and Marco Baroni. Communicat- ing artificial neural networks develop efficient color naming systems. Proceedings of the Na- tional Academy of Sciences, 118(12):220165661111 2021. doi: www.107//pna.201666600
URL
httpp:: www pnas wttp://////oooooo/oo/oo/oo//o abs www.aa...11.... 10tp////00003300........a 201 6569118 18.
Rahma Chaaboumi, Florian Strub, Florent Altché, Eugene Tarassov, Corentin Tallec, Elnaz Davoodi, Kory Wallace Mathewson, Olivier Tieleman, Angeliki Lazaridou, and Bilal Piot. Emergent com- munication at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL htt.ttttt///peeeeeettee
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, Special Interest Group of the ACL, Pp. 1724 1734. ACL, 2014. URL hEtps://Gol .re.l.rrgl1e 311.51/v11 11--119
Edward Choi, Angeliki Lazaridou, and Nando de Freitas. Compositional obverter communication learning from raw visual input. In 6th International Conference on Learning Representations, ICLR 2018 Vancouver. BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https:/ openreview
/ net Iet//orrmmidddknnttt
Brian Conrad and Michael Mitzenmacher. Power laws for monkeys typing randomly: the case of
unequal probabilities. IEEE Transactions on Information Theory, 50(7):1403-1414, 2004. doi:
00.1109//T.22004833775
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecom- munications and Signal Processing)- Wiley-Interscienne,, 2006.
Roberto Dessi, Eugene Kharitonov, and Marco Baroni. Interpretable agent communication from scratch (with generic visual processor emerging on the side). In Mare' Aurelio Ran- zato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neu- ral Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual,
pp. 26937-26949, 2021. URL https:: www.oeeeiiinnss..uiss..... wwwrips..pcccmm wttpcc/pppppp221111
hast/e220cc933s55eeee Satell114455bbbc 1322220055550eeee444aaaaaaaCrrr3
htmll
Chris Dyer, Adhigunaa Kuncoro, Miguel Ballesteros, and Noah A. Smith. Recurrent neural network grammars. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguis- ties: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pp. 199- 209. The Association for Computational Linguistics. 2016. doi: 10.18653/v1/116- 1024. URL https::/ddi www.oogg...............o 118653v1lmml 10244
Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, and Shimon Whiteson. Learn ing to communicate with deep multi-agent reinforcement learning. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neu- ral Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp-
2137-2145, 2016. URL https: http:::/proceedee https:://roceeinss.r neurips........................i pc/ppapet//11//aa.tt C 276.011114442222411.......
Abstract htmll.
Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. Cycli- cal annealing schedule: A simple approach to mitigating KL vanishing. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLI 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers), pp. 250-200002000 Association for Computational Linguistics, 2019. doi: 10. www./66//11/1111..1 1021. URL https://dob .ww..oorgoo.. 00. 1et.1111533mmm19/
11

---

Published as a conference paper at ICLR 2024
Yarin Gal and Zoubin Ghahramani. A theoretically grounded application of dropout in recurrent neu- ral networks. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Ro- man Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp-
1019-1027, 2016. URL httpp:::///rcceeeep iageeeiiigg.... weureeursps.. wtt.cc/pppaae htc//ppprr//21//hasss 1165llc97aa9aaalllleeee
4c FF2529f2 Abst ract ml
Lukas Galke, Yoav Ram, and Limor Raviv. Emergent communication for understanding human language evolution: What's missing? CoRR, abs/2204.10599, 2022. doi: 10.48550/arXiv22244 10590. URL A
John Hale. A probabilistic earley parser as psycholinguistic model. In Language Technologies 2001: The Second Meeting of the North American Chapter of the Association for Computational Linguistics, NAACL 2001, Pittsburgh, PA, USA, June 2-7, 2001. The Association for Computa- tional Linguistics, 2001. URL https://aclamss wtt.a/thh.oooooooo//.o.... 1UZ17
Zellig S. Harris. From phoneme to morpheme. Language, 31(2):190-222 1955. URL http:
7 7/www. o
Serhii Havrylov and Ivan Titov. Emergence of language with multi-agent games: Learn ing to communicate with sequences of symbols. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neu- ral Information Processing Systems 2017, December 4-9, 2017. Long Beach, CA, USA, pp. 2149 2159, 2017. URL htEDs: httpp//twceccncccce/oppp neuz www.uipps.c...o... wttppss.c....//..///// ///hhh/ TD22229 www.://wwcccooaaaaaaaa.
ract htmll
Irina Higgins, Loic Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual con- cepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL /ttps::/openzeveee www.:eeooooo11d-//ea.....
Nguyen Le Hoang, Tadahiro Taniguchi, Yoshinobu Hagiwara, and Akira Taniguchi. Emergent com- munication of multimodal deep generative models based metropollishhstinns naming game. Frontiers in Robotics and AI, 10, 2024. ISSN 2296-9144. doi: 2ttp URL 1290664
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., 98): 1780, 1997. http:://ww...oooo....eoooooratta
https: www. www.:fooonttsoooooo wttp:siinnnggrrtrtitooooooo
Jun Inukai, Tadahiro Taniguchi, Akira Taniguchi, and Yoshinobu Hagiwara. Recursive metropolis- hastings naming game: Symbol emergence in a multi-agent system based on probabilistic generative models. CoRR, abs/2305.19761, 2023. doi: 10.48550kaXi..2335.19766 URL
Eric Jang, Shixiang Gu, and Ben Poole. Categariialepppataeeeettizaaaaa reparameterizzaioon with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. htt.ps://oppeiie (eerummmd=rkE3y885ee
Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Çaglar Gülçchrre, Pedro A. Ortega, DJ Strouse, Joel Z. Leibo, and Nando de Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9- 15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 3040-3049. PMLR, 2019. URL htt.:ttp../// htte:://proeediime igg m pr Jaques html
Eugene Kharitonov, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni. Entropy mini- mization in emergent languages. In Proceedings of the 37th International Conference on Ma- chine Learning. ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Ma- chine Learning Research, pp. 5220-52200222 5230. PMLR, 2020. URL http:://proceddnggs.ttt mml. 1esss/v11//knne tonov htm
12

---

Published as a conference paper at ICLR 2024
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. 2015. URL http: //arxiv. org./b...r//baaao.o 412.4.226 6980.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http: /ttps//aaxvvva www.iiioorr//aaoo. abgg/bbs/1ag 6114
Simon Kirby, Tom Griffiths, and Kenny Smith. Iterated learning and the evolution of language. Cur- rent Opinion in Neurobiology, 28: 18:1081114, 2014. ISSN 0959-4388. doi: http:://oii.or//10.000 sci encedi rect. com cww.//ceence/arcccl....... p11 00/5809993338840122222 SI: Communication and language.
J.comb.2014.0..045 URL bttbs: htos:///ww// /cd
Simon Kirby, Monica Tamariz, Hannah Cornish, and Kenny Smith. Compression and com- munication in the cultural evolution of linguistic structure. Cognition, 141:87-102, 2015. ISSN 0010-0277. doi:
URL https://wir
Alexej Klushyn, Nutan Chen, Richard Kurle, Botond Cseke, and Patrick van der Smagt. Learn- ing hicrarchical priors in vaes. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 2866-2875, 2019. URL pttps::/rrcceeennsse.ee 1612266633
wtwp:eipp.cccpppppeemmm wtt.s/aaa////11////s.... a35 Abst htmll.
Satwik Kottur, José M. F. Moura, Stefan Lee, and Dhruv Batra. Natural language does not emerge 'naturally' in agett dialog. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pp. 2962 2967. Association for Computational Linguistics, 2017. URL https: / /t0.::://ooi..r/11....1 1100///...11//1/1111111
Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandetrini. Representational similarity analysis connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 2008. doi: 10.3389/neuro. 06.004.2008,
ISSN 1662-5137.
URL hetps: www. Front
Drc/articles/ 3389/ (339//eero... neuro. 06. 004 2008.
Tatsuki Kuribayashi, Yohei Oseki, Ana Brassard, and Kentaro Inui. Context limitations make neu- ral language models more human-like. In Yoav Goldherg, Zornissa Kozareva, and Yue Zhang (eds.). Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 10421-10436. Association for Computational Linguistics, 2022. doi: 10. 11/166//w1//2222mmmmmmmaaa 712. URL https:///ol.oo3/11..
emn www.oommaa....mmmmmmmmmmmmmmmmmm 712.
Angeliki Lazaridou and Marco Baroni. Emergent multi- communiiiionoii in the deep learning era. CORR, abs/2006.0241 2020. URL htll://...///aav.o/////bb/////// 02419.
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. In 5th International Conference on Learning Representations, ICIR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedingy. OpenReview.net, 2017. URL
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of lin- guistic communication from referential games with symbolic and pixel input. In 6th Inter- national Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL nEEPS: //openrevi ew. net./ vet.ffor ftr.m/ormmmmmmm - All.
13

---

Published as a conference paper at ICLR 2024
Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman. Multi-agent communication meets natural language: Synergies between functional and structural language learning. In Dan Ju- rafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, ACI. 2020, Online, July 5- 10, 2020, pp. 7674-7674. Association for Computational Linguistics, 2020. URL https: 07.00.
main. 683.
Sergey Levine. Reinforcement learning and control probabilisttic inference: Tutorial and review. CoRR, abs/1 $00,,0009, 2018.
Roger Levy. Expectation-base syntactic comprehension. Cognition, 106(3): (133::126-11111 117771777 2008.
ISSN 0010-0277. doi: http:///dduoggg////00ffoorrrroop URL https: www.
SCi icceededinnn rect. com sc ltt.eeeceerrrrcceeeeieiic/e////e
David K. Lewis. Convention: Philosophical Study. Wiley-Blackwell, 1969.
Fushan Li and Michael Bowling. Ease-of-teaching and language structure from emergent communication. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché-Bue, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Informa- tion Processing Systems 32: Annual Conference Neural Information Processing Sys- tems 2019, NeurIPS 2019. December 8-14, 2019, Vancouver, BC, Canada, pp. 15825- 15835, 2019. URL https: http:://ppocceeeeeee neur ips.rpps... ptcc/paper/201/////
blic
htmll
Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor- critic for mixed coperaatie-coomptiiiie environments. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neu- ral Information Processing Systems 2017, December 4.9, 2017, Long Beach, CA, USA, pp- 6379 6390, 2017. URL https: http:///ccceeee....oo neuz wtsauuipss.ccc. wttpp//ppppee//////////pe 2017hash// 6Ka47550 www.haa4HHaabllllaa 991a ohh4- Abst html.
Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relax- ation of discrete random variables. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Truck Proceedings. OpenReview.net, 2017. URL
net wtt.///oummmidioo. 155L5g
André Martinet. Éléments de linguistique générale. Armand Colin, 1960.
Paul Michel, Mathieu Rita, Kory Wallace Mathewson, Olivier Tieleman, and Angeliki Lazaridou. Revisiting populations in multi-agent communication. In The Eleventh International Confer- ence on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https:// wwtp:/ttt//pffffe wwtppdfffddd-u-d -HREN-URRIPPP
George A. Miller. Some effects of intermittent silenie The American Journal of Psychology, 70(2): 311-314, 1957. URL http:::/www..sttt wttp://rtt//tttete/333//3
Andriy Mnih and Karol Gregor. Neural variational inference and learning in belief networks. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014, volume of JMLR Workshop and Conference Proceedings, pp. 1791- 1799. JMLR.org. 2014. URL http: httt:::/proceed.ee wgw.eeddiigg.... mww. www.s//w22222. mn www.hi.hh.....m.m htmll
Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pp. 1928-1937. JMLR.org, 2016. URL http:://rocceeeee V48
.mlr. wwt.presss/
Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations. In Sheila A. Mcllraith and Kilian Q. Weinberger (eds.), Proceedings of the Thirty- Second AAAI Conference Artificial Intelligence, (AAAI- the 30th innovative Applications of
14

---

Published as a conference paper at ICLR 2024
Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium Educational Advances in Artifi- cial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp. 450--152.
AAAI Press, 2018. URL https:/ www. aaai. .rg. wctp//oc//aooo index. phpp AAAI/ /AAAT18
paper/view 17007L
Limor Raviv, Antje Meyer, and Shiri Lev-Ari. Larger communities create more systematic lan- guages. Proceedings of the Royal Society B: Biological Sciences, 286(1907):20111222, 2019.
doi: 10.1098/nsp..209.122.. 126.. URL
abs/ 10.1099/ 2019. 1262
Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B. Cohen, and Simon Kirby. Compositional lan- guages emerge in a neural iterated learning model. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https::/ net/ test/forre HookkeNkk
Cinjon Resnick, Abhinav Gupta, Jakob N. Foerster, Andrew M. Dai, and Kyunghyun Cho. Capacity, bandwidth, and compositionality in emergent language learning. In Amal EI Fallah Seghrouchni, Gita Sukthankar, Bo An, and Neil Yorke-Smith (eds.), Proceedings of the 19th International Con- ference Autonomous Agents and Multiagent Systems, AAMAS '20, Auckland, New Zealand, May 9-13, 2020, pp. 122-1133. International Foundation for Autonomous Agents and Multia- doi: N0.555/339966113339992 URL https://dd acm. orq/do1
gent Systems, 2020.
761 3398892
Ryokan Ri, Ryo Ueda, and Jason Naradowsky. Emergent communication with attention. CoRR, abs/2305.10920, 2023.
doi: 10.48500arXi..2355.09999 URL https: org/I0. 48550/arXiv 20....3359.099 10920.
Mathicu Rita, Rahma Chaabouni, and Emmanuel Dupoux. "lazimpa": Lazy and impatient neural agents learn to communicate efficiently. In Raquel Fernández and Tal Linzen (eds.), Proceedings of the 24th Conference on Computational Natural Language Learning. CoNLL 2020, Online, November 19-20, 2020, pp. 335-343, Association for Computational Linguistics, 2020. URL
Mathicu Rita, Florian Strub, lean-Basticn Grill, Olivier Pietquin, and Emmanuel Dupoux. On the role of population heterogeneity in emergent communication. In The Tenth International Confer- ence on Learning Representattions, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022a. URL TtEtps::/openreeeee
net Sot
Mathieu Rita, Corentin Tallec, Paul Michel, Jean-Bastien Grill, Olivier Pietquin, Emmanuel Dupoux, and Florian Strub. Emergent communication: Generalization and overfitting in lewis
games. In NeurlPS, 2022b. URL http:: nips CC/ wwtppppr.ppppccppeeeeee.. 51 wwlls/pppppp
l
Jason Tyler Rolfe. Discrete variational autoencoders. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL /tttps://openeeeee
John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel. Gradient esti- mation using stochastic computation graphs. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Pro- cessing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp. 3528- 3536, 2015. URL https: /tw.:s:://roceeiinns..eeee. http://cc....................... pc/pppps//211///aee.
Abstract htmll
Yikang Shen, Shawn Tan, Alessandro Sordoni, and Aaron C. Courville. Ordered neurons: Integrat- ing tree structures into recurrent neural networks. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
.ne. / forum URL https::/ www.e/ww.etttttttttte Bd 6qiR5F7
B11
Brian Skyrms. Signals: Evolution, Learning, and Information. Oxford University Press, Oxford, GB, 2010.
15

---

Published as a conference paper at ICLR 2024
Nathaniel J. Smith and Roger Levy. The effect of word predictability reading time is logarithmic. Cognition, 128(3):302-311 319, 2013. ISSN 0010-0277. doi: http://cc...ooeeeeeeeeerrraaaa/a
2013.02.013. URL https:: cttp://www.llleeeee...ccoooooooc
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1): 1929 1958, 2014. doi: 10.5555/2627733.2676663 URL https://al, orm
5555/26274 2670315
Milos Stanojevic, Shohini Bhattasali, Donald Dunagan, Luca Campanelli, Mark Steedman, Jonathan Brennan, and John T. Hale. Modeling incremental language comprehension in the brain with combinatory categorial grammar. In Emmanuele Chersoni, Nora Hollenstein, Cassandra Jacobs, Yohei Oseki, Laurent Prévot, and Enrico Santus (eds.), Proceedings of the Workshop on Cog- nilive Modeling and Compulational Linguistics, CMCL 2021, Online, June 10, 2021, pp. 23- doi: 10. 10.18653/w1/2011..o .mmll- URL
38. Association for Computational Linguistics, 2021.
/ttps://doi.org//100 1ttp///....133////22000000
cmcl-1.3.
Masahiro Suzuki and Yutaka Matsuo. A survey of multimodal deep generative models. Adv
Robotics, 36(5-6):261-27,, 2022. doi: 10. 10100000611 19106669222222223222222 URL https:
1/do1. 10t00000001000110000100100110100
2035253
Kumiko Tanaka-Ishi. Entropy as an indicator of context boundaries: An experiment using web search engine. Robert Dale, Kam-Fai Wong, Jian Su, and Oi Yee Kwong (eds.), Natural Language Processing IJCNLP 2005, Second International Joint Conference, Jeju Island, Korea, October 11-13, 2005, Proceedings, volume 3651 of Lecture Notes in Computer Science, pp. 93- 105. Springer, 2005. URL https:// org 100 007//1162211
Kumiko Tanaka-Ishii. Articulation of Elements, pp. 115-124 Springer International Publishing, Cham, 2021. URL https::/dol. http://o...rrr//// 100011///-----0003337777
Kumiko Tanaka-Ishi and Yuichiro Ishii. Multilingual phrase-based concordance generation in real-time. Inf. Retr, 10(3):275-295, 2007. URL
S10791- 006 9.
Tadahiro Taniguchi, Yuto Yoshida, Akira Taniguchi, and Yoshinobu Hagiwara. Emergent com- munication through metropolis-hastings naming game with deep generative models. CoRR, Abs/2205.113922 2022.
doi: 10.485500axX...2255.2333 48550/arXiv 12392.
URL :ttps://dof. /do..ozg/1...
Mycal Tucker, Roger Levy, Julie A. Shah, and Noga Zaslavsky, Trading off util- ity. informativeness, and complexity emergent communication. In NeurIPS, 2022.
URL
fi 2tt.ee//pprr//////mmaa
htmll
Ryo Ueda and Koki Washio. On the relationship between zipf's law of abbreviation and interfering noise in emergent languages. In Proceedings of the ACL-IJCNLP 2021 Student Research Work- shop, ACL 2021, Online, JUli 5-10, 2021, pp. 60-70. Association for Computational Linguistics. 2021. doi: 10. 10.18653/w/2021..om lacl- URL https: https://oo...rgg.......... acl-srw.2
Ryo Ueda, Taiga Ishii, and Yusuke Miyao. On the word boundaries of emergent languages based on harris's articulation scheme. In The Eleventh International Conference Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL
net het/pdE71d=bb- XASt
Arash Vahdat, Evgeny Andriyash, and William G. Macready. Dvae#: Discrete variational autoencoders with relaxed boltzmann priors. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Anmual Conference on Neural Information Processing Systems 2018, NeurIPS 2018. December 3-8, 2018, Montréal, Canada, pp. 1869-1878, 1878, 2018a. URL http:///....ooeee......aa/...//. htt.://s...///aeee/////////s EE
htmll
16

---

Published as a conference paper at ICLR 2024
Arash Vahdat, William G. Macready, Zhengbing Bian, Amir Khoshaman, and Evgeny Andriyash.
DVAE++: discrete variational autoencoders with overlapping transformations. In Jennifer G. Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference Learning, ICML. 2018, Stockholmsmüsssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 5042-5051. PMLR, 2018b. URL http:/7 htm.
Machine
proceedings wtt.a//s........eee/////////
Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. CoRR, abs/1 807 007.03748, 2018. URL httt:://arxiv. wwwp:/oogggaaaaaaa abs/ aa..ab/0..1774444.
Oskar van der Wal, Silvan Boer, Elia Bruni, and Dieuwke Hupkes. The grammar of emergent languages. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020. Online, November 16-20, 2020, pp. 3339 3359. Association for Computational Linguistics, 2020. URL 1tt.:/00...6555/22000000
emn I p-ma mm-mai..270)
Ethan Wilcox, Jon Gauthier, Jennifer Hu, Peng Qian, and Roger Levy. On the predictive power of neural language models for human real-time comprehension behavior. In Stephanie Denison, Michael Mack, Yang Xu, and Blair C. Armstrong (eds.), Proceedings of the 42th Annual Meeting of the Cognitive Science Society- Developing a Mind: Learning in Humans, Animals, and Ma- chines, CogSci 2020, virtual, July 29 August 1, 2020. cognitivescennessxiieeyrggg 2020. URL
prpoosci.mindmosss.. mindmooe ing. http://///////////////////////11
Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Mach. Learn., 8:229-256, 1992. URL ftttps://@el.orr/110..
BFC0442648
Ronald J. Williams and Jing Peng. Function optimization using romneceennisriinfrrreeenetd learning
reinforcement
algorithms. Connection Science, 3:241-268. 1991.
Zhenlin Xu, Mare Niethammer, and Colin Raffel. Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language. In NeurIPS, 2022. tttp:://baaerr...ps..coo
Noga Zaslavsky, Karee Garvin, Charles Kemp, Naftali Tishby, and Terry Regier. The evolution of
color naming reflects pressure for efficiency: Evidence from the recent past. Journal of Language Evolution, 7(2):184-199, 04 2022. ISSN 2058-458X. doi: 10. 101/////////....011.. URL netes:
//i... 10010.00/1001133//11330
George K. Zipf. The psycho-biology of language. Houghton Mifflin, 1935.
George K. Zipf. Human Behaviour and the Principle of Least Effort. Addison-Weshe Wesley, 1949.
17

---

Published as a conference paper at ICLR 2024
A HAS-BASED BOUNDARY DETECTION ALGORITHM
Algorithm Boundary Delection Algorithm
1:
- 0; w 4 1; B + 0
2:
while < do
3: Compute BE($:++w-1
4: BE $1:++--1) BE ( s threshold & w then
5:
B < B + w
6:
end
7:
i < 1 then
8:
w t w w+1
9:
else
10:
1 w
11: end if
12:
end while
We show the HAS-based boundary detection algorithm (Lamakn-IshT. 2015: Tanala- Ish Ishm 2TMM in Algorithm M1 Note that this pseudo code is simplified version by Ueda ct al (2023)) B is a set boundary positions. threshold is hyperparameter.
18

---

Published as a conference paper at ICLR 2024
B PROOFS ON THE EXISTENCE OF IMPLICIT PRIORS
B.1 PROOF OF EQ. 9 AND EQ.
Remark Since Par (m a)) x exp(-a/ml) by definition,
1
pptior (m a) exp(-a/ml),
Zo
where Za is normalizer to ensure that perior as follows:
P (M a) is as distribution. Za can be written
Za Σ m exp( -a/mm))
Lmn
exp(-a) a)
1=1 {(1A) - 1) cxp(-aa)
(There are (IAA1 1)(-1 messages of length 1.)
1 {(1AA] - 1) exp(
exp(p
(1.4) exp(eeppppop -a)
1- 1- 1) exp( ) Li
exp(a) - |A| +
Lemma 1. The following equation holds:
V..0 Bar (a
fo(x, m)m v. log Sp(m xx],
where fo :0:& X' M + an any function differentiable w.nt
Proof of Lemma
Let supp P) be the support of given probability measure P. For the notational convenience, define M2 == ( ([))). Then,
V. Ea- (fe
V.. Σ Σ
(a)( I
xEX MEM!
Σ
Σ (orj(z) (5 V)) V x) f)fo()) m))
Vo
mE.!
V 5 Sa(2,))
Σ
Σ (ani(x) (5 V S 1x)
SΦ (m (x)
ZEX mE.y
Σ
Σ (a)(()) (S x) 2.) Sm () 0.0 log So(m |x) (f),))
ZEX mEM!
Σ
(x)( x) (V VV.ooo(z, m) fo(z, 0. log So x))
ZEX mE.,
(a (1)) Vo.efo(z, + fe m) V.. log (m x)).
Remark As Chaaboun et all (2019) pointed out, Lemma can be seen as a special case of the stochastic computation graph approach by Schulman et all (2013).
Lemma 2. The following equation holds:
((x-)es(),SS((())(((())))) 0.0 log Sp(m [x)] 0,
where g X - R is an any function. The function is called baseline in the literature of reinforce- ment learning.
Proof of Lemma
Let supp( P) be the support of given probability measure P. For the notational convenience,
19

---

Published as a conference paper at ICLR 2024
define M³ s (M (M)).) Then,
Ex (M-mattlmmaat s. log (p x)]
Σ (z)g(() meM,
Σ
(a(m/x) 0. log SΦ (m x))
xE
V0.0$ (m x)
Σ (z)g(()
Σ
Sa(m|x)
So x))
#E
MEM,
Σ
Pay (z)g(()
Σ
Vo x))
meM:
Σ TE
(a)(T))(1)VV 0.
Σ mEM,
Sm(m (x)
Paj(t)g(z)
=0
Proof of Eq. and Eq. III As parie (M) is special case of Pa a), it is sufficient to prove Eq.
Recall that what we have to prove, i.e., Eq. II is follows:
V Joon a:))= = 0.0
Dlog Pa (I, m)],
where
1
ppeio (m a)) Za
exp( exp(-a/ml))
ploial (æ,, m.) Ro (x m)) perior Perio
Here, let us define T' (for convenience) as:
:= Ee V.. log Rm(x/m) + (log R& (a|mm
x)].
what follows, we first prove the left-hand side Eq. I is equal to T. i.c. V V.oemcom (Ø, a) T. We next prove that the right-hand side of Eq. I is equal to T, i.e.,
(log Plein (I, (I.m)) m)] = We finally conclude that V Tcon (Φ) 0: = Pa.e (. (- [oog plnint (z, m)], and thus Eq. ID holds.
0.0
On the one hand, the left-hand side of Eq. I can be transformed as follows:
V w.m (8,0:a)
= Ve.0 En Pay (m)) flog Ra(x!m) - a. m/]]
(By the definition of Toom (b, 0; a))
6.0 log Ro(x a oolomml (log Ro (z (o m) - ( (m)())
(From Lemma II defining for m) = log Rep m) a/ml)
V log Ro(x/m) + (log Rø(x|m) a/ml) V S. x)] P
(because Vololm| 0)
= =T.
On the other hand, the right-hand side of Eq. [ can be transformed as follows:
V 0.0 Eew Par (1))) Dlog
ploint 0,a (a, m ax)]
Vo.0 Ex m llog Ro(x m) + log parior (m [:]]
(By the definition of Jamn (#, 0 a))
= =Be-Pan(),mssllie
V 0.0 Ro (2) + V logior (mla)+
Parior (mla)) logSS
20

---

Published as a conference paper at ICLR 2024
(From Lemma M. defining f0 m)) == log Ro(m x) log Pa (m a))
-er v. log Ro m) + {log Ro (x m) - a/ml log Za JV 2. log Spm x]]
(Because V log Pa (m (x) 0 and log Pa (m a)) -armm log Za)
(e) V log Ro m) + flog Ro m)) a/ml] V Φ.0 log Sp(m x))
(From Lemma M defining g(x) = log 7.a)
=T
Therefore, Eq. I holds. For Eq. E. consider the special case, i.e., 0.
B.2 PROOF EQ.
Recall that Jelba (Φ) 0: (x) is defined as:
Jelbold, 0: at) == Ex (m u) [log Ro( m]] (M))) Parior (M))).
Also, recall that what we have to prove, i... Eq. I3 is as follows:
Jebooa (eb a) V. @b, 0: a) Vo.0 Ex (HL( =x)]]
Proof of Eq. IR
First, Jolbo Φ, 0; ar) can be transformed as follows:
[d, a)
= Eay (-) (1 flog (x m)] (KL(S Pario (M [a)]]
(By the definition of Jetho (Φ) 0 a:)
Ro(x (2 m)] Rm (+1x) flog Smm x)) log pprior (m a)]]
=
(By the definition of DKL)
flo (x m) + log pprior (m)] - -Ez S. (+aa) [log Sp(m x)]
(By the linearity of E)
Pa [o(x)) m) log porior (m)) EEnPa ( (M x))]
(By the definition H)
-Ba-Pa(),m-s((ial Dog Ro( m) a/ml] + Eaw ( (M =)
log Z..
(By the definition of patrrr)
(d. ())) 1 Eew (H(S x))] (H
log Za-
(By the definition of Jeon (Ø, 0 a:)
Thus,
Telhoo (Ø, a)
V Vo.oJoom 1 0: V Eaw (H [x)]] V 0. log Za
Vo.oJro 03)00)) + Vo.0 Eew (H( x)))-
()
21

---

Published as a conference paper at ICLR 2024
B.3 CURIOUS CASE ON LENGTH PENALTY
Remark 3 (Curious Case on Length Penalty). When a > 7, Perie (M) converges to monkey typing (M), unigram model that generates a sequence until emitting eos:
model P
monkey
lim
pprior (m)
-iitio monkey (m),
Lina 100
(17)
ppeior
(me m1:--1)
pptior monkey
(m+)
1 exp(-a+ exp(-ai
(m, eos)
monkey
(m, + eos)
where = log( 1) 0. Intriguingly, it gives another interpretation of the length penalty. Though originally adopted for modeling sender's laziness, the length penalty corresponds to the implicit prior distribution Paru (m). whose special case is monkey typing that totally ignores a context 11 The length penalty may just regularize the policy length-insensitive policy rather than a length-sensitive, lazy one.
Proof of remark & On the one hand, for any message mm _,, a monkey typing model can be transformed as follows:
pprior monkey
(2)
ml
pprior monkey
(at (1:1-1)
(By definition)
mm|-1
moakey (eos 01:/m||11 t=1
pridee (a: (1:111
(Only the last symbol is eos)
(1 - (IA) - expeexx()) exp(xxx(pm)))) 1))
(By definition)
((exp(a) - |A| + 1) exp((x(mm)
On the other hand, from remark MI we have:
1 [(IA) - 1) exp( am
lim Zo Iran +000
lim
Ian +000
exp(a) - |A| +1
exp(a) [A| +1
when a log([-A) 1 1). Therefore, we have:
pprior (m (a) = lim exp(-a/ml) (exp(a) - -|4| + 1)xx(((amml)
pprice monkey (m)
Lanax 720
Lina +00
Zo
when a > log(|A| - 1).
Remark
In fact, it has been shown (Miller, 1957 Conrad & Mitzenmachei, 2004) that even the monkey typing model perior monkey (m) follows ZLA.
22

---

Published as a conference paper at ICLR 2024
C SUPPLEMENTAL INFORMATION ON EXPERIMENTAL SETUP
Sender Architecture: The sender S6 is based on GRU (C Cho at 2UE) with hidden states of size 512 and has embedding layers converting symbols to 32-dimensional vectors. In addition. LayerNorm (Ba el all 2016) is applied to the hidden states and embeddings for faster convergence. For t 2 0,
Pnos
0)
Embedding (m:; @.emb)
(t > 0)
LN 0,1
LayerNorm(e:),
(18)
Encoder(x:; @o-emb)
(t=0)
h
( @gm)
(t>
LN LayerNorm(h:)).
where DEes. Dy-emb: Po-emb, Pgu Φ. The object encoder is represented the additive composition of the embedding layers of attributes. The sender So also has the linear layer (with bias) to predict the next symbol. Fort 2 0,
Sp(A++1 m1a) Softmax(Wshhee bs),
(19)
where Ws. bs Φ.
Receiver Architecture: The receiver has a similar architecture to the sender. The receiver Rg is based on GRU etll (0T))) with hidden states of size 512 and has embedding layers converting symbols to 12-dimensional vectors. addition, LayerNorm (Ba et all 2016) and Gaussian dropout (Srivastava all 2014) applied to the hidden states and embeddings. The dropout scale set to 0.001. For 2 0,
0 Boos
(t = 0)
60,1
Embedding (m;; Osemm))
(t > 0)
LN 0, LayerNorm(e d.).
(20)
O G N. (g))) gm))
(1= 0) (t> 0)
he.r
0 LayerNorm(h, O dr),
where Opons Osemb, Oo-emb: gm E 0, O Hadamard product, and de, d, are dropout masks. Note that the dropout masks are fixed over time 1, following Gal & Ghahramani (2(16). The receiver Ro also has the lincar layer (with bias) to predict the next symbol. For 2
(A m1a)
Softmax W Wghh bR):
(21)
where Wn. dn 6.
Optimization: We have to optimize ELBO in which latent variables are discrete sequences, simi- larly to Maih & Gregor (201E). By the stochastic computation approach (Schulman el all 2015), we obtain the following surrogate objective:"
Σ [m]
I Jsurrogate (Φ, 0) == log Re( /
m) 3
t=1 log pprior (me m1a-1)
|ml
StopGrad(Co.e: Be(r, m1s-1) bmsan) log Sø(m; æ, mi:-1);
(22)
where StopGrad(()) is the stop-gradient operator, Be(x, m1a-1) is state-dependent baseline parametrized by Ç. bmean is inutteinddeeedeen)) mean baseline, and
Co. = log Ro m) + Σ |ml
(log Pr
(mu miu-1) log So m1s-1)).
n=t
The architecture of the state-dependent baseline (ę(œ, m1a-1) 1) is defined as:
Be( m1:-1)) (h ur wwg
+ bB.1) + b
We mean by "surrogate" that V Tou Exr Pay me & Vo.0.. Taungnnn i.c., Monte Carlo approx- imation of Touo is easily performed by using Jsurogate.
23

---

Published as a conference paper at ICLR 2024
where WB, E 2256x512 1 optimize the baseline's 's parameter Ç by simultaneously minimizing the sum of square errors:
R wB2 R226, bB, R are learnable parameters in Ç. We
Em_ Tbsseline (C) = Σ Iml (Co.0. E
(23)
Note that optimize the sender's parameter & according Isurrggate but not Tasselnne so not to break the original problem setting of ELBO maximization w.r.t Φ, 0. In addition, to further reduce the variance, we use the indeeeindeenndntt) mean baseline mmean which the batch mean of Co.o. [I, m1:-11). We use Adam (Kingma & Ba 2015) as an optimizer. The learning rate is set to 10 The batch size is set to 8192. The parameters are updated 20000 times for each run. To avoid posterior collapse.
initially set < and annealed to I via the REWO algorithm (Klushyn et all 2019). We set the constraint parameter of REWO to 0.3, which intuitively means that it tries to bring /3 closer to while trying to keep the exponential moving average of the reconstruction error log Ro m) less than 0.3. I
I2 the conventional setting, Rilactal (202l) adopted the ad-hoc annealing of the length-penalty cnefficient a. It might roughly correspond 9-annealing in VAF.
Our code is available https: Enub. com ww..c/mmmmmmmm emecom. gigna n Jame
betavar
24

---

Aw. (C3), C-TopSim (C.), and W-TopSim (C3) are shown in order from the left. While Figure D in the main content shows the result for threshold Published as a conference paper at ICLR 2024
D SUPPLEMENTAL INFORMATION ON EXPERIMENTAL RESULT
w.F (KK)) C-TopSim (C3). and W-TopSim (C.) are shown in order from the left. The x-axis represents (Natt: Teval) while the y-axis represents the val- ues of each metric. The shaded regions and error bars represent the standard enror of mean. The threshold parameter is set to 0.25. The blue plots represent the results for our ELBO-based objec- tive Tours, the orange ones for (BLI)) the conventional objective Jeany plus the entropy regularizer, and the grey ones for (BL22) the ELBO-based objective whose prior is Pario The apparent inferior performance of Aw,c for Tours compared to the baselines might be misleading. It is because Jours greatly improves both C-TomSim and W-TopSim. The larger scale of their improvements could result in seemingly worse Aw,c, but this does not necessarily indicate poorer performance.
Figure 3: Results for Thou (C Tse
Tibou
A
C-TopSim
W-TopSim
5.7
14.
12
4.2
63
0.5
3.3
20
4.3
631
6.3
3.4
4.5
61
64
4.8
12.90
G4a
ai
ain
13.90
3681
160
12.00
1.10
0.66
3651
1644)
(Pam Plaa)
(Pate Rega)
(Pat (9pa))
(P
Our ELBO based objective Jam
(BL1) Conventional objective Jaw +entropy regularzzer
(BL2) ELBO-based objective prior spate
(L3)), C-TopSim (C3), and W-TopSim (C3) are Figure 4: Results for Thoo (CD), nseg Aw.c
shown in order from the left. The x-axis represents (Nam, Thyal while the y-axis represents the val- ues of each metric. The shaded regions and error bars represent the standard error of mean. The threshold parameter is set 0.25. The blue plots represent the results for our ELBO-based objec- tive Jours, the orange ones for (BLJI) the conventional objective Jeon plus the entropy regularizer, and the grey ones for (BL2) the ELBO-based objective whose prior is phrior The apparent inferior performance of Aw, for Jours compared to the baselines might be misleading. It is because Jours greatly improves both C-TomSim and W-TopSim. The larger scale of their improvements could result seemingly worse Aw,e, but this does not necessarily indicate poorer performance.
Tbboa
Tlae
A Ag,e
C-TopSim
W-TopSim
4.2
4.3
10
6.3
62
4.5
62
(Pat))Patt Pha))
(Par [
(Vat Paal)
[Paat Nlcal
Our ELBD I-O-based objective J.m.
(BL1) Convertional objective J.s + entropy regularizer
(BL2) ELSO based objective prior pros
We show the supplemental results Figure and Figure in which Thoo Tiseg
Figure shows the result for threshold 0.25 and Figure A for threshold = 0.5. In Figure g the blue line plotted for Raeg does not decrease monoton-
ically, i.e., C bit violated when using our objective Jours well as the baseline objectives. implies that the threshold parameter 0.5 is not appropriate in our case.
25

---

Published as a conference paper at ICLR 2024
14
Message Length
13
12
Alphabet size
|A|=
11
Alphabet size |A]
Alphabet size |
10
Alphabet size ||]
9-
Alphabet size |A|
0
200
400
600
800
1000
Objects Sorted by Frequency
Figure 5: Mean message length sorted by objects' frequency across 32 random seeds. A moving average with window size of 10 is shown for readability.
E
SUPPLEMENTAL EXPERIMENT ON ZIPF'S S LAW OF ABBREVIATION
E.
SETUP
Object and Message Space: We set X := {1. 1000), (xy((x) x x Lmax = 30, and |A| E (5, 10, 20, 30, 40) following (Chaaboum et 2019).
Agent Architectures and Optimization: The architectures of sender Sş and Rg are the same as described in Section I. The optimization method and parameters are also the same as described in Section 9
E.2 RESULT AND DISCUSSION
The experimental results are shown in Figure E As can be seen from the figure, there a tendency for shorter messages to be assigned to high-frequency objects.
also observed that the larger the alphabet size [.A], the longer the messages tend to be. It is probably because, as the alphabet size |A| increases, the likelihood of eos becomes smaller in the early stages of training.
As intuitive analogy for this, let assume that sender Sq (M[ and prior message distribution po (M) are roughly similar to uniform monkey ryping (UMT) in the early stages of training:
So (mg x.mii-11
perior (me m11-1) x
(24)
[A]
[AI
The length distribution of UMT is geometry distribution with a parameter p |A|-- (Imagine situation where you roll the dice with "eos" written on one face until you get that face). Then, the expected message length of UMT is pー [A]. Therefore, the alphabet size IA] increases, the sender ( and prior message distribution P3 in the early stages. Consequently, they are more likely to converge to longer messages. Designing neural network architectures that maintain a consistent scale for eos regardless of alphabet size is challenge for future work.
parior (M) are inclined to prefer longer messages
26

---

Published as a conference paper at ICLR 2024
F SUPPLEMENTAL INFORMATION FOR RELATED WORK
Table 2: We show the characteristics of this paper and related EC work in several respects to clarify the positioning of this paper. We mean by "Generative?" whether their formulation based on generative perspective such VAE and VIB, by "Variable message length?" whether message length in their settings is variable, by "Compositionalty?" whether they study the compositionality (e.g., TopSim) of emergent languages, by "ZLA?" whether they investigate if emergent languages follow ZLA, by "HAS?" whether they investigate if emergent languages follow HAS, by "Prior?" whether they are aware of the existence of prior distributions and carefully choose an appropriate one.
The check mark (v) indicates yes, and the blank indicates no.
Variable message length?
Composi- tionality?
Paper
Generative?
ZLA?
HAS?
Prior?
Chaabouni et all (2021)
Tucker all (2022)
Chaahouni all
(2((1
Rita et all
(2(12
Deda & Washio
Andreas
(2()
Li & Bowling (2
haabouni et all
(2(121))
Ren ct
(21124)
Resnick all
(22(22
laniguchi et all
(2(0)
Inukai (2(12
Deda et a|
(212
Ours
27


Reviews:
[{"review_id": "p0T6HjGlvr", "reviewer": "Reviewer_unmd", "summary": "The paper attempts to reframe the conventional Lewis's signaling game within the context of beta-VAE and ELBO, with a focus on the impact of prior distributions on emergent languages. The authors argue that selecting appropriate prior distributions can lead to the emergence of more natural language segments, while the conventional prior may hinder adherence to linguistic properties like Zipf's law of abbreviation (ZLA) and Harris's articulation scheme (HAS).\n\nThe weak points of this paper include:\n(1) The paper is hard to read. The theoretical section includes symbols and equations without full explanation.\n(2) The experiments are weak. The compared methods lack descriptions, and the performance improvement is not well explained.\n(3) The studied problem lacks of enough audience.", "questions": "I suggest the authors address my concerns mentioned in Weaknesses.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The author well introduces the problem, which is well motivated.\n2. The authors provide a deteailed proof in supplementary material.", "weaknesses": "(1) The paper is hard to read. The theoretical section includes symbols and equations without full explanation.\n(2) The experiments are weak. The compared methods lack descriptions, and the performance improvement is not well explained.\n(3) The studied problem lacks audience in the community.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iqUWRUGLzG", "reviewer": "Reviewer_knBD", "summary": "This paper introduces a new perspective on Lewis’s signaling game as beta-VAE and reformulates the game’s objective as ELBO. Based on this modification, it analyzes the influence of the implicit prior function on the properties of word lengths and segmentation of the emergent languages. It also shows that a learned prior distribution of the emergent languages can help evolve a language following Zipf’s law and Harris’s articulation scheme while the previous conventional objectives do not encourage meaningful segments.", "questions": "Based on the current formulation, it seems that the distractors on the receiver’s side are not considered. How would you incorporate the context of the distractors and their corresponding influences [1,2] into the prior design? \n\n[1] Lazaridou, Angeliki, Alexander Peysakhovich, and Marco Baroni. \"Multi-agent cooperation and the emergence of (natural) language.\" arXiv preprint arXiv:1612.07182 (2016).\n\n[2] Evtimova, Katrina, et al. \"Emergent communication in a multi-modal, multi-step referential game.\" arXiv preprint arXiv:1705.10369 (2017).", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The originality of this paper is good. The authors propose a generative point of view of the signaling game and analyze the possible causing factors of the current problems of the emerging less meaningful linguistic properties using the conventional objectives. This can provide a fresh study framework for emergent communication. The rigorous formalization and mathematical equations can integrate previous designs of regularizers and help with future objective design, offering a valuable contribution to the field.\n\n2. The quality of the experiments and analysis is good. They compare different baselines controlling different priors of the objectives. The properties of word lengths, segments, and compositionality are carefully checked. \n\n3. This paper is of good clarity. It is easy to follow the argument of this paper.", "weaknesses": "No obvious weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SwkBBeNfyL", "reviewer": "Reviewer_E7HT", "summary": "This paper reanalyzes an emergent communication-signalling game in terms of\na VAE.  Within this analysis, the optimization of a signalling game is using an\n\"implicit prior\" which leads to statistical properties of the emerging language\nwhich do not match human languages.  Introducing linguistically-inspired priors\ninto signalling game by way of the VAE framework improves the resulting\nemergent languages' statistical properties (i.e., adhering Zipf's Law of\nAbbreviation and Harris's Articulation Scheme more closely).", "questions": "- What exactly is the \"prior\" in the emergent language game?  I understand that\n  it is implicit, but does that mean that is embedded in the objective function\n  (i.e., the $D_\\text{KL}$ term is constant)?  Or is it instead the case that\n  the sender's architectural biases represent the prior?\n- In addition to the theoretical analysis, what else can the authors point to\n  to support the claim that an EC signalling game is analogous to a VAE?\n\n\n### Other comments\n\n- If the authors are assuming a REINFORCE objective for the signalling game,\n  that should be mentioned earlier than Sec 4.1.\n- What is $A_t$ in Eq 3?\n- In Sec 2.2, I do not think the section compositionality is relevant or\n  important; it should removed, in that case.\n- Before Eq 5, what is $\\mathcal A^*$?  Is it supposed to be a Kleene star?\n- What exactly is the uniform prior?  Is it just a constant probability mass\n  over every possible sequence?  If so, how do we know that is the \"implicit\n  prior\" and not something like a uniform _unigram_ prior instead, for example.\n\n\n- Sec 3.1: It is not clear to me how (9) is derived from (2).  I looked at Section\n  B.1, but it was very unclear what was happening because rather than starting\n  with (2) and going to (9), it talks about \"transforming\" different sides of\n  the equation.\n  - It would also be helpful to give an indication of what from Schulman et al.\n    (2015) is being applied (i.e., the what the \"stochastic computation\n    approach\" is).\n  - As a result, I'm not convinced that the reconstruction game, absent\n    modifications to the traditional object (e.g., length penalty), assumes\n    a uniform prior of messages.\n  - It seems like the $P(m) = \\mathbb E_{x\\sim{}P_\\text{obj}}[S(m|x)]$ should be the\n    prior over messages.  I very well might be misunderstanding something here\n    due to terminology.  Am I conflating here that \"prior\" as the distribution\n    of messages the receiver produces given the distribution over inputs with\n    \"prior\" in the sense of our objective function which we are optimizing\n    against (in which case \"prior\" does not refer to anything concrete in the\n    EC environment but rather only to the optimization process by analogy to\n    a VAE's optimization)?\n- Sec 3.2: what is a \"heuristic variant of [a] VAE\"?\n- Sec 3.4:\n  - The very first paragraph of this section, I think, is glossing over\n    critical question in the paper: what is the connection between the \"prior\"\n    and the actual EC setup.  I understand that the EC setup is analogous to\n    a VAE, but what exactly is the analog of the VAE's prior?\n  - I think the use of \"approximately\" is dangerous when trying to make\n    theoretical claims; I understand that it is unavoidable in something as\n    messy as EC, but it still needs to be accompanied by some justification in\n    order to keep the theoretical claims strong.\n\n\n### Minor notes\n\n- \"they are not\" reproduced emergent lanuaguages   has awkward phrasing\n- Right after Eq. (1), it should be $\\log(|\\mathcal A| - 1)\\ge0$ in the case\n  that $|\\mathcal A| = 2$.\n- Sec 3.3:\n  - what is $\\mathcal M$ -- the set of all messages? \n  - What is $\\mathcal A$, again?\n  - Eq 15 limit notation here would be more appropriate", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- (major) The paper aims at re-analyzing a common EC setting in a more\n  formalized way, yielding the potential for theoretical insights that would\n  not otherwise be possible.\n- (major) Furthermore, I think this analysis is largely in the correct\n  direction with analyzing the signalling game as a VAE, looking at inductive\n  biases, and tying in linguistic concepts like Zipf's Law of Abbreviation and\n  Harris's Articulation Scheme (although this wide scope is also a bit of\n  concern; cf. \"Weaknesses\").\n- (minor) The experiments partially satisfy HAS which is known to hold for\n  human languages.", "weaknesses": "- (major) A critical part of the paper is the \"prior\" within a VAE or\n  signalling game, but I did not get a concrete sense of what this prior\n  actually is in the context of a signalling game with neural network-based\n  agents (I expand on this in \"Questions\").  As a result, it makes me unsure\n  how well the theoretical claims actually apply to a real setup.\n- (major) The paper, I think, tried to do too much, and ends up not spending\n  enough time on the core claims, namely, the signalling game can be\n  re-analyzed as a VAE.  I think the paper would benefit greatly from cutting\n  away all but the essential claims and going through those more slowly and\n  thoroughly.\n    - For example, this shows up in the experiments which seem more concerned\n      with evaluating the existence of ZLA/HAS in the newly proposed setting\n      rather than establishing empirically that the signalling game behaves\n      like a VAE.\n- (minor) The notation and the proofs are not very clear, and it made it\n  slow/difficult to work through the equations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wu1rcCkTX5", "reviewer": "Reviewer_TgUR", "summary": "The authors discuss connections between classic emergent communication in Lewis signalling games and Beta-VAEs.\n\nIn some traditional EC works, a speaker and listener must coordinate such that the listener can reconstruction a speaker's \"target\" observation, given communication. In many ways, this mirrors classic reconstruction training. Prior works have often found that the resulting communication from such training is often \"unhumanlike\" in several ways, including ZLA and HAS metrics. This work argues that such undesireable properties are likely a result of implicit priors that most EC works encode. \n\nBy connecting EC to Beta-VAE methods, the authors uncover theoretical interpretations of different terms in EC and open up the important directions for experiments (such as varying prior distributions or Beta).\n\nIn experiments, the authors show that, by using a learnable prior in training agents, they appear to achieve greater separation of EC into \"word-like\" units.", "questions": "1. In the Weaknesses section, I raised questions about why an entropy term would actually increase word segmentation. To repeat it here: is there any mathematical reason that the losses used during training should concentrate surprisal in just a few timesteps (which would induce word-like clumps) instead of spreading the surprisal loss more evenly across time?\n\n2. I struggled to understand Figure 5. What is it depicting? What do the legend entries/different lines correspond to?\n\n3. Just a clarifying question about the results for Criterion C3: the authors' proposed method is worse than baselines, correct? I recognize that topsim values for the proposed method improved generally, but for the narrow metric of the difference between topsim values, there is a decrease, right?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "## Originality\nI'm am somewhat torn about the originality of this work. On the one hand, I think connection EC literature to other theoretically-rich approaches like beta-VAE is a very good idea. On the other hand, the authors note that some prior literature appears to have considered a generalization of this problem (\"Section 6: Tucker et al. defined a communication game... based on VIB, which is known as a generalization of beta-VAE.\"), which makes me think this work is not proposing novel ideas.\n\n## Quality\nOverall, the work seems quite careful and sound in discussing the mathematical underpinnings of many EC methods.\n\n## Clarity\nSome aspects of this paper were quite clear (in particular, the introduction and conclusion are very good), but I found other aspects harder to read. I suspect this is somewhat due to having a fair amount of notation is not immediately interpretable without remembering definitions from earlier pages (e.g., \"monkey typing model\" or n_bou).\n\n## Significance\nI think this work falls within an important (and significant) field of connecting EC to other training methods and objects. I remain somewhat confused about the relationship to prior art, however, so I am unsure of the significance of this individual work.", "weaknesses": "Overall, I like aspects of this work, but there are a few important unresolved questions or weaknesses that I would want to see addressed before accepting, in particular about relations to prior art.\n\n## Relation to prior art\nThe authors do a good job noting related prior literature, but I remain somewhat confused by the position of this paper relative to such literature. In particular, the authors write that:\n\n> Moreover, Resnick et al. (2020) explicitly formulated the objective as ELBO, though it is not directly applicable to this paper...\n\n> Tucker et al. (2022) defined a communication game called VQ-VIB, based on Variational Information Bottleneck (VIB, Alemi et al., 2017) which is known as a generalization of beta-VAE. Also, Chaabouni et al. (2021) formalized a color naming game with a similar motivation.\n\nIf prior art has used the same formulation and considered a generalization of the problem this paper is considering, what are the contributions of this paper? Honestly, I suspect there are many unique contributions made in this paper, but the contrast relative to prior art should be made much more obvious. Even just adding a sentence at the end of each related works section saying, e.g., \"While Tucker et al., Alemi et al., and Chaabouni et al. consider similar frameworks to us, we introduce novel metrics and results\" or something to that effect would help a lot. Ideally, the authors would run experiments comparing to Resnick's method.\n\n## Presentation of results\nI found the results somewhat difficult to read. Figure 2 contains the main results, and with enough flipping between pages, I could eventually figure out how to interpret them, but generally I encourage authors to make figures more self-contained. For example, listing a baseline as BL1 is not as informative as using a name/label that actually describes characteristics of the baseline (e.g., conventional + entropy).\n\n## Why did segments become more meaningful.\n\nThe analysis in this section, while addressing a very important question, is slightly unsatisfying. First, parts of the writing are very casual (e.g., The receiver must be surprised several times...\"), whereas in reality the receiver just needs to receive, over multiple timesteps, enough bits to reconstruct the input. It is unclear what it means to \"be surprised\" as a binary term.\n\nSecond, I question the fundamental conclusion of this paragraph. The authors appear to suggest that the competing terms for entropy and reconstruction are what give rise to word boundaries. In other words, communication wants to often be predictable (because of the entropy term), which creates word-like clumps. However, as the authors note, the speaker needs to communicate some information in at least some timesteps to convey the meaning to the listener. Is there any mathematical basis, given the training terms used, for why that information should be concentrated in just a few timesteps (which would match word-like clumps) as opposed to evenly distributed across time? For example, in a simple four-timestep case conveying 4 bits, is there any advantage (as measured by decreased loss) to transmitting [2 bits, 0 bits, 2 bits, 0 bits] vs. [1 bit, 1 bit, 1 bit, 1 bit]?\n\n## Minor:\nAppendix D would greatly benefit from a little bit more text explaining what the graphs present. There are also some sentences that need editing (e.g., \"threshold is set to 0.25.\")", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)