On the role of population heterogeneity in emergent communication
Authors: Mathieu Rita
Florian Strub
Jean-Bastien Grill
Olivier Pietquin
Emmanuel Dupoux
Venue: ICLR 2022 Poster (2022)

Abstract: Populations have often been perceived as a structuring component for language to emerge and evolve: the larger the population, the more systematic the language. While this observation is widespread in the sociolinguistic literature, it has not been reproduced in computer simulations with neural agents. In this paper, we thus aim to clarify this apparent contradiction. We explore emergent language properties by varying agent population size in the speaker-listener Lewis Game. After reproducing the experimental paradox, we challenge the simulation assumption that the agent community is homogeneous. We first investigate how speaker-listener asymmetry alters language structure to examine two potential diversity factors: training speed and network capacity. We find out that emergent language properties are only altered by the relative difference of factors between speaker and listener, and not by their absolute values. From then, we leverage this observation to control population heterogeneity without introducing confounding factors. We finally show that introducing such training speed heterogeneities naturally sort out the initial paradox: larger simulated communities start developing more systematic and structured languages.

Published as a conference paper at ICLR 2022
ON THE ROLE OF POPULATION HETEROGENEITY IN EMERGENT COMMUNICATION
Mathieu Rita
Florian Strub
Jean-Bastien Grill DeepMind jbgrill edeepmi nd com
INRIA, Paris
Deep.innd
ma mathieu. ril.uu.iit@iirisee fr
fat.rubdeeepm ndd. com
Olivier Pietquin
Emmanuel Dupoux
Google Research, Brain Team pietquinégoogle. com
EHESS,ENS ENS-PSL.CNRRSINRE
Meta AI Research
emmanue 1 dupouxêgma i com
ABSTRACT
Populations have often been perceived as a structuring component for language to emerge and evolve: the larger the population, the more structured the language. While this observation is widespread in the sociolinguistic literature, it has not been consistently reproduced in computer simulations with neural agents. In this paper, we thus aim to clarify this apparent contradiction. We explore emergent language properties by varying agent population size the speaker-listener Lewis Game. After reproducing the experimental difference, we challenge the simula- tion assumption that the agent community is homogeneous. We then investigate how speaker-listener asymmetry alters language structure through the analysis potential diversity factor: learning speed. From then, we leverage this observa- tion to control population heterogeneity without introducing confounding factors. We finally show that introducing such training speed heterogeneities naturally sort out the initial contradiction: larger simulated communities start developing more stable and structured languages.
INTRODUCTION
Language emergence has been explored in linguistics and artificial intelligence for two main rea- sons (Lazaridou & Baroni, 2020). On the one hand, artificially reproducing language emergence may help to understand the evolution of human languages (Steels, 1997; Briscoe, 2002; Wagner et al., 2003). On the other hand, language is known to be structured, and compositional (Bickerton, 2007), and imitating such properties would enhance machine learning representations. As result, there exists a constant back and forth between cognitive sciences, linguistics, and artificial intel- ligence to retrieve the core ingredients of language emergence (Kirby et al., 2008). In this paper, we explore how the size of population may impact the structure of emerging languages by using neural reinforcement learning methods.
Especially, we explore the following socio-linguistic hypothesis: larger communities create more systematic languages (Raviv al., 2019a; 2020). This hypothesis has been supported by a number of ethnographic (Gary Lupyan, 2010) and socio-linguisties (Raviv et al., 2020) observations as well as behavioral studies mimicking language emergence in a controlled setup (Raviv et al., 2019a). Few neural language emergence papers have explored how community size impacts language structure so far, but the available evidence mitigated at best. Tieleman al. (2019) observed small but consistent regularization effect when pairing auto-encoders within a population. Similarly, Cogswell et al. (2019) observed slight improvements in language compositionality with a large population but only reported them in few experimental settings. Finally, Graesser et al. (2019) studied the impact of contact-agents for different population sizes, but they did not observe a correlation between the population size and the convergence speed, success rate, or mutual agent intelligibility.
The following question arises: why does community size not improve language properties in recent emergent communication literature, although it key structuring factor in broader linguistics lit-

---

Published as a conference paper at ICLR 2022
erature? We argue that recent emergent communication models are limited as they ignore individual learning capacities by working only with homogeneous populations. Consequently, they miss cou- pling effects emerging from agents' asymmetries. As a result, we hypothesize that community size effects could occur as soon as local heterogeneities are introduced into populations.
In this work, we explore the effects of population size with neural agents in the well-known Lewis referential game (Lewis, 1969). In this game, a speaker describes a hidden object to a listener, which must then reconstruct object properties. Both agents thus need to co-develop a communication pro- tocol to solve the task. The population-base variant of this game randomly pairs one speaker and one listener from their respective communities. The goal to observe whether increasing the num- ber of agents enhances the communication protocol qualities, e.g. success rate, compositionality, generalization etc. (Kottur et al., 2017; Chaabouni et al., 2020; Lazaridou et al., 2018).
Firstly, we reproduce Lewis reconstruction setting and confirm the experimental difference: when increasing the number of agents, we do not observe improvements over various emergent language metrics. We thus question the current paradigm to model population in the language emergence literature. In particular, all agents are trained uniformly. i.e., their learning speed, capacity, sampling are identical (Tieleman et al., 2019; Cogswell et al., 2019; Fitzgerald, 2019). Secondly, we evaluate the impact of a potential source of model heterogeneity: agents learning speed. We observe that the absolute value of speaker- -peaker-listener listener speed is not important, yet their relative value is crucial. We hence shed light on the strong correlation between language structures and agents relative training facilities. Thirdly,
push this reasoning further by distributing learning speeds across the pop- ulation thus creating heterogeneous populations. We there observe an improvement of language scores combined with a variance reduction when increasing population sizes. In other words, larger communities of neural agents start developing more stable and structured languages when being heterogeneously designed. This observation brings first stone toward solving the empirical and computational contradiction.
Our experiments partially removed the apparent contradiction between the socio-linguistic obser- vations and the recent emergent communication literature. They illustrate how crucial population training dynamics are in shaping emergent languages and how population heterogeneity may have been underestimated in the recent emergent communication literature. All in all, our contributions are three-fold: (i) we empirically show that the community size is not a structuring factor in lan- guage emergence by in itself in the classic homogeneous Lewis setting; (ii) give evidences that speaker- listener relative dynamics strongly affects language properties; (iii) we provide the first computational cues remove the apparent difference between the sociolinguistic literature and cent neural emergent communication works.
2 RELATED WORK
Population size in sociolinguisties. Population size core parameter defining the social envi- ronment agent interacts with. Its impact on language structures has largely been studied hu- mans (Nettle, 2012; Bromham et al., 2015; Reali et al., 2018: Raviv et al., 2019a) and animals (Blum- stein & Armitage, 1997; McComb & Semple, 2005; Wilkinson, 2013). By analyzing 2000 lan- guages (Dryer & Haspelmath, 2013), a clear correlation was drawn between population size and diverse language features, c.g. larger communities tend to develop simpler grammars (Gary Lupyan, 2010; Meir et al., 2012; Reali et al., 2018). As part of their research on the influence of network structures on language emergence (Raviv et al., 201 20;9;;;; 2020), Raviv al. (2019a) went one step further by arguing that the community size is predictive of language structure and diversity. To do so, they split 150 people into different groups of given community size isolate confounding factors. While people played speaker-listener Lewis game, they observe that the greater the com- munity size, the simpler and more consistent the generated language. Here, we intend to test this assumption in the context of neural language emergence. All things considered, we adopt setting close to Raviv et al. (2019a)'s 's when computationally modeling human population.
Populations in experimental emergent communication. Experimental language emergence has mainly been studied with two methods: behavioral studies (Kegl, 1994; Sandler et al., 2005) and simulations (neural and non neural) (Wagner et al., 2003; Lazaridou & Baroni, 2020). From be- havioral studies and non neural simulations, two main approaches have emerged in the past twenty years: experimental semiotics (Galantucci & Garrod, 201 I; Garrod et al., 2007) and iterated learn-
2

---

Published as a conference paper at ICLR 2022
ing (Kirby & Hurford, 2002; Kirby et al., 2008; Beckner et al., 2017). According to experimental semiotics studies, languages are mainly subject to an expressivity pressure; they argue that mes- sages should be highly informative to allow communication within a group (Fay & Ellison, 2013). According to iterated learning paradigm, structures emerge from a compressibility pressure; they argue that memory limitations compel messages become simpler to be easily learned (Tamariz & Kirby, 2015), which is also referred to as transmission bottleneck (Smith et al., 2003). Kirby et al. (2015) then combined those twO approaches and show that languages emerge as trade-off between expressivity and compressivity during cultural evolution. Recently, similar ideas have been modeled in emergent communication frameworks involving neural agents (Ren et al., 2020; Lu al., 2020). However, seminal non-neural simulations all used diverse optimization methods and models across studies (Wagner et al., 2003); is hence hard to generalize a global trend of language emergence due to the experimental specific, and sometimes contradictory conclusions. Modern neural agents man- age to simplify and standardize agents' modeling, paving the way for holistic models of emergent communication (Lazaridou & Baroni, 2020). Our paper is related to this last set of neural works.
Recent works in emergent communication have been debating the prerequisites to the emergence of language universals such compositionality (Li & Bowling, 2019; Ren et al., 2020; Lazaridou et al., 2018; Mordatch & Abbeel, 2018; Resnick et al., 2020; Kottur et al., 2017; Choi et al., 2018; tukasz Kuciński al., 2020), generalisation (Baroni, 2020; Chaabouni al., 2020; Hupkes al., 2020; Denamganai & Walker, 2020), efficiency (Chaabouni et al., 2019; Rita et al., 2020) or sta- bility (Kharitonov et al., 2020; Lu et al.. 2020). Among them, a few works explored how different population structures may impact properties of emergent languages. Inspired by iterated learning methods, Ren et al. (2020); Li & Bowling (2019); Lu al. (2020) look at language evolution across multiple generations of agent pairs, i.e. population is spread over time. However, we here consider a population where multiple agent-pairs coexist simultaneously within a single generation. There, Graesser et al. (2019) show that community of agents start coordinating their language when at least three agents are present in the community. They later assume that increasing the community size may impact the emergent shared language, but did not observe it in their initial experiments. A similar hypothesis was also made by Bouchacourt & Baroni (2019) while analyzing the influence of symmetric agents in the emergence of common language. With different research objectives, Cogswell et al. (2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission. Analogously, Tieleman al. (2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers' latent representations by increasing population size. Eventually, Fitzgerald (2019) suggest that pop- ulations improve generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties. We here analyze how popu- lation size affects those discussed properties. We align with emergent communication literature and show that naively increasing community size does not consistently improve language properties. We then challenge the homogeneity assumption made in most population designs.
METHOD
We here describe the different components of language emergence in a population-based Lewis Game, namely, game rules, notations, training dynamics, and evaluation metrics. Finally, we define how we alter population dynamics by asymmetrizing agents and injecting heterogeneities.
3.1 RECONSTRUCTION GAME.
We study emergent communication in the context of the Lewis reconstruction games (Lewis, 1969). There, speaker observes all the attributes of an object. The speaker then outputs descriptive message, which a second agent, the listener, receives. The listener must accu- rately reconstruct each value of each attribute to solve the task. Both agents are finally rewarded in light of the reconstruction accuracy. Note that another variant of this game requires the listener to retrieve the correct object within list of distractors, but both settings are inherently similar.
Game Rules:
Game Formalism:
The observed object u E y is characterized by |K| attributes where each attribute may take [V] values. We encode the observed object by a concatenation of one-hot repre- sentations of the attributes 2a E V of the object v for each attribute A E K. For each new run, the set of objects is split into training set X' and test set. The intermediate message m E W is sequence of 7 tokens, m (m) where each token is taken from vocabulary W of dimension
3

---

Published as a conference paper at ICLR 2022
[W]. finishing by a hard-coded end-of-sentence token EoS. The speaker and listener are two neural agents respectively parametrized by f and Φ. The speaker follows a recurrent policy т&: given an in- put object v, it samples for all token m, with probability To (m, mm<e v). We denote To (m]) (u) the probability distribution of the entire message given an input object v. The listener outputs for each the values V: n (fn//m)) At training time, the speaker message is generated by sampling the policy m TT(-lu). At test time we use the greedy message me arg m To (m m=::") v).
k attributes probability distributions
Game Objective: As in (Chaabouni et al., 2020), we define the listener training goal to be the average of the multi-classification log- log-likelihood loss per attribute:
Σ Σ NEA MEWT KEJK|
Tm(mlu) log (17()
(1)
|
In our setting, we want the speaker to optimize the same objective. To so, we define the speaker game reward as the negative loss of the listener,
1 E log (T (T)())))) K
if
re re met)
otherwise.
Following the gradient policy theorem (Sutton et al., 2000), maximize the speaker reward by minimizing the following objective over 0:
ΣΣ log Ty (m/las mer) re(u, mes),
Co
(3)
[2
DEX FET
where m is sampled according to the speaker policy Tg.
3.2 POPULATION- DAPESE RECONSTRUCTION GAME
We first create population of N speakers and N listeners, thus obtaining total number of 2N agents. Following (Tieleman et al., 2019), at each step, we uniformly sample one speaker and one listener and pair them together. We then proceed as in the classic one pair Lewis game: both agents play the game with batch of inputs and receive an optimization step minimizing (Equation 3 & I). This operation is repeated until convergence, i.e., all speaker-listener pairs have stable losses. While standard, we note that this training procedure relies on strong latent assumptions: (i) each speaker (resp. listener) uniformly sampled, i.e., there is no preponderant agent within the population, (ii) the communication graph is fully connected and uniform, i.e., all speakers may be paired with all listeners with the same probability, (iii) agents cannot be differentiated, i.e., agents have no information about the identity of their partners, (iv) speakers and listeners are all similar, i.e., there is no difference in the agent definitions nor in the optimization process. Overall, those hypotheses create homogeneous training setting. In practice, the agents only differ by their initialization and optimization updates, e.g., stochastic agent pairing, game generations, and message sampling.
3.3 ASSESSING EMERGENT LANGUAGE IN LEWIS GAMES.
We here introduce various metrics to assess emergent languages structure and quality. To do so, we first need to introduce two distances in the input/object space and in the message space. We define the distance between two objects v and v' as the proportion of distinet attributes computed by Dong v²) І En L A v'a). For the distance between two messages and m' we use the edit-distance (Levenshtein et al., 1966) and note it Dmes(m, (m")))
Within populations, we measure how close speakers' languages are by introducing a distance between the two languages. Given two set of speaker weights O2 and 6g and their respective language Lø, and Leg.
Speakers synchronization:
DL Le,)= = EuEX,mI~Te, (lle). (-Io)))))))))))))))))) [D Dme (m1, m2)].
(4)
computes the average distance between two speakers' messages over all the dataset. When con- sidering an entire population of N speakers, we can then compute the synchronization score r rsyme by averaging the distance between all pairs of speakers: Fayne- 1 NNN-1 Eits Dr (Lo, Le, ).
4

---

Published as a conference paper at ICLR 2022
Entropy: To measure language coherence, study the entropy of the speaker language No:
Hg = Bv.m-na(-hv log((T = EvEx
h(TT(melmess v))
(5)
where h w + w log(w). our setting, we seek to minimize entropy as it reduces language com- plexity and denotes language stability (Kharitonov et al., 2020). When Ho is minimal, the speaker uses unique message to refer to an object. When Hy is high, the speaker generates synonyms to refer to the same object. While some entropy is beneficial with noisy communication channels, they do not improve robustness in our case; makes the listener's task harder. To case reading, we display the negative entropy (Neg-Entropy), so all metrics increases correspond to language improvement.
Topographic Similarity: We use topographic similarity as a quantitative measure of composition- ality (Brighton & Kirby, 2006; Lazaridou et al., 2018). Topographic similarity captures how well similarities between objects are transcribed by similarities in the message space by computing the Spearman correlation (Kokoska & Zwillinger, 2000; Virtanen et al., 2020) between pairwise object distances (Doss ( U1, 22)) and the corresponding message distances (Dmes (M m,,)).
Test accuracy quantifies how well agents generalize to unseen objects. It measures the average reconstruction success on the test set where we define a success when the greedy pre- diction perfectly matches the input object. The greedy prediction e is the one which maximizes the likelihood computed by the listener given the greedy message m. 0 arg max, IL T (u))m) with ni arg max, To (m Ma v) the greedy message.
Generalization:
Stability: We consider that a metric is stable if has low variation across the different populations. To do so, we simply measure the stability of language metric by computing the standard deviation across seeds. In this paper, all experiments are run over six seeds.
3.4 ALTERING POPULATION DYNAMICS
We present how diversity can be introduced within population either by asymmetrizing speaker- listener in minimal size population (N = 2), or distributing heterogeneity within larger populations.
We aim vary specific agent features toward creating diversity in the population and target one core component: the training speed. We control agents' training speed by changing the number of gradient updates. Formally, we introduce the probability p of an agent to be optimized after each iteration of the game. Therefore, the lower p. the slower agent training. This approach has two positive features: (i) neither alters the communication graph nor the agent sampling, preserving other homogeneity hypotheses in the population, (ii) it is more stable than modifying the learning rate as it avoids large destructive updates.
Diversity factor:
We provide a control parameter to characterize diversity the scale of minimal size population (N = 2). Noticeably, N 2 is the closer setting to single-pair of agents where speaker-listener training speed asymmetry, we introduce the relative training speed Papeed := p5 /p- where y (resp. p²) is speaker's learning speed (resp. listener's learning speed).
Control parameters of local asymmetry:
compute speakers synchronization. To analyze the effect of
Distributing Population Heterogeneity: To create heterogeneous populations, we characterize ev- cry single speaker and listener with individual properties. We then sample them once at the begin- ning of the training. Formally, when altering training speed, we sample p; Log-N (Mp. op) for every agent in the population, where Log-A a log normal distribution. In Appendix D. 1, we also explore another diversity factor, i.e. agent capacity, complete our analysis on heterogeneity.
EXPERIMENTAL SETTING
Game Properties: In the main paper, we use |K| 4 attributes with | 4 values. We also report scores with |K] = attributes and [V| = 10 values in the Appendix F.1 to illustrate that our observation still holds in different setting. Finally, message length T 10. For each run, we generate objects in yK uniformly at random, and we split them into train and test sets, which respectively contain 80% and 20% of the objects.
vocabulary size of [W] 20 and
Neural architectures: The speaker first encodes the input object v in a one hot encoding and then processes it with linear layer. It uses it to initialize single layer LSTM (Hochreiter & Schmidhu-
5

---

Published as a conference paper at ICLR 2022
Speakers chnoiiiazzaiion
Entropy
Compasitionalit
Generalization
0.8
0.95
0.7
0.6
0.4
20
-1.5
10
10
20
0.80
20
N
Figure 1: Emergent languages properties of homogeneous populations of increasing sizes. Based on socio-linguistics results, all curves should trend upward with population size.
ber, 1997) with layernorm (Kiros et al.. 2015) and hidden size of 128. Finally, the LSTM output is fed to a linear layer of dimension [W] followed by softmax activation. The listener is composed of uo table of dimension 128 followed by LSTM with layernorm and hidden size of 128.
Then, for each attribute & E {1. .... K), we apply a linear projection of dimension [V] followed by softmax activation to the last LSTM output.
Optimization: For both agents, we use Adam optimizer with learning rate of 5c10-3, f1 0.9 and B2 = 0.999 and training batch size of 1024 when optimizing their respective loss. For the speaker, we set the entropy coefficient of 0.02. Finally, we re-normalize the reward by using F(ar, wee) r(z,we))-% to reduce the gradient variance (Sutton et al., 2000), where T/ and of are of respectively the average and the standard deviation of the reward within the batch at each time step.
We use the EGG toolkit (Kharitonov et al., 2019) starting framework. The code is available at bstpp::/ psttts://ee htt///gtttuut com htt//MMtthhetitee Population. All experiments are run over six www.it.b..o//aaaheeuueeeeeeee
seeds. In all figures, bars show standard deviation.
RESULTS AND DISCUSSION
We first reproduce the impact of the population size on language properties in the homogeneous Lewis setting. We then study speaker-listnner asymmetry by altering networks training speed, and highlight the importance of relative training speed in shaping language. We finally use training speed as non-confounding heterogeneity factor when varying the community-sine and thus tackle the initial contradiction.
5.1 COMMUNITY SIZE IS NOT ALONE A LANGUAGE STRUCTURING FACTOR
Language properties are not enhanced by community size. In Figure we observe that in- creasing population size does not improve language properties: speaker synchronization and com- positionality remain almost constant; entropy and generalization are even deteriorating, Spearman correlations respectively -0.44, -0.15, -0.52, -0.14, i.c. there is no positive correlation between those metrics and population size. In addition, we do not observe any gain of language stability by increasing population size: increasing community size does not reduce the standard deviation of the language metrics across seeds. Finally, we note that speakers synchronization is high despite using large communication channel, also observed by (Graesser al., 2019). Overall, we confirm that community size does not improve the language properties of neural agents.
The potential pitfall of homogeneity. We hypothesize that the absence of positive correlation between population size and language properties may be explained by the simplicity of population modeling- The homogeneity assumption seems restrictive as it may lack the inherent diversity of real human communities. Although agents are not identically initialized, they have the same training dynamics in expectation. Consequently, we formulate the subsequent hypothesis: Community size may be structuring factor as soon heterogeneity is introduced in populations design.
5.2 LANGUAGE PROPERTIES ARE CONTROLLED BY SPEAKER-LISTENEE ASYMMETRIES
Before generating heterogeneous populations, we explore the impact of learning speed hetero- geneities at the scale a minimal size population (N = 2) in order not to add confounding factors and better intuit population dynamics. As described in Section 3.4, here modify the learning speed for minimal size population. Note that we compute this ratio through multiple values of p /pL, where p E (0.01, 0.02, 0.04, 0.1, 0.2, 0.5, 1.5.

---

Published as a conference paper at ICLR 2022
Speakers synchronization
Nog Nog-Entropy
Compositionaliy
Generallzztion
1.00
1.0
0.7
0.75
0.9
-5
0.50
0.6
0.25
-10
0.5
10
10
10
10
10-
10
10
10
10"
10"
10"
10
10
10
10"
16'
Pspeed
Pipeed
Pipeed
Pipeed
Figure 2: Emergent language properties a function of speed ratio Pspeed with N 2. Left right on the X axis, the speaker have higher training speed while the listener have lower training speed.
Relative differences between speaker and listener significantly affect language properties. In Figure 2, we display the evolution of language properties with Pspeed- We observe that asymmetriz- ing speaker-listener learning speed significantly influences language properties. When the speakers have large update probability Ps. we note a global improvement of language properties: (i) speakers synchronization rate is very high, (ii) language neg-entropy is reaching almost O, (iii) composition- ality is significantly improved and (iv) test accuracy is close its optimal value. On the contrary, when listeners have a large update probability compared to the speakers, we note the opposite trend. In Appendix C, we also test different game setups for completeness, and obtain similar conclusions.
As sanity check, we then verify that altering does not fundamentally change the language prop- erties per itself. We thus sweep over the gradient update probability p while setting a minimal population size of N = 2, and compute the Spearman correlation in Table 1. We observe that hyperparameter p has no statistically significant correlations with most of the language scores, i.e. their Spearman coefficients are below 0.4 with p-values above 0.05. Therefore, the impact of this correlation is small enough in our experiments to be neglected. In sum, it is not the magnitude of the parameters that is critical to shape language, but the relative ratio between them.
Understanding learning speed asymmetries. We here try to provide some pieces of intuitions to understand Pspeed by looking the extreme cases. In the limit Pspeed < 1, i.e. Ps < Pt the listener is almost optimal wrt. the speaker at each speaker update. It implies that listener's loss is optimally minimized for each message referring to single input. Then, as soon speakers develop a language where each message refers to single input, the game is over as speaker's s rewards are immediately maximal. It means that many languages can emerge from this limit case, including almost degenerated languages.
In the limit Papeed >> 1, i.c. Ps >> Pt, the speaker is almost optimal wrt. the listener at each listener update. The speaker thus targets the messages providing the highest rewards as in a stationary RL task. We can safely assume that the set of messages providing the highest rewards for each input is small. It then explains why the speaker converges to low entropy language. In addition, when N = 2, both speakers solve the same communication task on same quasi-stationary environment and obtain similar rewards, they are then more likely to align on similar language. This use-case also suggests that when speakers are "fast enough", a common interlocutor is the only requirement for the emergence of a common language. Finally, push further the assumption by interpreting the gain of compositionality through "ease-of-teaching that compositional codes are easier teach for listeners. We then hypothesize here that 'faster' speakers develop codes easier to learn for 'slow' listeners and thus higher quality languages.
(Li & Bowling, 2019). The authors show
In Appendix D. we show that trends noticed in Figure 2 can be reproduced varying the ratio be- tween speaker's 's and listener's capacities. It suggests that other parameters influencing the relative speaker- listener co-adaptation affect language properties. We provide first hints on an indirect influ- ence of this capacity ratio on agents' relative training speed. More generally. we suspect that many parameters may be related to training speed, e.g. community- graph (Graesser al., 2019), newborn agents (Ren et al., 2020; Cogswell et al., 2019), or broadcasting by using different speaker-listener ratios as illustrated in Appendix D.2, but we leave for future analysis.
Table 1: Spearman Correlation between h, p and language scores. p-values < 0.05 are underlined. Compo
Param.
Sweep
Sp. sync Neg-Entropy
Gene.
(0.01, 0.02, 0.04, 0.1, 0.2, 0.5, I.
0.13
0.33
0.05
-Se-3
7

---

Published as a conference paper at ICLR 2022
Speakers synchronization
Neg-Entropy
Compositionality
Generalization
0.75
0.70
0.63
0.60
0.55
0.50
Homogeneous population
Best pair (N-2)
Hetorogenenus population
Figure 3: Emergent languages properties of heterogeneous populations of increasing sizes. For heterogeneous populations, we start recovering the aocio-linguistce correlation. Orange dashed line shows homogeneous populations and green dashed line the best pair for 22 across all Pepeed-
5.3 COMMUNITY SIZE STRUCTURES LANGUAGES OF HETEROGENEOUS POPULATIONS
Previous sections have identified that learning speed a powerful controlling factor and could alter language structure when asymmetrizing speaker-listene, pairs. However, these experiments remain limited for (i) we have not yet observed larger population size tends to generate more stable and structured language, (ii) this asymmetry remains artificial and specific to our Lewis game setting. Therefore, we here leverage this training speed factor within population, characterizing individual agents with differrnnt update probabilities. We hence sample agent update probability with Log-N(Tp> Op) with 7pp = and op = 1 if not specified otherwise.
Larger heterogeneous population leads to higher quality language. In Figure 3, we observe that emergent language metrics are now positively correlated with population size. When increasing the population size from 20 speaker synchronization and neg-entropy go from low to almost op- timal values, compositionality increases by 22% in average and generalization remains stable. As detailed in Appendix F,, speaker synchronization, neg-entropy and compositionality have signifi- cant correlation with Spearman coefficients equal to 0.55, 0.24, and 0.21 (p-values< < 0.05). Those are the first hints toward our initial goal: when individuals are heterogencous, large population size correlates with higher quality and stable language properties. In Appendix F.2, we report the results with distribution changes. When the distribution is more concentrated, trends are less significant and we note that metrics do not have the same sensitivity to heterogeneities: a minimal dispersion of the distribution is required to observe significant effects. When changing the distribution without changing variance (3 distribution instead of log-normal distribution), trends are not notably affected.
The overall gain over language metrics can interpreted in light of Section 5.2. Language score evolution depends on the relative speaker-listener learning speeds. In heterogeneous populations, the speaker-listener
probability of sampling extreme learning speed p and thus enforcing speaker- listener asymmetries increases with N. One explanation would be that extreme agents, especially slow listeners, behave as kinetic bottlenecks, forcing the speakers to structure their languages. Thus, language emergent properties would be particularly determined by fast speakers and/or slow listeners.
Larger heterogeneous population leads to more stable language. Similarly, the population size correlates with the variance of the language scores across seeds. Hence, when increasing the popu- lation size from 2 to 20; the standard deviations of speakers synchronization, neg-entropy, compo- sitionality are divided by 10, 15 and 2: yet generalization reduction is that smaller communities are more likely to be disparate because of the sampling. For instance, if we characterize a population with the empirical average of the training speeds 7p. has high variance with small N. On the other side, when N Vgrows, Thp gets closer to the average learn- ing speed, lim Tip 7pp- and its variance gets smaller. other words, larger heterogeneous communities have more stable languages because of Law of large numbers in our models.
less affected. One hypothesis for this
Increasing diversity enforces population size benefits. Figure 4, we challenge further the im- pact of heterogeneity by varying the standard deviation op of the learning speed distribution. For each value pp- we run the game with populations of size N relative evolution of the scores between the two population sizes. We note that the higher o, the larger the gain of synchronization, neg- eeg-eenropy, generalization when increasing N. The relative gain of neg- neg-entropy is almost equal to 100% when o, 1 while speaker synchronization gain is close to 200% for large values of op. For generalization, the 10% relative gain is noteworthy as it corre- sponds reaching close to 100% test accuracy. The more discrepancy exists within the population
and N = 10 and compare the
8

---

Published as a conference paper at ICLR 2022
Relative dift.
Mean (im %g
Speaker synchrorization
100
Neg-entropy
100
Compositionality
20
Generalization
-100
-100
-20
%
%
%
Relative diff.
Std (in %)
100
100
100
-100
2
-100
2
-100
4 -100
0
6o
Figure 4: Relative variations in average language scores (top, blue line) or standard deviations (bottom, orange line) between populations of size N = 10 and N = 2 as a function of increasing heterogeneity oy {0, 0.1, 0.5, 1, 2, 3, 4, 53. is defined by 100 (ØN=10- =N=2)/ / ØN= where UN is the average of the mean (or std) across seeds for a population of size N. The higher diversity the better the benefits: language is of higher quality (blue), and the language more stable (orange).
(large op), the more beneficial is to have large populations. Furthermore, speaker synchronization, neg-entropy and generalization standard deviations reduce up to almost 80% when having op 2. In sum, this result corroborates that larger populations lead to more stable language.
On the overall performances of heterogeneous populations. On Figure 3, we see that language properties of homogeneous populations with Pspeed = 1 decrease when population size is increas- ing. On the contrary, metrics get higher for heterogeneous populations (generalization stays almost constant). Thus, when modeling large populations, there is a gain distributing heterogeneities within populations. However, metries remain below the best pair observed for N = 2 when varying Pspeed- It means that enlarging populations does not lead to an absolute better language than the best one we can obtain with a minimal size population. Though, when population size increases, hetero- geneous populations metries get closer to the best pair scores. Therefore, population size tends to synchronize agents on language whose properties are close to those of the best single pair.
6 CONCLUSION
One objective of language emergence simulations is to identify simple (non-biological) rules that characterize language evolution (Steels, 1997; Briscoe, 2002; Wagner et al., 2003; Kirby, 2001). However, those models require a delicate balance between simplicity to ease analysis and complex- ity to remain realistic. In this paper, we argue that the current population papsed models may actually be too simplistic as they assume homogeneous communities. We show that this simplification is potential root cause of the experimental difference between neural observations and the sociolinguis- tic literature. Namely, larger populations should lead to more stable and structured languages, which is not observed in recent emergent models. Yet, as soon as we add diversity within populations, this contradiction partially vanishes. We advocate for better integrating population diversity and dynam- ics in emergent literature if we want computational models to be useful. In this journey, we also observe that the relative training speed of the agents may be an underestimated factor in shaping languages. Worse, other agent properties may be confounded with it, such as network capacity.
This work also opens many questions. First, our observations were performed in the Lewis game so far, and it would be interesting to observe the impact of diversity in more complex settings. Second, while heterogeneity investigated, e.g., how to correctly model diversity, are there some causal components? Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial experimental difference such as varying the communication network topology (Wagner, 2009), the proportion of contact agents (Wray & Grace, 2007; Clyne, 1992; Graesser et al., 2019) or the proportion of second learners (Ellis, 2008). Fourth, although heterogeneous populations recover a sociolinguistic result, the average scores remain below the best emergent protocol. We leave for future work how heterogencous populations may be leveraged to structure further the language (c.g. more complex tasks, larger population). Finally, training speed is a natural controlling parameter for computational models, but it is unclear how it may relate to human behavior. Overall, we hope that this paper provides new insights toward modeling populations in language emergence, and it is part of this constant back and forth between cognitive sciences, linguistics, and artificial intelligence.
structuring factor in population, the core factors are yet to be

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
Authors would like to thank Corentin Tallec, Rahma Chaabouni, Marco Baroni, Emmanuel Chemla, Paul Smolensky, Abdellah Fourtassi, Olivier Tieleman, Kory Mathewson for helpful discussions and the anonymous reviewers to their relevant comments. M.R. was supported by the MSR-Inria joint lab and granted access the HPC resources of IDRIS under the allocation 2021-AD01 1012278 made by GENCI. E.D. was funded in his EHESS role by the European Research Council (ERC- 2011-AdG-295811 10 BOOTPHON), the Agence Nationale pour la Recherche (ANR-I7-EURE-001 Frontcog, AAR-1O-IDEX0001 PSI.* ANR-19-P3IA-OO from CIFAR (Learning Machines and Brains) and Meta Al Research (Research Grant).
PRAIRIE 31A Institute) and grants
REFERENCES
Marco Baroni. Linguistic generalization and compositionality in modern artificial neural networks. Philo- sophical Transactions of the Royal Society B: Biological Sciences, 375(1791):0010007, 2020. doi:
10.10098/stb22019.3377. 10.1098/rstb. 2019.0307.
Clay Beckner, Janet B Pierrehumbert, and Jemnifer Hay. The emergence of linguistic structure in online iterated learning task. Journal Language Evolution, 2(2):160-176, 2017.
Derek Bickerton. Language evolution: A brief guide for linguists. Lingua, 117(3)510--22 2007.
Daniel T. Blumstcin and Kenneth B. Armitage. Does sociality drive the evolution of communicative complex- ity? comparative test with ground-dwelling sciurid alarm calls. The American Naturalist, 150(2):179-22) 100- 1997.
Diane Bouchacourt and Marco Baroni. Miss tools and mr fruit: Emergent communication in agents learning about object affordances. In Proc. of the Association for Computational Linguistics (ACZ.) 2019.
Henry Brighton and Simon Kirby. Understanding Linguistic Evolution by Visualizing the Emergence of Topo- graphic Mappings. Artificial Life, 12(2):229-242, 04 2006. ISSN 1064-5462. doi: 10.1 1622/61l. 2006. 12.2. 229.
Ted Briscoe. Linguistic evolution through language acquisition. Cambridge University Press, 2002.
Lindell Bromham, Xia Hua, Thomas G. Fitzpatrick, and Simon J. Greenhill. Rate of language evolution is affected by population size. Proc. of the National Academy of Sciences. 112(7):0097-2102, 2015. ISSN 0027-8424. doi: 10. 1007:/paa.141990411111
Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, and Marco Baroni. Anti-efficient encoding in emergent communication. In Proc. of Advances in Neural Information Processing Systems (NeurlPS), 2019.
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Composi- tionality and generalization in emergent languages. In Proc. of the Association for Computational Linguistics (ACL), 2020.
Edward Choi, Angeliki Lazaridou, and Nando de Freitas. Multi-agent compositional communication learning from raw visual input. Proc. of International Conference on Learning Representations (ICLR), 2018.
Michael Clyne. Linguistic and sociolinguistic aspects of language contact, maintenance and loss. Maintenance and loss of minority languages, 1:17, 1992.
Michael Cogswell, Jiasen Lu, Stefan Lee, Devi Parikh, and Dhruv Batra. Emergence of compositional language with deep generational transmission. arXiv preprint 1rX4v:1904.000 090699077 2019.
Kevin Denamganai' and James Alfred Walker. On (emergent) systematic generalisation and compositionality in visual referential games with straight- throuuhh gumbel-sofimax estimator. arXiv preprint 17X1v:2012.10776. 2020.
Matthew S. Dryer and Martin Haspelmath (eds.). WALS Online. Max Planck Institute for Evolutionary An- thropology, Leipzig, 2013. URL https: /wwwwwaa.a..e info/.
Nick C Ellis. The dynamics of second language emergence: Cyeles of language use, language change, and language acquisition. The modern language journal, 92(2):232 249, 2008.
Nicolas Fay and T Mark Ellison. The cultural evolution of human communication systems in different sized populations: usability trumps lcarnability. PloS one, 888)e71781, 2013.
10

---

Published as a conference paper at ICLR 2022
Nicole Fitzgerald. To populate is regulate. arXiv preprint arXiv:1911.04362, 2019.
Bruno Galantucei and Simon Garrod. Experimental semiotics: review. Frontiers in human neuroscience, 5:
2011.
Simon Garrod, Nicolas Fay, John Lec, Jon Oberlander, and Tracy Maclcod. Foundations of representation: Where might graphical symbol systems come from? Cognitive science, 31:961-87, 11 2007. doi: 10.1080/ 0364021070170366
Rick Dale Gary Lupyan. Language structure is partly determined by social structure. PLoS ONE 5, 1, 2010.
Laura Graesser, Kyunghyun Cho, and Douwe Kiela. Emergent linguistic phenomena in multi-agent communi- cation games. In Proc. of Empirical Methods in Natural Language Processing (EMNLP), 2019.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9()): 135511735 1997.
Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni Compositionality decomposed: how do neural networks generalise? Journal of Artificial Intelligence Research, 67:757-795, 2020.
Judy Kegl. The nicaraguan sign language project: An overview. Signpost, 7:24-31, 1994.
Eugene Kharitonov, Rahma Chaahouni, Diane Bouchacourt, and Marco Baroni. Egg: a toolkit for research on emergence of language in games. In Proc. of Empirical Methods in Natural Language Processing (EMNLP), 2019.
Eugene Kharitonov, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni. Entropy minimization in emergent languages. In Proc. of International Conference on Machine Learning (ICML), 2020.
Simon Kirby. Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity. IEEE Transactions on Evolutionary Computation, 5(2):102-110, 2001.
Simon Kirby and James Hurford. The emergence of linguistic structure: overview the iterated learning model. Simulating the evolution of language, pp. 121 121-147, 2002.
Simon Kirby, Hannah Cornish, and Kenny Smith. Cumulative cultural evolution in the laboratory: An experi- mental approach to the origins of structure in human language. In Proc. of the National Academy of Sciences of the United States of America (PNAS). National Academy of Sciences, 2008.
Simon Kirby, Monica Tamariz, Hannah Comish, and Kenny Smith. Compression and communication in the cultural evolution of linguistic structure. Cognition, 141:87-102, 2015.
Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralha, and Sanja Fidler. Skip-thought vectors. Proc. of Advances in Neural Information Processing Systems (NIPS), 2015.
Stephen Kokoska and Daniel Zwillinger. CRC standard probability and statistics tables and formulae. Cre Press, 2000.
Satwik Kottur, José Moura, Stefan Lee, and Dhruv Batra. Natural language does not emerge 'naturally' in multi-agent dialog. In Proc. of Empirical Methods in Natural Language Processing (EMNLP), 2017.
Angeliki Lazaridou and Marco Baroni. Emergent multi-agent communication in the deep learning era. arXiv preprint arXiv:2006. 02419, 2020.
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of linguistic commu- nication from referential games with symbolic and pixel input. In Proc. of International Conference on Learning Representations (ICLR), 2018
Vladimir Levenshtein et al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physies doklady, volume 10:8, pp. 707-710, Soviet Union, 1966.
David K. Lewis. Convention: A Philosophical Study. Wilcy-Blackwer 1969.
Fushan Li and Michael Bowling. tasc-of-tcaching and language structure from emergent communication. In Proc. of Advances in Neural Information Processing Systems (NeurIPS), 2019.
Yuchen Lu, Soumye Singhal, Florian Strub, Aaron Courville, and Olivier Pietquin. Countering language drift with seeded iterated leamnnng In Proc. of International Conference Machine Learning (ICMI.), 2020.

---

Published as a conference paper at ICLR 2022
Karen McComb and Stuart Semple. Coevolution of vocal communication and sociality in primates. Biology
letters, 1(4):381-385, 2005.
Irit Meir, Assaf Israel, Wendy Sandler, Carol Padden, Mark Aronoff. influence community on language structure: evidence from two young sign languages. Linguistic Variation, 12(2):247- 2012.
Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations.
In Proc. of Conference on Artificial Intelligence (AAAI), 2018.
Daniel Nettle. Social scale and structural complexity in human languages. Philosophical Transactions of the Royal Society Biological Sciences, 167(1159))18229-88888 2012. doi: 10. 1..1099/sthh.211..000
Limor Raviv, Antje Meyer, and Shiri Lev-Ari. Larger communities create more systematic languages. Proc. of the Royal Sociery B: Biological Sciences, 286( 286(1997) 2019a. doi: 00.1099/ssb.2219.122.
Limor Raviv, Antje Meyer, and Shiri Lev-Ari. Compositional structure can emerge without genera-
tional transmission. Cognition, 182:151-164, 2019b. ISSN 0010-0277. doi: https://doi.or//0000006
L.cognition.2018.09.000 URL https:/ wwww sclencedirett. www/ httm///ccccc/arrcc///p... 50010027718302464
Limor Raviv. Antje Meyer, and Shiri Ari. The role of social network structure the emergence of linguistic structure. Cognitive Science, 44(8):e 12876, 2020.
Florencia Reali, Nick Chater, and Morten H. Christiansen. Simpler grammar, larger vocabulary: How popu- lation size affects language. Proc. of the Royal Society B: Biological Sciences, 285(1871):2017256, 2018. doi: 10. 11..1/0//...20...2.... 2017.288..
Yi Ren, Shangmin Guo, Matthicu Labeau, Shay Cohen, and Simon Kirby. Compositional languages emerge neural iterated learning model. In Proc. of International Conference on Learning Representations (ICLR), 2020.
Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M. Dai, and Kyunghyun Cho. Capacity, bandwidth, and compositionality in emergent language learning. In Proc. of Autonomous Agents and Multiagent Systems (AAMAS), 2020.
Mathieu Rita, Rahma Chaahouni, and Emmanuel Dupoux. "Lazimpa"" Lazy and impatient neural agents learn to communicate efficiently. In Proc. of the Conference on Computational Natural Language Learning (CoNLL), 2020.
Wendy Sandler, Irit Meir, Carol Padden, and Mark Aronoff. The emergence of grammar: Systematic structure in new language. Proceedings of the National Academy of Sciences, 102(7):2661-266) 2005.
Kenny Smith, Simon Kirby, and Henry Brighton. Iterated learning: A framework for the emergence of lan- guage. Artificial life, 94):371-386, 2003. 2003.
Luc Steels. The synthetic modeling language origins. Evolution of communication, 1(1):1-34, 1997.
Richard Sutton, David A McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In Proc. of Advances in Neural Information Processing Systems (NIPS), 2000.
Mónica Tamariz and Simon Kirby. Culture: copying, compression, and conventionality. Cognitive science, 39 (1):171-183, 2015.
Olivier Tieleman, Angeliki Lazaridou, Shibl Mourad, Charles Blundell, Doina Precup. Shaping represen- tations through communication: community size effect in artificial learning systems. Visually Grounded Interaction and Language ( (ViGII.) Workshop, 2019.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau. Evgeni Burovski, Pearu Peterson, Warren Weekesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Erie Jones, Robert Ker,, Eric Larson, CJ Carey, Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientifie Computing Python. Nature Methods, 17:261-272, 2020. doi: 10. 1000008/4411----990066--
Elliott Wagner. Communication and structured correlation. Erkennmis,, 71(3):3773333, 2009.
12

---

Published as a conference paper at ICLR 2022
Kyle Wagner, James A Reggia, Juan Uriagereka, and Gerald S Wilkinson. Progress in the simulation of emer- gent communication and language. Adaptive Behavior, 11(1):37-6 2003.
Gerald S. Wilkinson. 12. Social and Vocal Complexity in Bats:. Harvard University Press, 2013.
Alison Wray and George W. Grace. The consequencees of talking to strangers: Evolutionary corollaries of socio- cultural influences linguistic form. Lingua, 117(3)5543-578, 2007. ISSN 0024-3841. The Evolution of Language.
Lukasz Kuciniski, P. Kolodziej. and Piotr Milos. Emergence of compositional language in communication through noisy channel. Language Reinforcement Learning (LaReL), ICML Workshop, 2020.
13

---

Published as a conference paper at ICLR 2022
A RANDOM BASELINES
We report the values of speakers synchronization, neg-entropy, compositionality (topographic simi- larity) and generalization (test accuracy) for randomly-initializen and untrained agents:
Table 2: Average speakers synchronization, neg-entropy, compositionality and generalization for randomly- initialized and untrained agents. We report the standard deviation across 10 seeds
Experiment
Sp. syne
Beg-Entropy
Compo.
Gene.
Random agents
0.09 + 0.03
-88.4 + 0.23
0.04 + 0.02
0.24 + 0.02
B NON-CONFOUNDING PARAMETERS
Speakers synchronization
Neg entropy
Topographic arity
accuracy
10
0.8
1.00
0.9
-02
0.55
-0.6
-0.6
0.6
-0.8
10-1
10-1
10
P
Figure 5: Variations of language metries with increasing values of gradient update probability p.
In Figure 5, we show the variations of language scores with increasing values of speaker probability update Ps and listener probability update PL. while keeping them identical (p Ps PL). In Table 1. we show the Spearman correlation between the metrics and We do not observe significant correlation. In addition, both the relative and absolute variations of the parameters negligible compared to experiments displayed in Figure 2.
C VARYING SETUP PARAMETERS WHEN STUDYING SPEAKER-LISTENER ASYMMETRIES
In this Section, we show how trends displayed in Figure 2 evolve when performing some changes of parameters in the setup. As shown in Figure 6, when modifying agents' hidden sizes (speaker's hidden size is equal to 64 and listener's hidden size is equal 512 while they are both equal 128 in Figure 2), the trends are the same. However, we notice a shift of the curves along the x-axis. It means that the ratio of learning speed at which metrics start becoming optimal controlled by the parameters of the setup. Figure 6 highlights that the trends are sensitive to agents' capacity.
Language similarity
Neg-Entropy
Topographic simil arity
accuracy
0.7
0.6
0.4
0.5-
0.8
0.7
10a1
30
15
10
15c1
30a
10
10'
10-
10a
10
10
107
10
12
Pipned
Repand
Plpaadd
Prpend
Figure 6: Emergent language properties function of speed ratio Pspeed with N 2. From left to right on the x-axis, the speaker have higher training speed while the listener have lower training speed. Compared to Figure 2. we have changed the initial parameters of the problem by modifying agents' hidden sizes. Here: speakers' hidden size is equal to 64 and listeners' hidden size is equal 512
14

---

Published as a conference paper at ICLR 2022
D OTHER SPEAKER-LISTENER ASYMMETRIES INFLUENCING EMERGENT LANGUAGE PROPERTIES
In complement of Section 5.2, show in this Appendix that trends observed in Figure 2 can be reproduced with other speaker-listener asymmetry parameters. In particular, we study the ratio between speaker's capacity and listener's capacity in minimal size population (N 2) in Ap- pendix D.I and the ratio between the number of speakers and listeners in Appendix D.2.
D.1 AGENTS' CAPACITY
2.8 Speaicess spochrenvation
Neg-nttmy
Teposraphic simiarity
18
Gettrallation
402
355
3.52
Figure Variations of language metries with increasing values of agents hidden size
1.0
0.8
1.0
5-
0.9
0.6
0.8
0.5
-15
0.4
20
10-1
10a
10
10a4
10c
10
10-1
107
16-
10:4
10
10
Figure 8: Emergent language properties as a function of capacity ratio Peapacity with N = 2. From left to right the x-axis, the speaker have higher capacity while the listener have lower capacity.
We here reproduce the trends observed in Figure 2 with another speaker- listener asymmetry. For- mally, we introduce Prapacity := 1/ / where S (resp. h=) is speaker's 's hidden size (resp. lis- tener's 's hidden size). For the following experiments, we compute this ratio with multiple values of h /h- l for h E {64, 128, 256, 512, 1024).
The absolute capacity of agents has no significant impact on language properties. As done in Section 5.2, we first perform a sanity check and verify that altering h and h while keeping h = h² =: h does not fundamentally change the language properties per itself. We sweep over while setting minimal population size of N 2, and compute the Spearman correlation in Table 3. We observe that h have no statistically significant correlations with speaker synchronization, com- positionality and generalization (Spearman correlation inferior 0.3 or p-values above 0.05). The only noteworthy correlation is between & and the neg-entropy with a Spearman correlation of 0.89 and a p-value< 0.05. However, as shown in Figure the variations are two orders of magnitude lower than the neg-entropy variations in Figure 8. Therefore, the impact of this correlation is small enough in our experiments to neglected.
Table 3: Spearman Correlation between h, p and language scores. p-values 0.05 are underlined. Sp. sync
Param.
Sweep
Neg-Entropy
Compo.
Gene.
h
(64, 128, 256, 512, 1024 -0.14
0.89
0.29
0.11
The relative speaker-listener capacity significantly affects language properties In Figure 8, we see that when varying Pcopocstys we get trends similar to those observed in Figure 2. In Table 5, compute the Spearman correlation between the metrics and Prapacity and notice that there is significant correlation between Pcapacity and studied metrics.
Network capacity may be confounded with training speed. While the effect of training speed may be partially intuited in Section 5.2, the impact of capacity more tedious to analyze. As we have already observed that training speed is crucial parameter, thus test whether correlation
15

---

Published as a conference paper at ICLR 2022
between network capacity and learning speed can be established. In Figure 9 (resp. Figure 10), pair a fixed pretrained listener (resp. speaker) with newly initialized speakers (resp. listeners) of increasing capacities and see how fast those new speakers (resp. listeners) reach convergence. We observe that all trained agents converge to the same accuracy. However, we also note that they have different convergence speeds. Speakers (resp. listeners) of capacity 512, 128, 32 respectively have to see 100, 250, and 900 batches (resp. 50, 100 and 250 batches) to reach train accuracy of 95%. In short: the larger the network, the faster the training speed. We conclude that network capacity may be partially explained by the indirect impact of capacity on learning speed.
Seed
2
Suad
0.8
52
52
0.c
128
1000
1900
540
1000
156m
2003
550
1000
1500
2000
Boccho ssen
Selcts seen
S4tera sean
1.0
122
220
251
250
5:2
512
7, 532
1924
Ry- 1024
7,- 1021
500 1000
1300
2012
300
1000
1500
2003
550
1200
3350
2000
Figure 9: Training accuracy of speaker-listener pairs where the listener is pretrained and fixed. For all curves, a listener has been pretrained with a random speaker, frozen and paired with new initialized speakers of increasing sizes. Each row corresponds to a new pretrained listener each column to a new initialization of the speakers. Please note that some limit cases appear with the pure RL task, i.e. fixed listener. Speakers' convergence become unstable across seeds for small networks (hs = 32). Yet, the training is still stable in supervised task, i.e. fixed speaker, in Figure 10.
:20
120
250
236
h, 112
512
312
h. 1004
n. 1124
bu 2024
1.70
2015
200
400
102
2003
300
800
100
330
400
Seso
Seed
5eed
0.5
A 33
0.8
32
32
178
178
124
Butch 2007
B1.0% 3000
Training accuracy for speaker- listener pairs where the speaker is pretrained and fixed. For all curves, speaker has been pretrained with a random listener, frozen and paired with new initialized listeners of increasing sizes. Each row corresponds to new pretrained speaker ; each column to a new initialization of the listener
Figure 10:
D.2 ALTERING THE RATIO BETWEEN THE NUMBER OF SPEAKERS AND LISTENERS
In complement to Papeed in Section 5.2 and Poapanity in Appendix D.2, we extend the results to another population factor and intuit how it relates to learning speed. We introduce the ratio between the number of speakers and the number of listeners within a population Pagents == Nitateners Note Napcukers
that this ratio cannot be applied to a single speaker-listener pair. Yet, it introduces an asymmetry between the community of speakers and the community of listeners. We show the evolution of
16

---

Published as a conference paper at ICLR 2022
Speakers syn chronization
Neg Entropy
Compositonality
Generalization
0.6
0.75
0.50
0.4
0.4
0.25
0.2
10-1
16a
-10
10
10
10
16
Payones
Pagents
Papenes
Poponts
Figure 11: Language metrics for different ratio between N Napeakers and Nisteenrs where N Napeakers and Nissteners are the range {1, 4, 8, 16, 24, 50, 75, 100). Experiments are run with || and [V] = 10.
language metrics Figure 11. There, we observe than unbalancing N Speakers and Nisteners has strong impact language scores. Speaker synchronization reach upper and lower bound depending on the ratio, neg-entropy either is low or close to 0, and compositionality and generalization increase. Pagents is thus additional control factor of language properties. Extreme cases coincide with the training speed interpretation of Section 5.2. Indeed, at each iteration of the game, each speaker (resp. listener) has probability 1/N s/Nspeekers (resp. 1/Nissteners) be sampled. Thus, speakers have Nusteners/ /Neprkerss more of less learning steps than listeners. Consequently, when Pagents > 1, individual speakers (resp. listeners) have way more optimization steps than individual listeners (resp. speakers) and we fairly hypothesize that speakers (resp. listeners) train faster than listeners (resp. speakers). This ratio may therefore be confounding factor of a training speed mechanism. Note that, interestingly, Pagents can also be related to different sampling procedures, or broadcasting phenomena.
ADDITIONAL CORRELATIONS
Table 4: Spearman Correlation between community size and language scores for homogeneous and heterogeneous populations (1, 1, o, 1). These correlations related respectively to Figure and Figure 3. We underline correlation with < 0.05.
Experiment
Sp. sync Neg-Entropy Compo.
Gene.
Homogeneous population
-0.44
-0.52
-0.15
-0.14
Heterogeneous population
0.55
0.24
0.21
-0.04
Table 5: Spearman Correlation between controlling factor ratios and language scores in Figure 2.
We underline correlation with < 0.05.
Param. Sp. sync
Neg-Entrops
Compo.
Gene.
Popeed
0.96
0.97
0.63
0.34
Peapacity
0.49
0.77
0.33
0.09
F COMPLEMENTARY SETTING FOR POPULATION RESULTS
F.1 DIFFERENT INPUT SPACE
We first reproduce our core experiments different Lewis game setting, with |K] attributes containing each V 10 values. Results are displayed in Figure 12 and Figure 13 that respectively reproduce the results of Figure 1 and Figure 3. We observe that the main trends observed in the main paper unchanged when varying parameters of the input space.
F.2 DIFFERENT HETEROGENEITY DISTRIBUTION
We then vary the distribution agent update probability p within the population to evaluate how the observed trends sensitive the heterogeneity distribution.
17

---

Published as a conference paper at ICLR 2022
synchrorization
Neg-Entropy
Compostionalit
General
0.9
-95
-1.
0.6-
-2
0.5
0.2
10
10
20
10
15
20
10
15
20
N
N
N
Figure 12: Language scores with homogeneous population with N (1, 4, .... 20 |
and [V| 10. Corresponding figure with |K] 4 and [V] = 4 in the main paper is Figure
Speakers synchronizsiin isation
0.0
Negert
Compositonality
0.7
Generalization
-0.5
0.5
0.55
0.5
0.3
0.4
0.45
10
20
10
20
5
15
20
15
20
N
Figure 13: Language scores with heterogeneous population with N (1, 4, .... 20), 4/p -1,
dp = 0.5, K = and V = 10. Corresponding figure with | = 4 and [V] = in the main
paper Figure
Log-normal distribution with lower standard deviation We first test how language properties affected by a variance reduction. In Figure 14 agent update probability p sampled according -1 and 0.5. Compared to Figure 3, concentrated heterogeneity distribution. We see that speakers synchronization and neg-entropy increases are slower and that composition- ality increase now very low. Generalization remains almost unchanged. This plot suggests that the distribution of heterogeneities must be minimally wide to observe positive correlation with population size and that all metrics do not have the same sensitivity to heterogeneities.
to a log-normal distribution Log-N( Tp) with
standard deviation has been divided by 2 meaning
Speakers synchronization
Neg-Entrapy
Compositionality
Generalization
0.73
-0.2
-0.50
-0.75
0.90
Figure 14: Emergent language properties of heterogeneous populations of increasing sizes. Distri-
bution of agent update probability p Log-N (1p. ap) with 7pp and op 0.5
Distribution change We now test how language properties are affected by a distribution change. In Figure 15, agent update probability p are sampled according to Beta distribution (1,, 2). Vari- ances of (11, 2) and Log-N(-1) (11, 0.5) (Figure 14) have the same order of magnitude. As we can see, trends are similar for Figure 14 and Figure 15. Trends thus not notably affected by changing the log-normal distribution by a distribution of similar variance.
18

---

Published as a conference paper at ICLR 2022
Speakers synchronization
Nep-Entropy
Compositionality
Generalization
0.8
0.95
0.4
-1.5
0.80
10
20
N
Figure 15: Emergent languages properties of heterogeneous populations of increasing sizes. Agent update probability follows a 3 distribution (11
19


Reviews:
[{"review_id": "erajOuI5c88", "reviewer": "Reviewer_UKkL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n# Summary\n\nThis paper analyzes the structure of emergent languages in signaling games played by _populations_ of agents, motivated by a rich body of socio- and psycho-linguistics results showing that languages spoken by more people (and with more second-language learners) tend to be grammatically simpler (e.g. have a more impoverished morphology).  The authors measure various properties of the emergent languages as proxies for how \"systematic\" a language is, and show several things.  First, increasing the number of agents does not correlate with any of their measures, so population size alone does not suffice.  Second, in a minimally small population, various measures of \"diversity\" of the two agents do correlate with their systematicity measures.  Finally, in a large but diverse population, we do see correlations with population size and _some_ of the systematicity measures.  The paper is interesting and timely, and reports on a large number of experiments.  While I think the experiments could be more closely linked to the hypotheses from the sociolingusitic literature, the paper will be of interest to many researchers in emergent communication, NLP, and cognitive science, and could spur future work in this intersection.\n\n\n* Strengths:\n\t- Thorough and competently executed experiments\n\t- Largest-scale study of the impact of population size on emergent language structure\n\n* Weaknesses:\n\t- The experiments could be motivated more precisely from the cited literature (see comments below)\n\t- Experimental results were not always clearly reported (see comments below)\n\t\n\n\n# Comments / questions:\n\n* The cited Raviv et al studies crucially manipulated not just population size, but also _network structure_ (e.g. scale-free vs complete graphs). While it is completely fine for the present paper to only focus on population size and the homogeneity of the population, I would encourage the authors to at least mention this important extra dimension of the Raviv et al papers.  As presently written, the reader can get the impression that those studies only look at population size.  This could also be re-mentioned in Section 3.2, where the authors helpfully point out some of their modeling assumptions.\n\n* I would also encourage the authors to cite Wray and Grace 2007 \"The consequences of talking to strangers\" (https://www.sciencedirect.com/science/article/pii/S0024384105000999) in their related work section.\n\n* Why did the authors choose the full object-reconstruction loss as opposed to a choose-among-distractors loss (as is slightly more common on the emergent communication literature)?  Did they try the latter and find similar results, or do the results depend on the object-reconstruction setting?\n\n* Was there a reason not to use the (straight-through) Gumbel Softmax trick for training here?\n\n* Presentation of Entropy in \\S3.3: (i) equation (5) is negative entropy, as the authors note at the end of the paragraph.  But the equation is introduced as just being entropy.  I would encourage them to make the definition (e.g. of $h$) include the negative sign, so that it's normal entropy, and then just mention that they report negative.  (ii) I don't know what it means to say that minimizing entropy \"reinforces the information bottleneck principle\", since the latter is about two forms of mutual information.  I'd like to either hear something more explicit/detailed here or have it dropped.\n\n* Reporting results: (i) I would like to see actual numbers and a statistical analysis for e.g. the results in 5.1 / Fig 1, even if in an appendix (as is done in S5.3).  (ii) Similarly, they write: \"In addition, we do not observe any language systematicity. By systematicity, we mean languages whose properties are stable across seeds.\"  It's hard to know what exactly the authors did and are reporting here, so a little more detail would be welcome.  (iii) I think a baseline of randomly-initialized / untrained agents for these metrics would be informative to see.\n\n* p 7: \"language neg-entropy is reaching almost 0, meaning almost a bijective language mapping\".  I might be misunderstanding, but won't your entropy also be minimized by a _uniform_ language, that outputs the same sequence of symbols with probability 1 for every object?  Since the entropy is taken over next-tokens in the sequence in average over each object, I don't see any pressure for distinct messages for distinct objects in that measure, which is what a bijective mapping sounds like to my ear.  In other words: doesn't your measure of entropy just measure how _deterministic_ the mapping from objects to messages is, not necessarily how bijective it is?  I may be misunderstanding, but it would help to clarify if so.\n\n\n# Typographic comments:\n\n* p 3: \"community size and leaning properties\" --> \"community size and learning properties\"\n\n* p 4, eqn (4): \"$m_i \\sim \\pi_{\\theta_1}(\\cdot | v)\": the \"\\theta_1\" here should be \"\\theta_2\"\n\n* Caption on Fig 2: the \"x\" in \"x-axis\" should be in math mode\n\n* page 7: the quotation marks around \"fast enough\" should be fixed to ``fast enough'' (and same for \"ease-of-teaching\")\n\n* page 9: \"actually be to simplistic\" --> \"actually be too simplistic\"", "summary_of_the_paper": "This paper analyzes the structure of emergent languages in signaling games played by _populations_ of agents, motivated by a rich body of socio- and psycho-linguistics results showing that languages spoken by more people (and with more second-language learners) tend to be grammatically simpler (e.g. have a more impoverished morphology).  The authors measure various properties of the emergent languages as proxies for how \"systematic\" a language is, and show several things.  First, increasing the number of agents does not correlate with any of their measures, so population size alone does not suffice.  Second, in a minimally small population, various measures of \"diversity\" of the two agents do correlate with their systematicity measures.  Finally, in a large but diverse population, we do see correlations with population size and _some_ of the systematicity measures.  The paper is interesting and timely, and reports on a large number of experiments.  While I think the experiments could be more closely linked to the hypotheses from the sociolingusitic literature, the paper will be of interest to many researchers in emergent communication, NLP, and cognitive science, and could spur future work in this intersection.", "summary_of_the_review": "The paper provides a rich set of experiments looking at the effect of population size and diversity of emergent languages, and so will be of interest to many working at the intersection of emergent communication in NLP and cognitive science.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bgdUSa1O7k", "reviewer": "Reviewer_grQ9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n# Main assessment \nThis work is interesting and the topic well worth exploring. The contribution to the area of deep language emergence is very clear and timely. That being said, I do have some concerns. First, I think much more care needs to be put in positioning this paper in the literature.  There is a large body of (non-neural) research on the emergence of compositionality, structure, and generalization that emphasizes that a \"transmission bottleneck\" (e.g., agents not having the capacity to fully observe/learn the language of their community) is what makes these properties emerge. After all, that's why these properties are useful in the first place: to overcome agents' limitations when learning/imitating/using language. Similar arguments have been put forward for other properties of natural languages, such as ambiguity and vagueness. The authors' main argument follows the same line of reasoning but does not engage with this literature, nor synthesize what we already learned from it. A good starting point is offered by the literature on iterated learning, some of which the authors cite but for other purposes. Another starting point is given by the research that more closely follows Lewis' original setup (e.g., work by and building on \"Signals\" by Skyrms 2010; or Wagner's \"Communication and Structured Correlation\"). Second, and more importantly still, while I found the narrative of the paper convincing, I also found the main results rather weak. The bars in Figure 4 strongly overlap. This suggests that there is so much variation between runs that the means are not very trustworthy to draw strong conclusions from. The correlations for heterogeneous populations in Appendix C's Table 2 are analogously weak. I worry that noise may be at play; and an unjustified significance threshold of 0.05 does not dispel this concern. \n\n# Minor comments\nThere are some issues with the terminology used. The main issue the paper addresses is not a paradox (p. 1) nor is there a contradiction (p. 5).\n The question why population size predicts\nstructure and complexity in natural language but not in neural language emergence is interesting. However, there is nothing paradoxical or contradictory about it. At most, one may find this puzzling. Similarly \"predictive\" does not need to mean \"causal\" (p. 2) and \"particularly significant\" is a vacuous term (p. 9). This may sound nit-picky but these are all technical term.\n\nI could not quite follow all the details of the equations in Section 3. Part of the reason is that not all symbols are clearly defined.\n For instance, from the text above Equation (1), it's unclear whether T is a set (since its cardinality, |T| is mentioned) or a number (which is what I am led to believe from the formulae that follow, or the text [e.g., p. 5]).\n The equations look reasonable to me but it would be nice if this part of the manuscript was easier to follow.", "summary_of_the_paper": "# Summary \nThe manuscript \"On the role of population heterogeneity in emergent communication\" addresses the question why results on population size in deep language emergence have not, so far, mirrored the effects it is claimed to have on natural language. In a nutshell, in natural communication population size correlates with simpler grammars and less idiosyncratic languages. The effects of population size on the structure of neural emergent communication is less explored and has hitherto not reflected what we know about natural language. The authors argue that one of the reasons for this is that artificial populations are often homogeneous. They show, through simulations, that introducing asymmetries in the training speed of speakers and listeners leads to trends that are more in line with natural language: the size of heterogeneous populations weakly correlates with more aligned languages; higher neg-entropy; better generalization when communicating about novel objects; and more compositional languages (measured as topographic similarity).\n", "summary_of_the_review": "This manuscript is on the target. It is interesting and the research is well worth pursuing. However, it has crucial issues that need to be addressed to better appreciate its contributions and scope. In particular, the results do not lend much support to an issue that has already received attention in related (non-neural) research on language emergence.\n\n\n# Recommendation update\n\nIn light of the authors' changes and discussion, I have updated my recommendation. The literature review is much more complete now, and I think weakening some of the statements has made the contribution much clearer. I am still concerned about the rather weak effects and correlations reported, but now the framing is much more in line with the content.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XcsiUxRoLo", "reviewer": "Reviewer_zxeP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- This is a well-written paper with a very engaging flow.\n- This paper provides important insights for language emergence research in machine learning.\n\nWeaknesses:\n- The largest effects of heterogeneity are on speakers synchronization and negative entropy; there is low, no, or negative effect on compositionality and generalization. The paper is overclaiming to say that the emergent language is more structured with heterogeneity when the compositionality and generalization (metrics we arguably care about more) are not significantly affected with increasing population or speaker-listener asymmetry (when $\\rho_\\bullet \\geq 10^0$).\n- The claim that network capacity is a confounding factor of training speed relies on the speed of achieving 95% training accuracy (related: Figure 2 left of [1]). However, is this sufficient to claim that the effect is equivalent in the setting you are studying? How models fit training data and how they generalize to unseen data can have different dynamics. Do these \"equivalent\" models (fast model with high model capacity and slow model with low model capacity) achieve similar values on the four language property metrics in the paper when they get to 95% training accuracy?\n\nOther questions:\n- When you split the data into train and test, did you try splitting them systematically? It would be interesting to evaluate whether the emergent language can generalize OOD.\n- I find this hypothesis interesting: \"extreme agents, especially slow listeners, behave as kinetic bottlenecks, forcing the speakers to structure their languages. Thus, language emergent properties would be particularly determined by fast speakers and/or slow listeners.\" Is there any intuition for why the relative speed differences would have this effect?\n\nSuggestions:\n- In Section 3.4 Control parameters of local asymmetry: Typo: should be $p^S$ instead of $p_S$ in the fifth line of the paragraph.\n- In Section 3.4 Distributing Population Heterogeneity: \"update probability factor\" in the last line is not clear until later in the paper.\n- Figure 3 caption: after \"pretrained with a speaker\", add \"(resp. listener)\"?\n- Section 5.2: Typo: \"statically\" -> \"statistically\".\n\nReferences:  \n[1] Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J. and Amodei, D., 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.", "summary_of_the_paper": "The authors refer to prior work in sociolinguistic literature to state that larger communities create more systematic languages. However, they point out, this apparent correlation between language structure and population size has evaded machine learning practitioners studying language emergence. This paper claims that populations explored in machine learning have largely been homogeneous and that population heterogeneity is key to the emergence of structure in artificial agents. The authors reproduce the failure to achieve systematic languages by scaling up the population and then show that they can indeed get more structure by introducing heterogeneity. They explore introducing asymmetry between the speaker and the listener based on model capacity and learning speed, leading to an increase in language structure when the speaker is faster or has more capacity. The authors note that this effect only depends on the relative differences between the speaker and the listener and not the absolute values (there is a correlation with absolute values but the variation has a low magnitude). Further, they show that larger networks need fewer epochs to reach similar training accuracy, concluding that network capacity is a confounding factor of training speed in their setting. Finally, the authors create a population of heterogeneous agents by imbibing them with different learning speeds by updating an agent $i$ with probability $p_i$ after each round of the Lewis game. Four properties of the emergent language are studied: speakers synchronization, (negative) conditional entropy given an object, topographic similarity, and generalization. These metrics either improve or remain approximately at the same values as the population size increases.", "summary_of_the_review": "I believe this is an important paper to encourage future research into heterogeneity as a component for language emergence. The paper is well-written and the authors are able to scale up the Lewis game setting without observing a decline in desirable language properties. However, the paper in its current form makes claims that cannot be substantiated in the evidence, which I find to be a significant barrier for me to recommend acceptance.\n\nUPDATE AFTER REBUTTAL:\n\nAfter the discussion with the authors, the paper has toned down its claims and added clarity about what it does not claim: it does not present an overall better language emergence setup, and the goal is to primarily address the apparent decline in structure as population size goes up in machine learning in contrast to sociolinguistic studies. Thus, I am increasing my score for this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "EoLn29lJ__", "reviewer": "Reviewer_5n9H", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I read with interest this nicely written paper and I think that there is good evidence towards the authors' conjecture. Furthermore, the authors strive to produce some intuitive rationale to explain why heterogeneity leads to more structured language. I have a generally positive view of the work but, as a non-expert, I saw some weaknesses in the paper.      \n\nThe most important objective of the paper is to demonstrate that a larger heterogeneous population leads to higher quality language. I was expecting a large variety of scenarios showing the impact of this heterogeneity. However, a single scenario is considered for the training speed, by sampling p_i from a log-normal(eta, sigma) with eta=-1 and sigma=1/2. These parameters do not lead to large heterogeneity. The distribution of p_i has expectation equal to exp(-1+(0.5^2)/2)=0.42 with variance equal to 0.05. This seems a very concentrated distribution. Given the main objective of this paper, it is interesting to see what is the impact of more dispersed distributions. Maybe not only varying the log-normal parameters to create a large variance but even considering other distributions such as the more natural Beta(a,b) distribution for the parameter p_i (which is a probability). What would be the impact on the increasing trends of neg-entropy and synchronization (see Figure 4) of having larger heterogeneity in the population? \n\nAgain, given that the main objective is to show that larger heterogeneous populations leads to higher quality language, I missed a larger variation on the population size. It increases linearly from N=2 to N=20 (meaning 10 senders and 10 receivers). It is not possible to simulate with much larger populations, with a geometrical increase? We are likely to see a more clear effect if populations change in orders of magnitude. I missed a larger range for his variation. \n\nIn page 8 and wrt Figure 4, the authors say that \"compositionally and generalization have a small positive but statistically significant correlation with Spearman coefficients above 0.3 (p-values> 0:05)\". This is not correct. Appendix C shows that, for generalization in heterogeneous populations, the Spearman correlation index is equal to -0.07, a negative and very low value. It is hard to believe that this low value is statistically significant and, worse yet, it is negative. Indeed, there is not a visually clear presence of a trend with the N increase in the compositionality and generalization plots in Figure 4. \n\nAlso, the statistical test is significant when the p-value is small, traditionally smaller than 0.05. In the excerpt copied above and in several other passages throughout the paper, the authors mention that statistical significance corresponds to having a p-value larger than 0.05. It should be smaller, not larger. \n\n", "summary_of_the_paper": "This paper aims at solving a conflicting empirical observation and the present-state models for emerging languages. It has been observed that larger populations produce more structured languages. However, the state-of-art neural-based models have not been able to generate languages with such characteristics. This paper shows that a key ingredient is to allow population heterogeneity in the neural models rather than the current identically distributed specifications. ", "summary_of_the_review": "I have a generally positive view of the work but, as a non-expert, I saw some weaknesses in the paper.      ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors investigate the claim that agents in emergent communication games will converge to a symmetric homogenous state.  In particular, the authors show/argue for diversity in the population to close the gap between observed trends in neural agents and those expected when studying natural languages (e.g. around structure).  Reviewers were generally positive, though requested a number of rhetorical changes needed and additional literature.  These have been addressed.
Decision: Accept (Poster)