O N THE ROLE OF POPULATION HETEROGENEITY IN EMERGENT COMMUNICATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Populations have often been perceived as a structuring component for language to emerge and evolve: the larger the population, the more structured the language.
While this observation is widespread in the sociolinguistic literature, it has not been consistently reproduced in computer simulations with neural agents.
In this paper, we thus aim to clarify this apparent contradiction.
We explore emergent language properties by varying agent population size in the speaker-listener
Lewis Game.
After reproducing the experimental difference, we challenge the simulation assumption that the agent community is homogeneous.
We ﬁrst investigate how speaker-listener asymmetry alters language structure to examine two potential diversity factors: training speed and network capacity.
We ﬁnd out that emergent language properties are only altered by the relative difference of factors between speaker and listener, and not by their absolute values.
From then, we leverage this observation to control population heterogeneity without introducing confounding factors.
We ﬁnally show that introducing such training speed heterogeneities naturally sort out the initial contradiction: larger simulated communities start developing more stable and structured languages.
I NTRODUCTION
Language emergence has been explored in linguistics and artiﬁcial intelligence for two main reasons (Lazaridou & Baroni, 2020).
On the one hand, artiﬁcially reproducing language emergence may help to understand the evolution of human languages (Steels, 1997; Briscoe, 2002; Wagner et al., 2003).
On the other hand, language is known to be structured, and compositional (Bickerton, 2007), and imitating such properties would enhance machine learning representations.
As a result, there exists a constant back and forth between cognitive sciences, linguistics, and artiﬁcial intelligence to retrieve the core ingredients of language emergence (Kirby et al., 2008).
In this paper, we explore how the size of a population may impact the structure of emerging languages by using neural reinforcement learning methods.
Especially, we explore the following socio-linguistic hypothesis: larger communities create more systematic languages (Raviv et al., 2019a; 2020).
This hypothesis has been supported by a number of ethnographic (Gary Lupyan, 2010) and socio-linguistics (Raviv et al., 2020) observations as well as behavioral studies mimicking language emergence in a controlled setup (Raviv et al., 2019a).
Few neural language emergence papers have explored how community size impacts language structure so far, but the available evidence is mitigated at best.
Tieleman et al.
(2019) observed a small but consistent regularization effect when pairing auto-encoders within a population.
Similarly, Cogswell et al.
(2019) observed slight improvements in language compositionality with a large population but only reported them in few experimental settings.
Finally, Graesser et al.
(2019) studied the impact of contact-agents for different population sizes, but they did not observe a correlation between the population size and the convergence speed, success rate, or mutual agent intelligibility.
The following question arises: why does community size not improve language properties in recent emergent communication literature, although it is a key structuring factor in broader linguistics literature?
We argue that recent emergent communication models are limited as they ignore individual learning capacities by working only with homogeneous populations.
Consequently, they miss coupling effects emerging from agents’ asymmetries.
As a result, we hypothesize that community size effects could occur as soon as local heterogeneities are introduced into populations.
In this work, we explore the effects of population size with neural agents in the well-known Lewis referential game (Lewis, 1969).
In this game, a speaker describes a hidden object to a listener, which must then reconstruct object properties.
Both agents thus need to co-develop a communication protocol to solve the task.
The population-based variant of this game randomly pairs one speaker and one listener from their respective communities.
The goal is to observe whether increasing the number of agents enhances the communication protocol qualities, e.g. success rate, compositionality, generalization etc.
(Kottur et al., 2017; Chaabouni et al., 2020; Lazaridou et al., 2018).
Firstly, we reproduce Lewis reconstruction setting and conﬁrm the experimental difference: when increasing the number of agents, we do not observe improvements over various emergent language metrics.
We thus question the current paradigm to model population in the language emergence literature.
In particular, all agents are trained uniformly, i.e., their learning speed, capacity, sampling are identical (Tieleman et al., 2019; Cogswell et al., 2019; Fitzgerald, 2019).
Secondly, we evaluate the impact of two potential sources of model heterogeneities: (i) agents learning speed, (ii) agents capacities.
In both cases, we observe a similar pattern: the absolute value of speakerlistener speed and capacity is not important, yet their relative value is crucial.
We hence shed light on the strong correlation between language structures and agents relative training facilities.
Thirdly, we push this reasoning further by distributing learning speeds across the population thus creating heterogeneous populations.
We there observe an improvement of language scores combined with a variance reduction when increasing population sizes.
In other words, larger communities of neural agents start developing more stable and structured languages when being heterogeneously designed.
This observation brings a ﬁrst stone toward solving the empirical and computational contradiction.
Our experiments partially removed the apparent contradiction between the socio-linguistic observations and the recent emergent communication literature.
They illustrate how crucial population training dynamics are in shaping emergent languages and how population heterogeneity may have been underestimated in the recent emergent communication literature.
All in all, our contributions are three-fold: (i) we empirically show that the community size is not a structuring factor in language emergence by or in itself in the classic homogeneous Lewis setting; (ii) we give evidences that speaker-listener relative dynamics strongly affects language properties; (iii) we provide the ﬁrst computational cues to remove the apparent difference between socio-linguistic literature and recent neural emergent communication works.
R ELATED WORK
Population size in sociolinguistics.
Population size is a core parameter deﬁning the social environment an agent interacts with.
Its impact on language structures has largely been studied on humans (Nettle, 2012; Bromham et al., 2015; Reali et al., 2018; Raviv et al., 2019a) and animals (Blumstein & Armitage, 1997; McComb & Semple, 2005; Wilkinson, 2013).
By analyzing 2000 languages (Dryer & Haspelmath, 2013), a clear correlation was drawn between population size and diverse language features, e.g. larger communities tend to develop simpler grammars (Gary Lupyan, 2010; Meir et al., 2012; Reali et al., 2018).
As part of their research on the inﬂuence of network structures on language emergence (Raviv et al., 2019a;b; 2020), Raviv et al.
(2019a) went one step further by arguing that the community size is predictive of language structure and diversity.
To do so, they split 150 people into different groups of given community size to isolate confounding factors.
While people played a speaker-listener Lewis game, they observe that the greater the community size, the simpler and more consistent the generated language.
Here, we intend to test this assumption in the context of neural language emergence.
All things considered, we adopt a setting close to Raviv et al.
(2019a) ’s when computationally modeling human population.
Populations in experimental emergent communication.
Experimental language emergence has mainly been studied with two methods: behavioral studies (Kegl, 1994; Sandler et al., 2005) and simulations (neural and non neural)
(Wagner et al., 2003; Lazaridou & Baroni, 2020).
From behavioral studies and non neural simulations, two main approaches have emerged in the pastyears: experimental semiotics (Galantucci & Garrod, 2011;
Garrod et al., 2007) and iterated learning (Kirby & Hurford, 2002; Kirby et al., 2008; Beckner et al., 2017).
According to experimental semiotics studies, languages are mainly subject to an expressivity pressure; they argue that messages should be highly informative to allow communication within a group (Fay & Ellison, 2013). According to Iterated Learning paradigm, structures emerge from a compressibility pressure; they argue that memory limitations compel messages to become simpler to be easily learned (Tamariz & Kirby, 2015), which is also referred to as transmission bottleneck (Smith et al., 2003). Kirby et al.
(2015) then combined those two approaches and show that languages emerge as a trade-off between expressivity and compressivity during cultural evolution.
Recently, similar ideas have been modeled in emergent communication frameworks involving neural agents (Ren et al., 2020; Lu et al., 2020). Because seminal non-neural simulations used speciﬁc optimization methods and models across studies (Wagner et al., 2003), it is hard to generalize a global picture. Indeed, Wagner et al. (2003) keep listing standalone experiments.
Modern neural agents manage to simplify and standardize agents’ modeling.
It thus enables explore the determinants of different structural properties of human languages within a unique framework, paving the way for holistic models of emergence communication (Lazaridou & Baroni, 2020).
Our paper is related to this last set of neural works.
Recent works in emergent communication have been debating the prerequisites to the emergence of language universals such as compositionality (Li & Bowling, 2019; Ren et al., 2020; Lazaridou et al., 2018; Mordatch & Abbeel, 2018; Resnick et al., 2020; Kottur et al., 2017; Choi et al., 2018; Łukasz Kuci´nski et al., 2020), generalisation (Baroni, 2020; Chaabouni et al., 2020; Hupkes et al., 2020; Denamgana¨ı & Walker, 2020), efﬁciency (Chaabouni et al., 2019; Rita et al., 2020) or stability (Kharitonov et al., 2020; Lu et al., 2020).
Among them, a few works explored how different population structures may impact properties of emergent languages.
Inspired by iterated learning methods (Kirby et al., 2008), Ren et al.
(2020); Li & Bowling (2019)
; Lu et al.
(2020) look at language evolution across multiple generations of agent pairs, i.e. population is spread over time.
However, we consider here a population where multiple agent-pairs coexist simultaneously within a single generation.
There, Graesser et al.
(2019) show that community of agents start coordinating their language when at least three agents are present in the community.
They later assume that increasing the community size may impact the emergent shared language, but did not observe it in their initial experiments.
A similar hypothesis was also made by Bouchacourt & Baroni (2019) while analyzing the inﬂuence of symmetric agents in the emergence of a common language.
With different research objectives, Cogswell et al.
(2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission.
Analogously, Tieleman et al.
(2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers’ latent representations by increasing population size.
Eventually, Fitzgerald (2019) suggest that populations improve regularization and generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties.
In this paper, we aim at analyzing how population size affects those discussed properties.
We align with emergent communication literature and show that naively increasing community size does not consistently improve language properties.
We then challenge the homogeneity assumption made in all population designs.
M ETHOD
We here describe the different components of language emergence in a population-based Lewis
Game, namely, game rules, notations, training dynamics, and evaluation metrics.
Finally, we deﬁne how we alter population dynamics by asymmetrizing agents and injecting heterogeneities.
3. R ECONSTRUCTION G AME .
Game Rules:
We study emergent communication in the context of the Lewis referential games (Lewis, 1969).
There, a speaker observes all the attributes of an object.
The speaker then outputs a descriptive message, which a second agent, the listener, receives.
The listener must accurately reconstruct each value of each attribute to solve the task.
Both agents are ﬁnally rewarded in light of the reconstruction accuracy.
Note that another variant of this game requires the listener to retrieve the correct object within a list of distractors, but both settings are inherently similar.
Game Formalism: The observed object v ∈ V K is characterized by | K | attributes where each attribute may take |V| values.
We encode the observed object by a concatenation of one-hot representations of the attributes v k ∈ V of the object v for each attribute k ∈ K .
For each new run, the set of objects is split into a training set X and test set.
The intermediate message m ∈ W T is a sequence of T tokens, m = ( m t ) T − 1 t =0 where each token is taken from a vocabulary W of dimension |W| , ﬁnishing by a hard-coded end-of-sentence token EoS .
The speaker and listener are two neural agents respectively parametrized by θ and φ .
The speaker follows a recurrent policy π θ : given an input object v , it samples for all t a token m t with probability π θ ( m t | m <t , v ) .
We denote π θ ( m | v ) the probability distribution of the entire message given an input object v .
The listener outputs for each k attributes a probability distributions over the values V : π kφ ( v k | m ) .
At training time, the speaker message is generated by sampling the policy m ∼ π θ ( ·| v ) .
At test time we use the greedy message ˆ m t = arg max ¯ m π θ ( ¯ m | ˆ m <t , v ) .
Game Objective: As in (Chaabouni et al., 2020), we deﬁne the listener training goal to be the average of the multi-classiﬁcation log-likelihood loss per attribute:
[Equation]
In our setting, we want the speaker to optimize the same objective.
To do so, we deﬁne the speaker game reward as the negative loss of the listener,
[Equation]
Following the gradient policy theorem (Sutton et al., 2000), we maximize the speaker reward by minimizing the following objective over θ :
[Equation]
where m is sampled according to the speaker policy π θ .
3. P OPULATION -B ASED R ECONSTRUCTION G AME
We ﬁrst create a population of N speakers and N listeners, so we have a total number of agents of 2 N .
Following (Tieleman et al., 2019), at each step, we uniformly sample one speaker and one listener and pair them together.
We then proceed as in the classic one pair Lewis game: both agents play the game with a batch of inputs and receive an optimization step minimizing (Equation 3 & 1).
This operation is repeated until convergence, i.e., all speaker-listener pairs have stable test losses.
While standard, we note that this training procedure relies on strong latent assumptions: (i) each speaker (resp. listener) is uniformly sampled, i.e., there is no preponderant agent within the population, (ii) the communication graph is fully connected and uniform, i.e., all speakers may be paired with all listeners with the same probability, (iii) agents cannot be differentiated, i.e., agents have no information about the identity of their partners, (iv) speakers and listeners are all similar, i.e., there is no difference in the agent deﬁnitions nor in the optimization process.
Overall, those hypotheses create a homogeneous training setting.
In practice, the agents only differ by their initialization and optimization updates, e.g., stochastic agent pairing, game generations, and message sampling.
3. A SSESSING E MERGENT L ANGUAGE IN L EWIS G AMES .
We here introduce various metrics to assess emergent languages structure and quality.
To do so, we ﬁrst need to introduce two distances in the input/object space and in the message space.
We deﬁne the distance between two objects v and v (cid:48) as the proportion of common attributes computed by D obj ( v, v (cid:48) )
=
1 | K | (cid:80) k 1 { v k = v (cid:48) k } .
For the distance between two messages m and m (cid:48)
we use the edit-distance (Levenshtein et al., 1966) and note it D mes ( m, m (cid:48) ) .
Speakers synchronization: Within populations, we measure how close speakers’ languages are by introducing a distance between the two languages.
Given two set of speaker weights θ 1 and θ 2 and their respective language L θ 1 and L θ 2 ,
[Equation]
It computes the average distance between two speakers’ messages over all the dataset.
When considering an entire population of N speakers, we can then compute the synchronization score r sync by averaging the distance between all pairs of speakers: r sync
[Equation]
Entropy: To measure language coherence, we study the entropy of the speaker language H θ :
[Equation]
where h : w → − w log( w ) .
In our setting, we seek to minimize entropy as it reduces language complexity and denotes language stability (Kharitonov et al., 2020).
For instance, when H θ is minimal, the speaker uses a unique message to refer to an object.
When H θ is high, the speaker generates synonyms to refer to the same object.
While some entropy is beneﬁcial with noisy communication channels, they do not improve robustness in our case; it only makes the listener’s task harder.
Finally, to ease reading, we display the negative entropy (Neg-Entropy), so all metrics increases correspond to language improvement.
Topographic Similarity: We use topographic similarity as a quantitative measure of compositionality (Brighton & Kirby, 2006; Lazaridou et al., 2018).
Topographic similarity captures how well similarities between objects are transcribed by similarities in the message space by computing the
Spearman correlation (Kokoska & Zwillinger, 2000; Virtanen et al., 2020) between pairwise object distance ( D obj ( v 1 , v 2 ) ) and the corresponding message distances ( D mes ( m v 1 , m v 2 ) ).
Generalization: Test accuracy quantiﬁes how well agents generalize to unseen objects.
It measures the average reconstruction success on the test set where we deﬁne a success when the greedy prediction perfectly matches the input object.
The greedy prediction ˆ v is the one which maximizes the likelihood computed by the listener given the greedy message ˆ m : ˆ v = arg max v (cid:81) k
π kφ ( v k | ˆ m ) with ˆ m t = arg max ¯ m π θ ( ¯ m | ˆ m <t , v ) the greedy message.
3. A LTERING P OPULATION D YNAMICS
We here list potential diversity factors and how they can be distributed when asymmetrizing speakerlistener in minimal size population ( N = 2 ), or introducing heterogeneity within larger populations.
Diversity factors : We aim to vary speciﬁc agent features toward creating diversity in the population and target two core components: the training speed and the network capacity.
First, we control agents’ training speed by changing the number of gradient updates.
Formally, we introduce the probability p of an agent to be optimized after each iteration of the game.
Therefore, the lower p , the slower agent training.
This approach has two positive features: (i) it neither alters the communication graph nor the agent sampling, preserving other homogeneity hypotheses in the population, (ii) it is more stable than modifying the learning rate as it avoids large destructive updates.
Second, we control the network capacity by varying the agent hidden size h in the recurrent core.
Note that we only control training speed or network capacity, never both at once.
Control parameters of local asymmetry: We provide control parameters to characterize diversity at the scale of a minimal size population ( N = 2 ).
Noticeably, N = 2 is the closer setting to a singlepair of agents where we can compute speakers synchronization.
To analyze the effect of speakerlistener asymmetries, we introduce the relative training speed ρ speed := p S /p
L and the relative capacity ρ capacity := h S /h
L where p S (resp.
p L ) is speaker’s learning speed (resp.
listener’s learning speed) and h S (resp.
h L ) is speaker’s hidden size (resp.
listener’s hidden size).
Distributing Population Heterogeneity: To create heterogeneous populations, we characterize every single speaker and listener with individual properties.
We then sampled them once at the beginning of the training.
As explained in section 5.2, we only perform this experiment with the update probability factor. Formally, when altering training speed, we sample p i ∼ Log- N ( η p , σ p ) for every i agent in the population, where Log- N is a log normal distribution.
E XPERIMENTAL SETTING
Game Properties:
In the main paper, we use | K | = 4 attributes with |V| = 4 values.
We also report scores with | K | = 2 attributes and | V | = 10 values in the Appendix G.1 to illustrate that our observation still holds in a different setting.
Finally, we use a vocabulary size of |W| = 20 and a message length T = 10 .
For each run, we generate objects in V K uniformly at random, and we split them into a train and a test sets, which respectively contains 80% and 20% of the objects.
[Figure] Neural architectures:
The speaker ﬁrst encodes the input object v in a one hot encoding and then processes it with a linear layer.
It uses it to initialize a single layer LSTM (Hochreiter & Schmidhuber, 1997) with layernorm (Kiros et al., 2015) and a (default) hidden size of 128 .
Finally, the
LSTM output is fed to a linear layer of dimension |W| followed by a softmax activation.
The listener is composed of a look-up table of dimension 128 followed by a LSTM with layernorm and a (default) hidden size of 128 .
Then, for each attribute k ∈ { 1 , ..., K } , we apply a linear projection of dimension |V| followed by a softmax activation to the last LSTM output.
Optimization: For both agents, we use a Adam optimizer with a learning rate of 5 e 10 − 3 , β 1 = 0 .and
β 2 = 0 .
999 and a training batch size of 1024 when optimizing for their respective loss.
For the speaker, we set the entropy coefﬁcient of 0 .
Finally, we re-normalize the reward by using ¯ r ( x, w <t ) =
r ( x,w <t )
− η rt σ rt to reduce the gradient variance (Sutton et al., 2000), where η rt and σ rt are respectively the average and the standard deviation of the reward within the batch at each time step.
In practice, they are respectively equal to zero and one for all the timesteps but the last.
We use the EGG toolkit
R ESULTS AND DISCUSSION
This section ﬁrst reproduces the impact of the population size on language properties in the homogeneous Lewis setting.
We then study speaker-listener asymmetry by altering networks training speed and capacity, and highlight the importance of relative training speed in shaping language.
We ﬁnally use relative training speed as a non-confounding heterogeneity factor when varying the community-size, and thus tackle the initial contradiction.
5. C OMMUNITY SIZE IS NOT ALONE A LANGUAGE STRUCTURING FACTOR
Language properties are not enhanced by community size.
In Figure 1, we observe that increasing population size does not improve language properties: speaker synchronization and compositionality remain almost constant, and entropy and generalization are even deteriorating. As shown in Appendix E, Spearman correlations for those metrics are respectively -0.44, -0.52, -0.15, -0.14 (with p-values < 0 .
meaning that there is no positive correlation between those metrics and population size.
In addition, we do not observe any gain of language stability by increasing population size .
By stable languages, we mean languages whose properties have low variation across seeds for a given population size.
Here, increasing community size does not reduce the standard deviation of the language metrics across seeds.
Finally, we note that the speaker synchronization is high despite using large communication channel, as also observed by (Graesser et al., 2019).
Overall, we conﬁrm that community size does not improve the language properties of neural agents.
The potential pitfall of homogeneity.
We hypothesize that the absence of positive correlation between population size and language properties may be explained by the simplicity of population modeling.
The homogeneity assumption seems restrictive as it implies two main limitations: (i) although agents are not identically initialized, they have the same training dynamics in expectation, (ii) all agents have the same optimization process, whereas real communities are inherently diverse.
Consequently, we formulate the subsequent hypothesis : Community size may be a structuring factor as soon as heterogeneity is introduced in populations design. [Figure]
5. L ANGUAGE PROPERTIES ARE CONTROLLED BY S PEAKER -L ISTENER A SYMMETRIES
Before generating heterogeneous populations, we explore potential diversity factors at the scale of a minimal size population ( N = 2 ).
The aim is to characterize those factors at a small scale in order not to add confounding factors and better intuit population dynamics.
As described in
Section 3.4, we here modify the learning speed and network capacity for a minimal size population.
Note that we compute this ratio through multiple values of p S /p
L and h S /h L , where p ∈ { 0 . 01 , 0 . 02 , 0 . 04 , 0 . 1 , 0 . 2 , 0 . 5 , 1 . } and h ∈ { 32 , 64 , 128 , 256 , 512 , 1024 } .
Relative differences between speaker and listener signiﬁcantly affect language properties.
In
Figure 2, we display the evolution of language properties with ρ speed and ρ capacity .
We observe that asymmetrizing speaker-listener learning speed and capacity signiﬁcantly inﬂuences language properties.
When the speakers have a large update probability p s or are over-parametrized compared to the listeners, we note a global improvement of language properties: (i) speakers synchronization rate is very high, (ii) language neg-entropy is reaching almost 0, (iii) compositionality is signiﬁcantly improved and (iv) test accuracy is close its optimal value.
On the contrary, when listeners have a large update probability or are over-parametrized compared to the speakers, we note the opposite trend. In Appendix C, we show that the trends are translated along the x-axis when varying setup parameters.
Therefore, the ratios at which metrics start becoming optimal are dependent of the setup. As a sanity check, we then verify that altering p and h does not fundamentally change the language properties per itself.
We thus sweep over the network capacity h and the gradient update probability p while setting a minimal population size of N = 2 , and compute the Spearman correlation in
Table 1.
We observe that both hyperparameter p and h have no statistically signiﬁcant correlations with most of the language scores, i.e. their Spearman coefﬁcients are below 0 .
4 with p-values below 0 .
The only noteworthy correlation is between the network capacity h and the Neg-entropy with a
Spearman correlation of 0 . 89 and a p-value < 0 .
However, as shown in Appendix B the variations are two orders of magnitude lower than neg-entropy variations in Figure 2.
Therefore, the impact of this correlation is small enough in our experiments to be neglected.
In sum, it is not the magnitude of the parameters that is critical to shape language, but the relative ratio between them.
Understanding learning speed asymmetries.
We here try to provide some pieces of intuitions to understand ρ speed by looking at the extreme cases.
In the limit ρ speed (cid:28) 1 , i.e. p s (cid:28) p l , we can consider that the listener is almost optimal wrt.
the speaker at each speaker update.
It implies that listener’s cross-entropy loss is optimally minimized for each message sampled by the speaker that refers to a single input.
Then, as soon as speakers develop a language where each message refers to a single input, the game is over as speaker’s rewards are immediately optimal.
It means that many languages can emerge from this limit case, including languages with high entropy, small synchronization rates, poor compositionality, low generalization.
In the limit ρ speed (cid:29) 1 , i.e. p s (cid:29) p l , we can consider that the speaker is almost optimal wrt.
the listener at each listener update.
The speaker therefore targets the messages providing the highest [Figure] rewards as in a stationary RL task.
We can safely assume that the set of messages providing the highest rewards for each input is small.
It then explains why the speaker converges to a low entropy language.
In addition, when N = 2 , both speakers solve the same communication task and obtain similar rewards.
As a result, they are more likely to align on a similar language as they observe the same quasi-stationary environment.
This use-case also suggests that when speakers are “fast enough”, the only requirement to the emergence of a common language is a common interlocutor.
Finally, we push further the assumption by interpreting the gain of compostionality through “easeof-teaching” (Li & Bowling, 2019).
The authors show that compositional languages are easier to teach for listeners.
We then hypothesize here that ’faster’ speakers develop languages easier to learn for ’slow’ listeners and thus higher quality languages.
[Table]
Network capacity may be confounded with training speed.
While the effect of training speed may be partially intuited, the impact of capacity is more tedious to analyze.
As we have already observed that training speed is a crucial parameter, we thus test whether a correlation between network capacity and learning speed can be established.
In Figure 3 and Appendix F, we pair a ﬁxed pretrained listener (resp. speaker) with newly initialized speakers (resp.
listeners) of increasing capacities and see how fast those new speakers (resp.
listeners) reach convergence.
We observe that all trained agents converge to the same accuracy.
However, we also note that they have different convergence speeds.
Speakers (resp.
listeners) of capacity 512, 128, 32 respectively have to see 100, 250, and 900 batches (resp.
50, 100 and 250 batches) to reach train accuracy of 95%.
In short: the larger the network, the faster the training speed. In short: the larger the network, the faster the training speed.
We conclude that network capacity may be a confounding factor of training speed in our setting and that the high degree of similarity in Figure 2 may be partially explained by the indirect impact of capacity on learning speed.
Therefore, we only explore the training speed ratio in our next experiments.
More generally, we suspect that many other parameters may be latently related to speed of training, e.g. community-graph (Graesser et al., 2019), newborn agents (Ren et al., 2020; Cogswell et al., 2019; Li & Bowling, 2019), or broadcasting by using different speakerlistener ratios as illustrated in Appendix D, but we leave it for future analysis.
Besides, the Lewis game remains synthetic and capacity cannot be reduced to training speed in other settings (Kaplan et al., 2020; Resnick et al., 2020).
5. C OMMUNITY SIZE STRUCTURES LANGUAGES OF HETEROGENEOUS POPULATIONS
Previous sections have identiﬁed that learning speed is a powerful controlling factor and could alter language structure when asymmetrizing speaker-listener pairs.
However, these experiments remain limited for (i) we have not yet observed larger population-size tend to generate more stable and structured language, (ii) this asymmetry remains artiﬁcial and speciﬁc to our Lewis game setting.
Therefore, we here leverage this training speed factor within a population, characterizing individual agents with different update probabilities.
We hence sample agent update probability with p ∼
Log- N ( η p , σ p ) with η p = − 1 and σ p = 1 if not speciﬁed otherwise. 20 N 0.
0.8 Speakers synchronization    20 N Neg-Entropy    20 N 0. Compositionality    20 N 0.
Generalization Larger heterogeneous population leads to higher quality language.
In Figure 4, we observe that emergent language metrics are now positively correlated with population size.
When increasing the population size from 2 to 20 the speaker synchronization and neg-entropy go from low to almost optimal values, compositionality increases by 22% in average and generalization remains stable.
As detailed in Appendix E, speaker synchronization, neg-entropy and compositionality have a signiﬁcant correlation with Spearman coefﬁcients equal to 0.55, 0.24, and 0.21 (p-values < 0 . 05 ).
Generalization does not improve but no contraction is observed as initially noted with homogeneous population in Figure 1.
Those are the ﬁrst hints toward our initial goal: when individuals are heterogeneous, a large population size correlates with higher quality and stable language properties. In Appendix G.2, we report the results with distribution changes.
When the distribution is more concentrated ( p ∼ Log- N ( − 1 , 0 . 5 )), we notice that speakers synchronization and neg-entropy have slower increase, the evolution of compositionality becomes constant with respect to population size and generalization is almost unchanged.
It suggests that all metrics do not have the same sensitivity to heterogeneities and that a minimal dispersion of the distribution is required to observe signiﬁcant effects .
When p ∼ β (1 , 2) , COMING SOON. The overall gain over language metrics can be interpreted in light of Section 5.2.
Language score evolution depends on the relative speaker-listener learning speeds.
In heterogeneous populations, the probability of sampling extreme learning speed p from the distribution tails and thus enforcing speaker-listener asymmetries increases when N grows.
One explanation would be that extreme agents, especially slow listeners, behave as kinetic bottlenecks, forcing the speakers to structure their languages.
Thus, language emergent properties would be particularly determined by fast speakers and/or slow listeners.
Larger heterogeneous population leads to more stable language.
Similarly, the population size correlates with the variance of the language scores across seeds.
Hence, when increasing the population size from 2 to 20 ; the standard deviations of speakers synchronization, neg-entropy, compositionality are divided by 10 , 15 and 2 ; yet generalization is less affected.
One hypothesis for this variance reduction is that smaller communities are more likely to be disparate because of the sampling procedure.
For instance, if we characterize a population with the empirical average of the training speeds ˆ η p , it has high variance with small N .
On the other side, the bigger the community, the closer is the average training speed to the mean of distribution law, lim N →∞ ˆ η p = η p , and the less variance we have in the language properties.
In other words, larger heterogeneous communities have more stable languages because of Law of large numbers in our models.
Such a phenomenon may not occur in the homogeneous population, hence the initial apparent contradiction.
Increasing diversity enforces population size beneﬁts.
In Figure 5, we challenge further the impact of heterogeneity on population by varying the standard deviation σ p of the learning speed distribution.
For each value of σ p , we run the game with populations of size N = 2 and N =and compare the relative evolution of the scores between the two population sizes.
We note that the higher σ p , the larger the gain of synchronization, neg-entropy, generalization when increasing
N .
The relative gain of neg-entropy is almost equal to 100% when σ p > 1 while speaker synchronization gain is close to 200% for large values of σ p .
For the generalization, the 10% relative improvement is noteworthy as it corresponds to reaching close to 100% test accuracy.
The more discrepancy exists within the population (large σ p ), the more beneﬁcial is to have large populations.
Furthermore, speaker synchronization, neg-entropy and generalization standard deviations reduce up to almost 80% when having σ p > 2 .
In sum, this result corroborates that larger populations lead to more stable language.
[Figure]
C ONCLUSION
One objective of language emergence simulations is to identify simple (non-biological) rules that characterized language evolution (Steels, 1997; Briscoe, 2002; Wagner et al., 2003; Kirby, 2001).
However, those models require a delicate balance between simplicity to ease analysis and complexity to remain realistic.
In this paper, we argue that the current population-based models may actually be too simplistic as they assume homogeneous communities.
We show that this simpliﬁcation is a potential root cause of the experimental difference between neural computational observations and socio-linguistic literature.
Namely, larger populations should lead to more stable and structured languages, which is not observed in recent emergent models.
Yet, as soon as we add diversity within populations, this contradiction partially vanishes.
We advocate for better integrating population diversity and dynamics in emergent literature if we want computational models to be useful.
In this journey, we also observe that the relative training speed of the agents may be an underestimated factor in shaping languages.
Worse, other agent properties may be confounded with it, such as network capacity.
This work also opens many questions.
First, our observations were performed in the Lewis game so far, and it would be interesting to observe the impact of diversity in more complex settings.
Second, while heterogeneity is a structuring factor in population, the core factors are yet to be investigated, e.g., how to correctly model diversity, are there some causal components?
Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial experimental difference such as varying the communication network topology (Wagner, 2009), the proportion of contact agents (Wray & Grace, 2007; Clyne, 1992; Graesser et al., 2019) or the proportion of second learners (Ellis, 2008).
Finally, training speed is a natural controlling parameter for computational models, but it is unclear how it may relate to human behavior.
Overall, we hope that this paper provides new insights toward modeling populations in language emergence, and it is part of this constant back and forth between cognitive sciences, linguistics, and artiﬁcial intelligence. A R ANDOM BASELINES We report the values of Speakers synchronization, neg-entropy, topographic similarity and test accuracy for randomly-initialized and untrainded agents: [Table]
B N ON - CONFOUDING PARAMETERS
[Figure] [Figure]
In Figure 6 &
[Figure]
we show the variations of language scores with increasing values of p and h , while being identical for speaker and listener.
In Table 1, we shows the Spearman correlation between the metrics and the control parameters.
We do not observe signiﬁcant correlation when working with p .
In addition, both the relative and absolute variations of the parameters are negligible compared to experiments displayed in Figure 2.
C V ARYING SETUP PARAMETERS WHEN STUDYING SPEAKER - LISTENER ASYMMETRIES
In this Section, we show how trends displayed in Figure evolve when performing some change of parameters in the setup.
As shown in Figure 8, when modifying agents’ hidden sizes (speaker’s hidden size is equal to 64 and listener’s hidden size is equal to 512 while they are both equal toin Figure 2), the trends are the same.
However, we notice a translation of the curves.
It means that the ratio of learning speed at which metrics start becoming optimal is controlled by the parameters of the setup.
Figure 8 highlights that the trends are sensitive to agents’ capacity.
[Figure] [Figure]
We extends results of Section 5.2 to another population factor and intuit how it relates to learning speed.
We introduce the ratio between the number of speakers and the number of listeners within a population ρ agents := N listener N speaker .
Note that this ratio cannot be applied to a single speaker-listener pair.
Yet, it introduces an asymmetry between the community of speakers and the community of listeners.
We show the evolution of language metrics in Figure 9.
There, we observe than unbalancing
N s and N l has a strong impact on language scores.
Speaker synchronization reach upper and lower bound depending on the ratio, Neg-entropy either is very low or close to 0, and compositionality and generalization increase.
ρ agents is thus an additional control factor of language properties.
Extreme cases coincide with the training speed interpretation of Section 5.2.
Indeed, at each iteration of the game, each speaker (resp. listener) has a probability 1 /N speakers (resp.
1 /N listeners ) to be sampled.
Thus, speakers have N listeners /N
speakers more of less learning steps than listeners.
Consequently, when ρ agents (cid:29) 1 , individual speakers (resp.
listeners) have way more optimization steps than individual listeners (resp. speakers) and we fairly hypothesize that speakers (resp.
listeners) train faster than listeners (resp. speakers).
This ratio may therefore be a confounding factor of a training speed mechanism.
Note that, interestingly, ρ agents can also be related to different sampling procedures, or broadcasting phenomenon.
E A DDITIONAL C ORRELATIONS
[Table]
[Table]
F I NFLUENCE OF CAPACITY ON AGENTS LEARNING SPEED
[Figure] [Figure]
r a i n i n g a cc u r a c y
[Figure]
In Section 5.
[Figure]
we shed light on the link between the network capacity and the training speed.
We here complete this observation.
In Figure 10 (resp. Figure 11)
[Figure] we pair a ﬁxed pretrained listener (resp.
speaker) with newly initialized speakers (resp.
listeners) of increasing capacity
and then display the evolution of their training accuracies, and conﬁrmed further our observation.
Please note in Figure 10, some limit cases appear with the pure RL task, i [Figure] e. ﬁxed listener.
Speakers’ convergence become instable across seeds for small networks ( h S = 32 ).
Yet, the training is still stable in supervised task, i.e. ﬁxed speaker, in Figure 11. [Figure] [Figure] [Figure]
[Figure] [Figure] G C OMPLEMENTARY SETTING G. D IFFERENT INPUT SPACE
In this section
[Figure]
we reproduce our core experiments in a different Lewis game setting, with | K | =attributes containing each | V | = 10 values.
[Figure]
[Figure] [Figure] HETEROGENEITY DISTRIBUTION In this section, we vary the distribution of agent update probability p to evaluate how the observed trends are sensitive to the heterogeneity distribution.
Log-normal distribution with lower standard deviation In Figure 14 agent update probability p are sampled according to a log-normal distribution Log- N ( η p , σ p ) with η p = − 1 and σ p = 0 . Compared to Figure 4, standard deviation has been divided by 2 meaning a more concentrated heterogeneity distribution. We see that speakers synchronization and neg-entropy increases are slower and that compositionality increase is now very low. Generalization remains almost unchanged.
This plot suggests that the distribution of heterogeneities must be minimally wide to observe a positive correlation with population size and that all metrics do not have the same sensitivity to heterogeneities.
[Figure]
change Coming
[Table]
H C HANGES
Here is the list of changes made after reviewers’ comments.
When a change has been made, we show how the paper was before/after the change.
[Table]
Before:
Populations in simulated emergent communication.
Recent works in emergent communication have been debating the prerequisites to the emergence of language universals such as compositionality (Li & Bowling, 2019;
Ren et al., 2020; Lazaridou et al., 2018; Mordatch & Abbeel, 2018;
Resnick et al., 2020; Kottur et al., 2017; Choi et al., 2018; Łukasz Kuci ´ nski et al., 2020), generalisation (Baroni, 2020; Chaabouni et al., 2020; Hupkes et al., 2020; Denamgana ¨ ı & Walker, 2020), efﬁciency (Chaabouni et al., 2019; Rita et al., 2020) or stability (Kharitonov et al., 2020; Lu et al., 2020).
Among them, a few works explored how different population structures may impact properties of emergent languages.
Inspired by iterated learning methods (Kirby et al., 2008), Ren et al.
(2020); Li & Bowling (2019);
Lu et al.
(2020) look at language evolution across multiple generations of agent pairs, i.e. population is spread over time.
However, we consider here a population where multiple agent-pairs coexist simultaneously within a single generation.
There, Graesser et al.
(2019) show that community of agents start coordinating their language when at least three agents are present in the community.
They later assume that increasing the community size may impact the emergent shared language, but did not observe it in their initial experiments.
A similar hypothesis was also made by Bouchacourt & Baroni (2019) while analyzing the inﬂuence of symmetric agents in the emergence of a common language.
With different research objectives, Cogswell et al.
(2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission.
Analogously, Tieleman et al.
(2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers’ latent representations by increasing population size.
Eventually, Fitzgerald (2019) suggest that populations improve regularization and generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties.
In this paper, we aim at analyzing how population size affects those discussed properties.
We align with emergent communication literature and show that naively increasing community size does not consistently improve language properties.
We then challenge the homogeneity assumption made in all population designs.
After:
Populations in experimental emergent communication.
Experimental language emergence has mainly been studied with two methods: behavioral studies (Kegl, 1994; Sandler et al., 2005) and simulations (neural and non neural)
(Wagner et al., 2003; Lazaridou & Baroni, 2020).
From behavioral studies and non neural simulations, two main approaches have emerged in the pastyears: experimental semiotics (Galantucci & Garrod, 2011;
Garrod et al., 2007) and iterated learning (Kirby & Hurford, 2002; Kirby et al., 2008; Beckner et al., 2017).
According to experimental semiotics studies, languages are mainly subject to an expressivity pressure; they argue that messages should be highly informative to allow communication within a group (Fay & Ellison, 2013).
According to Iterated Learning paradigm, structures emerge from a compressibility pressure; they argue that memory limitations compel messages to become simpler to be easily learned (Tamariz &
Kirby, 2015), which is also referred to as transmission bottleneck (Smith et al., 2003).
Kirby et al.
(2015) then combined those two approaches and show that languages emerge as a trade-off between expressivity and compressivity during cultural evolution.
Recently, similar ideas have been modeled in emergent communication frameworks involving neural agents (Ren et al., 2020; Lu et al., 2020).
Because seminal non-neural simulations used speciﬁc optimization methods and models across studies (Wagner et al., 2003), it is hard to generalize a global picture.
Indeed, Wagner et al.
(2003) keep listing standalone experiments.
Modern neural agents manage to simplify and standardize agents’ modelling.
It thus enables explore the determinants of different structural properties of human languages within a unique framework, paving the way for holistic models of emergence communication (Lazaridou & Baroni, 2020).
Our paper is related to this last set of neural works.
Recent works in emergent communication have been debating the prerequisites to the emergence of language universals such as compositionality (Li & Bowling, 2019;
Ren et al., 2020;
Lazaridou et al., 2018; Mordatch & Abbeel, 2018; Resnick et al., 2020; Kottur et al., 2017; Choi et al., 2018; Łukasz Kuci´nski et al., 2020), generalisation (Baroni, 2020; Chaabouni et al., 2020; Hupkes et al., 2020; Denamgana¨ı & Walker, 2020), efﬁciency (Chaabouni et al., 2019; Rita et al., 2020) or stability (Kharitonov et al., 2020; Lu et al., 2020).
Among them, a few works explored how different population structures may impact properties of emergent languages.
Inspired by iterated learning methods (Kirby et al., 2008), Ren et al.
(2020); Li & Bowling (2019)
; Lu et al.
(2020) look at language evolution across multiple generations of agent pairs, i.e. population is spread over time.
However, we consider here a population where multiple agent-pairs coexist simultaneously within a single generation.
There, Graesser et al.
(2019) show that community of agents start coordinating their language when at least three agents are present in the community.
They later assume that increasing the community size may impact the emergent shared language, but did not observe it in their initial experiments.
A similar hypothesis was also made by Bouchacourt & Baroni (2019) while analyzing the inﬂuence of symmetric agents in the emergence of a common language.
With different research objectives, Cogswell et al.
(2019) exhibit a slight compositionality enhancement when increasing population size without the need of language transmission.
Analogously, Tieleman et al.
(2019) explicitly study community size and display a small but consistent gain of abstraction and structure within speakers’ latent representations by increasing population size.
Eventually,
Fitzgerald (2019) suggest that populations improve regularization and generalization compared to single speaker-listener pairs but underlines that there is not a clear correlation between community size and learning properties.
In this paper, we aim at analyzing how population size affects those discussed properties.
We align with emergent communication literature and show that naively increasing community size does not consistently improve language properties.
We then challenge the homogeneity assumption made in all population designs.
H.1. C HANGE 1.2 ( REFERRING TO S ECTION 2)
Before:
Raviv et al.
(2019a) went one step further by arguing that the community size is predictive, i.e. a causal factor, of language structure and diversity.
After:
As part of their research on the inﬂuence of network structures on language emergence (Raviv et al., 2019a;b; 2020), Raviv et al. (2019a) went one step further by arguing that the community size is predictive of language structure and diversity.
H.1.
C HANGE 1.3 ( REFERRING TO S ECTION 6)
Before:
Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial contradiction.
After:
Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial experimental difference such as varying the communication network topology (Wagner, 2009), the proportion of contact agents (Wray & Grace, 2007; Clyne, 1992; Graesser et al., 2019) or the proportion of second learners (Ellis, 2008).
H.1.
C HANGE 1.4 ( REFERRING TO S ECTION 5.3)
Before:
[Figure]
Previous sections have identiﬁed that learning speed is a powerful controlling factor and could alter language structure when asymmetrizing speaker-listener pairs.
However, these experiments remain limited for (i) we have not yet observed larger population-size tend to generate more systematic and structured language, (ii) this asymmetry remains artiﬁcial and speciﬁc to our Lewis game setting.
Therefore, we here leverage this training speed factor within a population, characterizing individual agents with different update probabilities.
We hence sample agent update probability with p ∼
Log- N ( η p , σ p ) with η p = − 1 and σ p = 0 .
5 if not speciﬁed otherwise.
Larger heterogeneous population leads to higher quality language.
In Figure 4, we observe that emergent language metrics are now positively correlated with population size.
When increasing the population size from 2 to 20 ; the speaker synchronization, neg-entropy, compositionality, and generalization increase by 41% , 64% , 8% , and 2% on average.
As detailed in Appendix E, compositionality and generalization have a small positive but statistically signiﬁcant correlation with
Spearman coefﬁcients above 0 .
3 (p-values < 0 . 05 ), and we do not observe a language score contraction as initially noted with homogeneous population in Figure 1.
Those are the ﬁrst hints toward our initial goal: when individuals are heterogeneous, a large population size correlates with higher quality and stable language properties.
After: [Figure]
Previous sections have identiﬁed that learning speed is a powerful controlling factor and could alter language structure when asymmetrizing speaker-listener pairs.
However, these experiments remain limited for (i) we have not yet observed larger population-size tend to generate more systematic and structured language, (ii) this asymmetry remains artiﬁcial and speciﬁc to our Lewis game setting.
Therefore, we here leverage this training speed factor within a population, characterizing individual agents with different update probabilities.
We hence sample agent update probability with p ∼
Log- N ( η p , σ p ) with η p = − 1 and σ p = 1 if not speciﬁed otherwise.
Larger heterogeneous population leads to higher quality language.
In Figure 4, we observe that emergent language metrics are now positively correlated with population size.
When increasing the population size from 2 to 20 the speaker synchronization and neg-entropy go from low to almost optimal values, compositionality increases by 22% in average and generalization remains stable.
As detailed in Appendix E, speaker synchronization, neg-entropy and compositionality have a signiﬁcant correlation with Spearman coefﬁcients equal to 0.55, 0.24, and 0.21 (p-values < 0 . 05 ).
Generalization does not improve but no contraction is observed as initially noted with homogeneous population in Figure 1.
Those are the ﬁrst hints toward our initial goal: when individuals are heterogeneous, a large population size correlates with higher quality and stable language properties.
In Appendix G.2, we report the results with distribution changes.
When the distribution is more concentrated ( p ∼ Log- N ( − 1 , 0 . 5 )), we notice that speakers synchronization and neg-entropy have slower increase, the evolution of compositionality becomes constant with respect to population size and generalization is almost unchanged.
It suggests that all metrics do not have the same sensitivity to heterogeneity and that a minimal dispersion is required to observe signiﬁcant effects .
When p ∼ β (1 , 2) , COMING SOON.
H.1.
C HANGE 1.5 ( REFERRING TO S ECTION 6)
Change
Before:
Yet, as soon as we add diversity within populations, this contradiction vanishes.
After:
Yet, as soon as we add diversity within populations, this contradiction partially vanishes.
Change
Before:
Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial contradiction.
After:
Third, we break the homogeneous assumptions in our work, but other procedures may exist that solve the initial contradiction such as varying the communication network topology (Wagner, 2009), the proportion of contact agents (Wray & Grace, 2007; Clyne, 1992; Graesser et al., 2019) or the proportion of second learners (Ellis, 2008).
H.
C ONCERNING R EVIEWER ZXE P COMMENTS
H.2.
C HANGE 2.1 ( REFERRING TO S ECTION 5.2)
Before
We observe that asymmetrizing speaker-listener learning speed and capacity signiﬁcantly inﬂuences language properties.
When the speakers have a large update probability p s or are over-parametrized compared to the listeners, we note a global improvement of language properties: (i) speakers synchronization rate is very high, (ii) language neg-entropy is reaching almost 0, (iii) compositionality is signiﬁcantly improved and (iv) test accuracy is close its optimal value.
On the contrary, when listeners have a large update probability or are over-parametrized compared to the speakers, we note the opposite trend.
After
We observe that asymmetrizing speaker-listener learning speed and capacity signiﬁcantly inﬂuences language properties.
When the speakers have a large update probability p s or are over-parametrized compared to the listeners, we note a global improvement of language properties: (i) speakers synchronization rate is very high, (ii) language neg-entropy is reaching almost 0, (iii) compositionality is signiﬁcantly improved and (iv) test accuracy is close its optimal value.
On the contrary, when listeners have a large update probability or are over-parametrized compared to the speakers, we note the opposite trend.
In Appendix C, we show that the trends are translated along the x-axis when varying setup parameters.
Therefore, the ratios at which metrics start becoming optimal are dependent of the setup.
[Equation]
Before
In short: the larger the network, the faster the training speed.
We conclude that network capacity is a confounding factor of training speed in our setting.
After
In short: the larger the network, the faster the training speed.
We conclude that network capacity may be a confounding factor of training speed in our setting and that the high degree of similarity in
Figure 2 may be partially explained by the indirect impact of capacity on learning speed.
H.
C
ONCERNING R EVIEWER UK K L COMMENTS
H.3.
C HANGE 3.1 ( REFERRING TO S ECTION 1)
Before
In Figure 1, we observe that increasing population size does not improve language properties: speaker synchronization and compositionality remain almost constant, and entropy and generalization are even deteriorating.
After
In Figure 1, we observe that increasing population size does not improve language properties: speaker synchronization and compositionality remain almost constant, and entropy and generalization are even deteriorating.
As shown in Appendix E, Spearman correlations for those metrics are respectively -0.44, -0.52, -0.15, -0.14 (with p-values < 0 .
meaning that there is no positive correlation between those metrics and population size.”
H
C ONCERNING R EVIEWER 5 N 9H COMMENTS
Changes following Reviewer 5n9H comments are either discussed in Appendix or directly changed in the main text (typography).
H.
C
ONCERNING R EVIEWER GR Q9 COMMENTS
H.5.
C HANGE
REFERRING TO
S ECTION
Before
Raviv et al.
(2019a) went one step further by arguing that the community size is predictive, i.e. a causal factor, of language structure and diversity.
After
Raviv et al.
(2019a) went one step further by arguing that the community size is predictive, of language structure and diversity.
H.5.
C HANGE 5.2 ( REFERRING TO S ECTION 5.3)
Before
Trends are particularly signiﬁcant for neg-entropy and speaker synchronization.
After
The relative gain of neg-entropy is almost equal to 100% when σ p > 1 while speaker synchronization gain is close to 200% for large values of σ p .