Emergent Communication: Generalization and Overﬁtting in Lewis Games
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Lewis signaling games are a class of simple communication games for simulatingthe emergence of language.
In these games, two agents must agree on a communication protocol in order to solve a cooperative task.
Previous work has shownthat agents trained to play this game with reinforcement learning tend to developlanguages that display undesirable properties from a linguistic point of view (lackof generalization, lack of compositionality, etc).
In this paper, we aim to providebetter understanding of this phenomenon by analytically studying the learningproblem in Lewis games.
As a core contribution, we demonstrate that the standardobjective in Lewis games can be decomposed in two components: a co-adaptationloss and an information loss.
This decomposition enables us to surface two potential sources of overﬁtting, which we show may undermine the emergence of astructured communication protocol.
In particular, when we control for overﬁttingon the co-adaptation loss, we recover desired properties in the emergent languages:they are more compositional and generalize better.
Introduction
Understanding the dynamics of language evolution has been a challenging if not controversial researchtopic in the language sciences [28, 12].
Given that the very ﬁrst human language cannot be unearthedfrom fossils [5], computational models have been designed to simulate the emergence of a structuredlanguage within a controlled environment.
In this line of work, Lewis signaling games [50] are amongthe most widespread playground environments to model language emergence: they are inherentlysimple, yet they exhibit a rich set of communication behaviors [16, 61].
Therefore, understanding
Lewis games dynamics may shed light on the prerequisites of language emergence.
In their original form, Lewis signaling games involve two agents: a speaker and a listener.
The speakerobserves a random state from its environment, e.g. an image, and sends a signal to the listener.
Thelistener then undertakes an action based on this signal.
Finally, both agents are equally rewarded basedon the outcome of the listener’s action.
The resolution of this cooperative two-player game requiresthe emergence of a shared protocol between the agents [50, 16].
One way to model the emergence ofsuch protocol is to give the agents the capacity to learn.
The agents, and therefore, the communicationprotocol, are shaped by a sequence of trials and errors over multiple games [72, 40, 66, 61].
Thislearning-centric approach allows for a ﬁne analysis of the language emergence dynamics [61, 32].
Italso raises challenging learning-speciﬁc questions: What are the inductive biases present in the agentarchitecture and loss function that shape the emergent language [39]?
How do agents generalize fromtheir training set?
Is the resulting language compositional [8]?
What is the impact of overﬁtting [48]?
Recently, there has been a resurgence of interest for such learning-based approaches followingadvances in machine learning [47].
In these approaches, the speakers and listeners are modeled as deepreinforcement learning agents optimized to solve instances of the Lewis games [48, 29, 56, 51, 25]
The vast majority of these works explore Lewis games from an empirical perspective.
However,some of the recent experimental results are at odds with experimental ﬁndings from the linguisticsliterature.
For instance, the emergent protocols lack interpretability [44], generalization does notalways correlate with language compositionality [9], successful strategies are not naturally adopted inpopulations [59, 11], and anti-efﬁcient communication may even emerge [10].
It is unclear whetherthose empirical observations result from a learning failure, e.g. optimization problems, overﬁtting,or whether they are symptomatic of more fundamental limitations of Lewis games for modelinglanguage emergence, e.g. lack of embodiment [27, 4, 54, 33].
Overall, it is crucial to establish newanalytical insight to analyze Lewis games in the learning setting.
In this paper, we introduce such an analytical framework to diagnose the learning dynamics of deepreinforcement learning agents in Lewis signaling games.
As a core contribution, we demonstrate undermild assumptions that the loss of the speaker and listener can be decomposed into two componentswhen resolving Lewis signaling games: (i) an information loss that maximizes the mutual informationbetween the observed states and speaker messages; (ii) a co-adaptation loss that aligns the speakerand listener’s interpretation of the messages (Section 2).
Based on this decomposition, we empiricallyexamine the evolution of these two losses during the learning process (Section 5).
In particular,we identify an overﬁtting problem in the co-adaptation loss between the agents which underminesthe emergence of structured language.
We then show that the standard setup used in the deeplanguage emergence literature consistently suffers from this overﬁtting issue (Section 5.1).
Thisrealization explains some of the contradictory observations [9] and experimental choice from pastworks [56, 51, 59].
Finally, we explore regularization methods to tackle this co-adaptation overﬁtting.
We observe that reducing the co-adaptation overﬁtting allows for developing a more structuredcommunication protocol (Section 5.2).
All in all, our contributions are three-fold: (i) we provide a formal description of Lewis games from alearning standpoint (Section 2.3); (ii) we apply this framework in experiments to show that degenerateresults are primarily due to overﬁtting in the co-adaptation component of the game (Section 5.1) ;(iii) we propose natural ways of tackling this overﬁtting issue and show that, when we control thereceiver’s level of convergence, we obtain a well-structured emergent protocol (Section 5.2).
Analyzing Lewis Games
We show that Lewis games’ objective decomposes into two terms: (i) an information loss thatmeasures whether each message refers to a unique input; (ii) a co-adaptation loss that quantiﬁes thealignment of the speaker’s and listener’s interpretation of the messages.
For the sake of simplicityand to ease the reader intuition, we focus on the reconstruction variant of Lewis games in the mainpaper, but generalize our analysis to a more general class of Lewis games in Appendix A.
2.1 Background: Lewis Reconstruction Games
Game formalism In reconstruction Lewis games, a speaker observes a random object of its environment.
The speaker then sends a descriptive message, which a second agent, the listener, usesto reconstruct the object.
The success of the game is quantiﬁed by how well the original object isreconstructed [36, 10, 58, 59].
Formally, the observed object denoted by x is selected from a setof objects denoted by X .
We denote by X the random variable characterizing x , sampled fromdistribution p .
The intermediate message sent by the speaker m belongs to the set of all potentialmessages M .
The speaker follows a policy ⇡ ✓ which samples a message m with probability ⇡ ✓ ( m | x )conditioned on object x .
We denote by M ✓ the random variable characterizing the message m ,sampled from ⇡ ✓ ( ·| X ) .
We denote by ⇡ ✓ ( m ) =
P x ⇡ ✓ ( m | x ) p ( x ) the marginal probability of amessage given policy ⇡ ✓ .
Given a message m , the listener outputs a probability distribution overinputs ⇢  ( ·| m ) , and the probability of reconstructing the entire object x given m is thus ⇢  ( x | m ) .
Game objectives In reconstruction games, the speaker and listener minimize the negative loglikelihood of the reconstructed object.
Both agents thus optimize the objective:
[Equation]
where optimizing the speaker is a reinforcement learning problem whose parameters ✓ are optimizedusing policy gradient [67] and optimizing the listener is a supervised learning problem whoseparameters  are optimized with gradient descent.
2.2 Building Intuition on the Lewis Reconstruction Game Learning Dynamics
To get a better intuition of the dynamic of Lewis reconstruction games, we can analyze the formtaken by the optimal listener, given speaker ⇡ ✓ .
As shown in Appendix A.1.1, given a message m ,the optimal listener’s distribution ⇢ ✓ ( ·| m ) can be written in closed-form:
[Equation]
At each update, the listener gets closer to its optimum ⇢ ✓ ( ·| m ) .
If we suppose that the listenerperfectly ﬁts ⇢ ✓ ( ·| m ) at any moment, the loss becomes:
[Equation]
where H ( X | M ✓ ) is the conditional entropy of X conditioned on M ✓ and I ( X ; M ✓ ) is the mutualinformation between X and M ✓ .
Thus, if the listener is optimal at every point in time, the speaker’stask merely becomes the construction of a message protocol that maximizes the mutual informationbetween objects and messages, i.e. the construction of an unambiguous message protocol.
In practice, the listener never perfectly ﬁts the optimum.
In the following, we elucidate the effect ofthis gap between the listener and its optimum on the dynamics of the game.
2.3 Analytical Result: The Lewis Reconstruction Games Loss Decomposition
In Lewis reconstruction games, the agents’ loss can be decomposed into two terms:
[Equation]
• An information term L info quantiﬁes the degree of ambiguity of the language protocol.
It is minimal when each message refers to a unique object; • A co-adaptation term L adapt quantiﬁes the gap between the listener and its optimum: the speaker’s posterior distribution.
This co-adaptive term is optimized both by the speaker and the listener.
When the listener is optimal, this co-adaptation objective is zeroed.
The proof is provided in Appendix A and extends to general cooperative rewards, e.g. covers theaccuracy reward, and more general variants of Lewis signaling games, e.g. discrimination games.
This decomposition gives us insights on the game dynamics and the constraints that shape languagesin the game with neural agents:
The information loss L info captures the speaker’s intrinsic objective: to develop an unambiguousprotocol.
L info is minimal, equals to 0 , when the communication protocol is unambiguous, i.e. everymessage from the speaker’s policy ⇡ ✓ refers to a unique object.
Conversely, L info is maximal, equalto H ( X ) , when the message protocol is fully ambiguous, and X and M ✓ are independent variables.
The co-adaptation loss L adapt is speciﬁc to learning agents.
This loss measures how far the listener⇢  is from its optimum ⇢ ✓ .
If L adapt = 0 , the listener and its optimum coincide.
L adapt has theparticularity to be optimized by the two agents.
From the listener’s side, it merely corresponds to theoptimization of its supervised task.
From the speaker’s side, it brings out that the speaker must adaptits language to the listener in addition to build an unambiguous message protocol.
In other words, theco-adaptation loss pushes the speaker to develop a language that can be easily recognized by listeners.
This pressure diminishes as the listener approaches its optimum.
From a practical perspective, Equation (4) yields the following individual gradients:
[Equation]
where the listener only receives gradients from the co-adaptation term, and the speaker receivesgradients from both terms.
This loss decomposition also ﬁnds echoes in the cognitive science literature in the form of anexpressivity vs. learnability trade-off [63]; see Section 6 for a detailed discussion.
2.4 Generalization Gaps in Lewis Reconstruction Games
We explore another facet of the loss decomposition that arises from learning.
As agents are trainedon partial views of their environment, it opens questions of overﬁtting and generalization to unseenobjects.
As is customary in machine learning, we consider agents trained on a ﬁxed, ﬁnite samplefrom the data distribution: the training set.
Let us denote by p train the empirical object distributionover the training set and X train an object sampled from p train .
Similarly let M train ✓ denote amessage sampled from ⇡ ✓ ( .
| X train ) , ⇡ train ✓ ( m ) =
P x ⇡ ✓ ( m | x ) p train ( x ) the marginal probabilityof a message on the training set, and ⇢ ✓ train ( x | m ) =
[Equation]
| x ) the speaker’s posterior 129 distribution with respect to the prior distribution p train .
The training loss can be written as follow:
[Equation]
Decomposing the gap between L train ✓ ,  and L ✓ ,  uncovers two sources of overﬁtting:
[Equation]
Intuitively, information overﬁtting occurs when the speaker only develops an unambiguous languageon the training set, but ambiguities remain on the total dataset.
Co-adaptation overﬁtting occurs whenthe two agents agree on a common communication protocol on the training data, but not on all data.
Method
This section gathers the methodological tools required to empirically study the loss decomposition.
3.1 Probing the Information and Co-adaptation Losses
The resulting loss of the probe listener is ˆ L info , and the loss of the listener is used to estimate ˆ L adapt .
Computing L info and L adapt directly necessitates estimating the posterior distribution of the speaker,⇢ ✓ ( .
| m ) .
Doing so requires summing over all X which is intractable.
Fortunately, deep model arelarge enough so that they can perfectly solve their task on their train set.
We can leverage this fact tocompute empirical estimates ˆ L info and ˆ L adapt of L info and L adapt respectively by using an auxiliary 141 listener trained to optimality.
We here detail an empirical probing mechanism to obtain estimates ˆ L info and ˆ L adapt given speaker⇡ ✓ and listener ⇢  .
As noted in Equation 2, the posterior ⇢ ✓ also corresponds to the optimal listener.
Therefore, we obtain an estimate of the posterior by training a listener to optimality, and use thisoptimal listener to decompose the loss.
In practice, to obtain this optimal listener, we freeze speaker⇡ ✓ and listener ⇢  and initialize a new, auxiliary listener from scratch, which we refer to as the probelistener.
As illustrated in Figure 1, the probe listener is trained to reconstruct object x from messagem , with x drawn from distribution p or p train and m sampled according to the frozen speaker policy⇡ ✓ ( .
| x ) , until a stopping criterion is met.
We then distinguish between the train and test estimates:
[Equation]
7) and,
[Equation]
8) where ⇢ train !
⇤ and ⇢ !
⇤ are the probe listeners trained over distributions p train and p respectively.
1 Notethat this probing mechanism, while tractable, is computationally costly as it necessitates training anew probe listener to convergence, and so we only use it as a valuable diagnosis tool.
3.2 Decreasing the Importance of the Co-adaptation Loss in the Speaker’s Loss
As explained in Section 2.3, the information loss alone is sufﬁcient for the speaker to develop anunambiguous language.
This begets the question: does the co-adaptation loss have any bearing on theemergent language at all?
We elucidate this question by reducing the weight of the co-adaptation termin the decomposition.
To that end, we use the probing method described above.
With ⇢ train !
⇤ ( x | m ) theprobe listener’s estimate of the speaker’s posterior on the train set, we build the following reward:
[Equation]
where ↵ is a weight in [0 , 1] .
As derived in Appendix B, the speaker’s loss then becomes:
[Equation] [Equation]
[Equation] [Equation]
Hence, ↵ balances the two speaker objectives (up to an approximation error).
When ↵ = 1 , the lossfalls back to the classic setting.
When ↵ = 0 , the co-adaptation term is removed on the speaker side;note that the Lewis game can still be solved since the listener still optimizes the co-adaptation term.
We experimentally analyse the effect of ↵ on resulting languages in Section 5.1.
3.3 Controlling the Listener’s Co-adaptation Loss Level of Convergence
As mentioned in 2.2, the inﬂuence of L adapt on the co-adaptation term in the speaker’s loss ismodulated by the listener’s level of convergence to its optimum.
To understand the effect of thisco-adaptation, we decouple the speaker and listener training and train the listener via three procedures:
Continuous listener
The
listener is continuously trained, jointly with the speaker.
This is the standardsetting in the emergent communication literature, and serves to report the baseline behavior.
Partial listener The listener is re-initialized after each of the speaker’s update and trained on the
training set for N step before updating the speaker again
This baseline
enables
ﬁne grained
analysis
of the inﬂuence of under-training (low N step
and over-training
large N step
) the listener.
Early stopping listener The listener is also re-initialized after each of the speaker’s update but isnow trained until an early stopping criterion is met on the validation set.
This allows us to get thebest estimate of the posterior ⇢ ✓ ( .
| m ) .
This can be seen as a variant of the partial listener with anadaptive number of steps N step .
Experimental settings
4.1 Game description
Unless speciﬁed, all our experiments are run on the reconstruction game deﬁned in Section 2.1.
Experiments are run over 6 seeds and reach > 99% training reconstruction scores unless otherwisestated.
Our implementation is based on the EGG toolkit [35] and the code is available at HIDDEN .
Environment We consider objects x =: ( x 1 , ..., x K ) 2 X
=:
X 1 ⇥ ... ⇥
X K characterized by K 184 attributes where attribute i may take |X
i | different values.
By design, this synthetic environment allows 185 us to test the ability of agents to refer to unseen objects by communicating their attributes [3, 44].
Each object is the concatenation of one-hot representations of the attributes ( x i ) 1  i  K .
Objects have 187 K = 6 attributes, each taking 10 different values, for a total of 1 million objects.
Training, validationand test sets are randomly drawn from this pool of objects (uniformly and without overlap), and arerespectively composed of 4000 , 1000 and 1000 elements.
Thus, the agents only have access to asmall fraction ( < 1% ) of the environment, making the generalization problem challenging.
Communication channel Messages m
=: ( m j ) Tj =1 2 M =: V T are sequences of T tokens where 192 each token is taken from a ﬁnite vocabulary
V , ﬁnishing by a hard-coded end-of-sentence token
EoS .
In our experiments, messages have maximum length T = 10 and symbols are taken from avocabulary of size |V| = 10 to prevent a bottleneck in the communication channel.
Speaker model The speaker follows a recurrent policy: given an input object x , it samples for allt 2 [1 , T ] a token m t with probability ⇡ ✓ ( m t | m <t , x ) .
The speaker takes in the object x as a vectorof size K ⇥ |X .
| and passes it through a linear layer of size 128 to obtain an object embedding, used 198 to initialize a LSTM [31] of size 128 with layer normalization [2].
At each time step, the LSTM’soutput is fed into a linear layer of size |V| , followed by a softmax, to produce ⇡ ✓ ( m t | m <t , x )
Listener model Given a message m = ( m 1 , ..., m T ) , the listener outputs for each attribute k aprobability distribution over the |X k
| values: ⇢ k  ( ·| m ) .
The probability of reconstructing the entireobject x given m is then ⇢  ( x | m ) :=
Q k ⇢ k  ( x k | m ) .
The listener passes each message m t throughan embedding layer of dimension 128 followed by a LSTM with layernorm of size 128 .
The ﬁnalrecurrent state h l T is passed through K linear projections of size |X .
| , each followed by a softmax,providing K independent probability distributions of sizes |X .
| to predict each attribute of x .
Optimization The agents are optimized using Adam [38] with a learning rate of 5 · 10  4 ,  1 = 0 .
9and  2 = 0 .
999 and a batch size of 1024 .
For the speaker we use policy gradient [67], with a baselinecomputed as the average reward within the minibatch, and an entropy regularization of 0 .
01 to thespeaker’s loss [73].
4.2 Evaluating emergent languages properties
Generalization We measure generalization by computing the average test reconstruction score overall the attributes of a probe listener trained on the training set using an early stopping criterion onthe validation set.
Indeed, the trained listener ⇢  may overﬁt to the training set, and so using it mayunder-estimate.
Using a separate listener removes this bias.
Compositionality Compositionality is a fundamental feature of natural language often seen as aprecondition to generalize [6, 68, 70].
We assess the compositionality by computing the topographicsimilarity [8, 48].
It is deﬁned as the Spearman correlation [43, 71] between the distance in inputspace, i.e. the average number of common attributes, and the distance in message space, i.e. theedit-distance between the corresponding messages [49].
As we here deal with large object space andstochastic policies, we use a bootstrapped estimate of topographic similarity as in [42] to get reliablenumbers.
We sub-sample 1000 elements x from the object space X , and sample the correspondingmessage m from the speaker’s policy ⇡ ✓ ( ·| x ) .
We compute the topographic similarity for this batch of 223 1000 pairs ( x, m ) .
We repeat this protocol 100 times and take the mean to measure compositionality.
Empirical results5.1 Visualizing the loss decomposition dynamics
We here visualize the loss decomposition dynamics.
Following the protocol of 3.2, we control L trainadaptin speaker’s loss with weight ↵ to understand the inﬂuence of the co-adaptation term on the language.
The co-adaptation task overﬁts rapidly We plot information and co-adaptation training dynamicsin the standard setting ( ↵ = 1 ).
Note that both train and test information losses quickly convergeto 0 , in other words the speaker succeeds in developing a protocol that is unambiguous on both thetraining set and the overall distribution.
On the other hand, the test co-adaptation loss diverges whilethe train co-adapation keeps dismishing, highlighting a clear overﬁtting problem.
The co-adaptation task promotes generalization We then display in Figure 2 the evolution of theinformation and co-adaptation losses for different co-adaptation weight ↵ (models selected by earlystopping).
We observe that up-weighting L traininfo tends to enforce both information and co-adaptationoverﬁtting.
Thus, even though the co-adaptation loss overﬁts, it is important to encourage the speakerto build a better language.
This is conﬁrmed when looking at generalization accuracies.
From ↵ = 0to ↵ = 1 there is a gain of 15 points of generalization.
In conclusion, we note that (i) balancingthe loss in favor of L traininfo has a negative impact on generalization, (ii) the co-adaptation loss
L trainadaptpushes the speaker to develop a language that generalizes better.
These experiments highlight two key ﬁndings: (i) co-adaptation is crucial for generalization ; (ii) instandard settings, the co-adaptation loss overﬁts substantially, whereas the information loss does not.
5.2 Countering co-adaptation overﬁtting
We here investigate whether limiting overﬁtting in the co-adaptation loss may push towards languagesthat generalize better and are more stuctured.
As described in 3.3, we compare three control baselines:
Continuous listener , Partial listener with varying levels of convergence, and Early stopping listener .
[Figure]
Countering co-adaptation overﬁtting improves generalization In Figure 3, we observe that thelevel of convergence of the Partial listener between each speaker’s update (controlled by N step )has a strong impact on the generalization of the emergent protocol.
Overall, we recover classicmachine learning trends when varying N step : when N step < 50 , both train and test accuracy are low— the agents underﬁt.
When 50 < N step < 250 , the train and test accuracy are almost optimal —the agents are in good training regime.
Finally, when N step > 250 , the train accuracy is maximalwhile the test accuracy collapses — the agents overﬁt.
These observations reveal that the levelof convergence of the listener has a substantial impact on the ﬁnal emergent language capacity togeneralize.
Recall that, in these experiments, the direct effect of the listener’s overﬁtting is mitigated, as we measure generalization using an auxiliary listener that is early stopped, and should thereforenot overﬁt as noted in Section 4.
The listener’s overﬁtting impacts the speaker’s update through theco-adaptation loss, which, by inducing a poorer ﬁnal language leads to a degradation in generalization.
Additionally, Figure 3 shows that the continuous listener, standard in the Lewis games literature,provides generalization performance similar to the worst overﬁtting listeners.
Controlling the listener’s co-adaptation level appears crucial to let the speaker develop a language thatgeneralizes well; this effect may have been underestimated in the standard Lewis learning dynamic.
Countering co-adaptation overﬁtting improves compositionality Figure 3 reveals that compositionality follows the same pattern.
In the underﬁtting regime, the topographic similarity is low but stilloutperforms the Continuous listener .
Similarly, it is also low in the overﬁtting regime.
In-betweenthe two — which corresponds to high generalization in Figure 3 — the topographic similarity reacheshigh values, which suggests that more compositional languages emerge.
This indicates that thelistener’s lack of co-adaptation overﬁtting promotes structured languages.
Compositionality correlates with generalization In Figure 3, we plot the correlation betweengeneralization and compositionality.
As opposed to [9], we observe a strong correlation betweengeneralization and topographic similarity when varying the Partial listener ’s level of convergence.
In particular, we identify two correlation branches: one belonging to the underﬁtting regime andthe second to the overﬁtting regime.
Together, they retrace the evolution of generalization andcompositionality with respect to N step .
We see that Continuous listeners belong to the end of thistrajectory, in the overﬁtting regime.
Note that the blue rectangle — which delineates the range ofvalues reached with the Continuous listener — corresponds to the classic learning setting in theliterature.
As shown, this range is tight, which may explain the initial negative results reported by [9].
In conclusion, the listener exerts a necessary pressure on the speaker to develop a structured languagethat generalizes better.
This pressure can be controlled by limiting the listener’s level of overﬁtting,which is inevitably too high when the listener is trained continuously as is usually done.
Comparison with standard regularization methods In practice, re-initializing the listener as donewith the Partial or Early stopping listener is costly.
We thus test whether performances comparable to
Figure 3 can be obtained by controlling the listener’s level of overﬁtting with standard regularizationmethods.
In Table 1, we report the inﬂuence of applying common regularization methods to thelistener on various metrics of the language.
We ﬁnd that regularization consistently results innoticeable improvements.
Moreover, once again, gains of generalization correlate with gains ofcompositionality.
These trends corroborate our hypothesis that controlling the listener’s learning iskey to encourage the speaker to develop more structured languages.
However, those methods remainunder the upper bound reached by the Early stopping listener , which suggests that further researchon regularization in cooperative games is warranted.
We complement this analysis in Appendix C.2 by studying the impact of regularization on thespeaker’s side, and show that such regularization does not result in similar improvements.
Thisindicates that the listener is the main contributor to the co-adaptation overﬁtting.
5.3 Scaling to the Image Discrimination Games
To validate our empirical ﬁndings beyond synthetic games, we scale our approach to complex gameswith natural images as advocated by [11].
We thus train our agents on a discriminative game on topof the CelebA [52] and ImageNet [60, 18] datasets while applying previous protocol.
We work on3 sizes of training set with increasing generalization difﬁculty.
We provide all the training detailsand game settings in Appendix D.1 and report our results in Table 1.
While agents generalize wellwhen trained on the entire training set, generalization issues occur on smaller training sets andperformances can indeed be improved by controlling the listener’s level of convergence.
However,
Appendix D.2 shows that gain of generalization does not correlate with gain of topographic similarity,supporting that agents’ language structure is not captured by the topographic similarity in imagebased settings [11, 1].
Related work
The decomposition of the loss function in the Lewis Game that we introduced ﬁnds echos in thecognitive science literature.
According to Skyrms [61], communicative organisms or systems areconfronted with two types of information: about the environmental states shared by the agents
[Table]
(called objective information), and about how an agent would react to a signal (called subjectiveinformation).
Communication protocols emerge as a trade-off between constraints related to thosetwo types of information [40, 41]: the sender should be expressive [22, 21] and transcribe theinformation available in the world with as little ambiguity as possible, which has been described as abias against ambiguity [64] ; sender and receiver should agree on the same referring system, whichhas been described as a conceptual pact [7].
The latter has been shown to impose compressibilityand learnability pressures promoting structure [69, 63, 75].
This analysis resonates well with ouranalytical decomposition of the loss function in the Lewis game.
The ﬁrst term of the decomposition, which we called the information loss, has been addressed byprevious work that assumed that linguistic structure and generalization emerge from the requirementof creating an unambiguous language.
In this line of work, studies have either manipulated thecomplexity of the environment [11, 26, 62, 53], restricted the bandwidth of the communicationchannel [44, 57], or added noise to the message [46, 76].
In our main experiment, we do notapply such information constraints to better focus on the second term of the decomposition, theco-adaptation constraint, less studied within a machine learning approach.
Previous work haveassumed that the co-adaptive dynamics encourage speakers to develop a more structured language forlearnability reasons [51].
Support for this hypothesis can be found directly via the implementation ofa neural variant of Iterated Learning [56] or the introduction of learning speed heterogeneities [59]and indirectly via the restriction of agents capacity [57], the variation of the communication-graph inpopulations [25, 37] or the addition of newborn agents [14].
In our paper, we demonstrate that a coadaptation term is always present in standard agents optimization protocols and show that controllingco-adaptation overﬁtting enhances language properties.
The existence of an overﬁtting regime foundunder the default setting (continuous training) may explain the counter-intuitive lack of relationshipbetween compositionality and generalization previously reported with neural agents [47, 9, 34, 19].
Conclusion
In this paper, we propose a methodological approach to better understand the dynamics in Lewissignaling games for language emergence.
It allows us to surface two components of the training:
(i) an information loss, (ii) a co- adaptation loss.
We shed light that the agents tend to overﬁt thisco-adaptation term during training, which hinders the learning dynamic and degrades the resultinglanguage.
As soon as this overﬁtting is controlled, agents develop compositional languages that bettergeneralize.
Remarkably, this emergent compositionality does not result from environmental factors,e.g. communication bottleneck [39], under-parametrization [44, 23], population dynamics [11, 59],memory restriction [14, 15] or inductive biases [58], but only through a trial-and-error process.
Therefore, we advocate for a better comprehension of the optimization and machine learning issues.
As illustrated in this paper, such understanding may unveil contradictions between computationalmodels and language empirical observations and better expose the existing synergies between learningdynamics and environmental factors [24, 74, 55, 13, 17, 20].