E XPRESSIVITY OF E MERGENT L ANGUAGES IS
A
T RADE - OFF BETWEEN C ONTEXTUAL C OMPLEXITY AND U NPREDICTABILITY
A BSTRACT
Researchers are using deep learning models to explore the emergence of language in various language games, where agents interact and develop an emergent language to solve tasks.
We focus on the factors that determine the expressivity of emergent languages, which reﬂects the amount of information about input spaces those languages are capable of encoding.
We measure the expressivity of emergent languages based on the generalisation performance across different games, and demonstrate that the expressivity of emergent languages is a trade-off between the complexity and unpredictability of the context those languages emerged from.
Another contribution of this work is the discovery of message type collapse, i e. the number of unique messages is less than that of inputs.
We also show that using the contrastive loss proposed by Chen et al.
(2020) can alleviate this problem.
I NTRODUCTION
Language games were ﬁrst introduced by Wittgenstein (1954) to explore the meanings of language utterances.
Instantiating this concept with the signalling game design from Lewis (1969) enables linguists to explore the emergence of linguistic structure (Kirby, 2001; Kirby & Hurford, 2002) where artiﬁcial languages are represented as symbolic systems.
The success of deep learning (DL) models on complicated cognitive tasks (Krizhevsky et al., 2012; LeCun et al., 2015; Silver et al., 2016) then inspired researchers to apply DL-based models to language games to investigate the agents’ ability to invent communication protocols without preset rules (e.g. Lee et al., 2018;
Lazaridou et al., 2018).
In the existing works (e.g. Lee et al., 2018; Lazaridou et al., 2018; Graesser et al., 2019; Zhang et al., 2019; Yuan et al., 2020), there are usually two types of agents, speakers that emit messages based on their observations (i.e. input target objects) and listeners that receive messages and act accordingly, as illustrated in Figure 1.
Based on the goals for listeners, we can categorise most of the games into the following three types: 1) referential games in which listeners need to select the target object observed by the speaker among a set of candidate objects (candidates for short) (e.g. Lazaridou et al., 2017; Ren et al., 2020); 2) reconstruction games in which listeners need to reconstruct the speaker’s observation (e.g. Chaabouni et al., 2020; Kharitonov et al., 2020); and 3) navigation games in which listeners need to go to the location speciﬁed by speakers (e.g. Lowe et al., 2017; Kaji´c et al., 2020).
We focus on referential games, illustrated in Figure 1a, which have been well investigated in both the linguistic community (e.g. Kirby et al., 2014; Winters et al., 2018) and emergent communication community (e.g. Lazaridou et al., 2018; Li & Bowling, 2019).
It is reasonable to assume that listeners in referential games can differentiate the target object from the other distractors in the context as long as the speaker’s message encodes some unique feature of the target.
Therefore, the speaker’s messages must convey a different amount of information for listeners to complete their task in games where candidates are similar in different degrees.
For example, to distinguish two very similar candidates, e.g. blue wide-stripped shirt and blue narrow-stripped shirt, it requires more information about the target to be encoded by the speaker’s message than to distinguish a shirt from a cup.
Experiments with human participants show that emergent languages are sensitive to the requirements of the communicative tasks for which they are used, with languages developing in which only the necessary discriminatory information is encoded (Winters et al., 2018).
[Figure] [Figure]
Following Kirby et al.
, we refer to such ability to encode information about input space (e.g. structure, dimensionality, distribution of samples) as expressivity .
In this work, we explore the factors that inﬂuence expressivity in the framework of DL-based referential games, and we argue that it is important to understand these factors such that agents could develop languages that are effective enough for completing various tasks.
Our contribution in this work is threefold.
First, we propose and verify a hypothesis about the determining factors of expressivity of emergent languages under the DL-based framework.
Following
Winters et al.
, we show that context size (i.e. the number of candidates the listener must select among when interpreting the speaker’s message) inﬂuence both the complexity and unpredictability of that context.
Complexity refers to the similarity between the items in the context: in order for the listener to select the target, more information needs to be encoded in the speaker’s signal if the context contains more similar distractors for a given target.
Unpredictability refers to the extent to which the information that needs to be encoded by the speaker is stable across contexts in successive episodes of communication: in games where the information required by the listener differs from trial (training epoch for DL agents) to trial, the speaker needs to encode more information on every trial in order to be sure to encode the information the listener needs.
1 As we show in Section 3.1, complexity and unpredictability are affected differently by the context size: as context size increases, complexity increases but unpredictability decreases.
Therefore, we propose and verify the following hypothesis about the determining factors of expressivity:
[H YPOTHESIS ] The expressivity of emergent languages is a trade-off between the complexity and the unpredictability of context in language games.
Our second contribution is a novel measure of expressivity based on partial ordering of languages in terms of their generalisation performances across tasks.
Although expressivity is related to the amount of information encoded in a language, we illustrate that mutual information (MI) is not an appropriate measurement for expressivity in Section 3.2.
Considering that one of our objectives is to facilitate a universally useful emergent language, we propose to measure expressivity of a language based on the generalisation performance of listening agents trained with that language across different games.
Since it is more challenging to quantitatively and directly measure expressivity, we focus on the partial ordering between languages in this ﬁrst step towards understanding expressivity.
Our ﬁnal contribution is the discovery of message type collapse, i.e. the number of unique messages is signiﬁcantly lower than the size of input space in relatively easy games, which can lead to highly ambiguous emergent languages.
To overcome the technical limitations imposed by GPU memory size on large-scale referential games, we introduce the contrastive loss proposed by Chen et al.
(2020) in referential games.
While comparing the behaviour of the contrastive loss and the loss function used in previous works (e.g Havrylov & Titov, 2017; Li & Bowling, 2019; Ren et al., 2020; Guo et al., 2020), we ﬁnd that this contrastive loss can greatly alleviate the collapse of message types, leading to more disambiguous and potentially more expressive emergent languages.
G AMES , A GENTS AND L EARNING M ETHODS 2. G AME E NVIRONMENT AND A RCHITECTURE OF A GENTS
Environment Components As shown in Figure 1b, formally speaking, the environment in our referential games consists of: 1) input space X from which the speaker’s observations are drawn, which consists of 10 , 000 possible inputs 2 where each x P X is a concatenation of 4 one-hot vectors whose length is 10 ; 2) message space M consists of 10 6 6-tokens-long sequences m (called as a message type ) where the size of the token inventory is 10 ; note that the tokens are initially meaningless, and meaning emerges through the negotiation of a communication protocol between speakers and listeners; 3) candidate set C which contains a target input x t and | C
| ´ 1 distractors t x d 1 , x d 2 , . . .
, x d | C |´ 1 u sampled uniformly at random from X without replacement.
Agents The architecture of the agents we implemented is also illustrated in Figure 1b: 4) speaker S consists of both a multi layer perceptron (MLP) for encoding the 40 -digits-long target input x t onto a 256 -digits-long embedding h S , and a decoder for generating a message m based on long-short term memory (LSTM) network (Hochreiter & Schmidhuber, 1997).
5) listener L is constituted by an
LSTM-based sequence encoder which encodes m into a 256 -digits-long embedding h L p m q , and
an
MLP for encoding the inputs which can be denoted by f L cenc p¨q .
Deﬁnition of an emergent language Following e.g. Ren et al.
(2020), we also deﬁne an emergent language L as a mapping function from the input space X to the message space M , i.e. L : X ÞÑ M , thus it can be represented as tp x i , L p x i qq|@ x i P X u .
Note that L could be a non-injective mapping function, i.e. it is possible for two different inputs to be mapped to an identical message thus the number of message types might be less than the number of unique inputs.
2. L EARNING METHODS OF AGENTS
In the previous works (e.g. Havrylov & Titov, 2017), the action module of L is usually a softmax function which produces a categorical distribution over the candidates based on the distance between h L p m q and embeddings of each candidate ( f Lcenc p x i q ) obtained through listener’s candidate encoder f Lcenc p¨q .
By comparing the loss function used in Havrylov & Titov (2017) and the contrastive loss function proposed by Hadsell et al.
(2006), we ﬁnd that the aim of both functions is the same, i.e. to make the distance between positive pairs ( x t and m ) closer while keeping the negative pairs ( x d and m ) as far apart as possible.
To effectively validate our hypothesis, we need to make the whole input space X as candidate set C in which case the batch is too large to be stored in GPU memory (since the space complexity is O p| X | 2 q ).
Therefore, to overcome this issue, we adapt the contrastive learning method proposed by Chen et al.
(2020) to referential games.
Suppose that we have a batch of input samples B “ t x 1 , x 2 , . . .
, x i , . . .
, x | B | u and the corresponding messages from the speaker are t m B 1 , m B 2 , . . .
, m B i , . . .
, m B | B | u (note that it is possible that m B i “ m B j for different i, j ) which would be further encoded into t h L m B 1 , h L m B 2 , . .
h L m Bi , . .
h L m B | B | u by the listener L .
Then the loss function for a sample x i is the deﬁned on a softmax function:
[Equation]
In this way, the number of candidates | C | is exactly the batch size | B | during training.
More details about the comparison between referential loss and this contrastive loss can be found in Appendix A.
As for updating the parameters, we use the Adam algorithm introduced by Kingma & Ba (2015), and the learning rate is set to 10 ´ 4 .
To allow the gradients being propagated through the discrete channel to overcome the sampling issue of messages, we apply the Gumbel-Softmax trick proposed by Jang et al.
(2020), and the temperature hyper-parameter τ is set to 1 . 0 .
Our implementation 3 of games is based on the framework EGG developed by Kharitonov et al.
The experiments across 6 random seeds, 18 source games, 11 target games took 4 , 216 hours in total, on Nvidia Tesla P100.
Deﬁnition of “source/target games” are given in the following Section 4.2.
C ONTEXT AND E XPRESSIVITY 3. C ONTEXT , AND I TS C OMPLEXITY AND U NPREDICTABILITY IN D IFFERENT G AMES
Context Communication between humans always takes place w.r.t.
some context (Winters et al., 2018).
For humans, that context includes: 1) features of the environment that can be directly perceived by the dialogue participants and used for interpreting an utterance (e.g. “cup” is interpreted as an object in the immediate physical context); and 2) information that cannot be directly perceived but can affect the interpretation of utterances (e.g. using and interpreting “cup” assumes some shared knowledge of what counts as a “cup”; once a particular cup has been established as salient in particular dialogue, further utterances of “the cup” or “it” will be interpreted as referring to that speciﬁc cup).
Inspired by this conclusion from pragmatics, in the DL-based language games, we deﬁne context as the supervision information involved in the calculation of losses, i.e. the context of a speciﬁc target x t , denoted as C p x t q , is the space of samples involved in the calculation of loss.
As for in referential games, the cross entropy loss is calculated based on the distance between the message embedding h L p m B i q and candidate embedding f Lcenc p x i q where x i P C , thus C p x t q “ C Ď X in referential games.
Complexity of context We assume that the goal of communication is to distinguish a target object from other possibilities in the context (as deﬁned above).
It therefore follows that the similarity of distractors in the context to the target inﬂuences the communicative precision required, and that greater precision is required to distinguish the target from a more similar distractor.
For example, it is relatively easy in natural language to distinguish e.g. a cat from a table (a relatively general label like "cat" or "animal" would sufﬁce), but harder to make ﬁne-grained distinctions between very similar objects e.g. a Scottish Fold Cat and an American Shorthair Cat (a specialist vocabulary or a lengthy description is necessary).
Following the assumption (described verbally above) that a context which contains more similar objects makes the game harder because there are fewer unique features that sufﬁce to distinguish the target from the distractors, we ﬁrst deﬁne a neighbour set in k -th degree of a given target x t as N k p x t q “ t x j : d p x t , x j q ď k u where d is a distance function that can properly capture the similarity between inputs, e.g Hamming distance in our setting.
The complexity of C p x t q is then deﬁned as the expectation of the probability that C p x t q contains an object from N k p x t q , i.e.
E x t r g p C p x t q , N k p x t qqs ,where g p C p x t q , N k p x t qq
[Equation]
In our referential games, since the sampling procedure is independent Bernoulli without replacement, the value of the above expectation is then 1 ´
[Equation]
¯ | C | which is a monotonically increasing function w.r.t | C | and a ﬁxed k . That said, larger contexts are more complex since they are more likely to include items which are very similar to the target.
Unpredictability of context Our notion of unpredictability comes from experimental work with human participants, e.g. Winters et al.
Suppose the aim is to distinguish a striped t-shirt from a number of distractors, and there are two sequences of context: 1) three runs of games where distractors are all cups; 2) three runs of games where distractors are a cup, a plain t-shirt, and a pencil.
In the ﬁrst sequence of games, participants would become certain that “t-shirt” is enough for distinguishing the target, whereas in the second sequence participants would learn that a more overspeciﬁed utterance (e.g. "striped t-shirt") is necessary to guarantee comprehension after a failure on the trial involving the plain t-shirt distractor.
That is, the context in the ﬁrst sequence is more predictable than the second.
Winters et al.
(2018) show that human participants are sensitive to this kind of unpredictability, and adapt their communicative strategies accordingly.
In DL-based games, we refer to the context at the e -th trial, i.e. the e -th training epoch, of a target x t as C e p x t q .
Following the above example, the unpredictability of context is then deﬁned as the proportion of C e ` 1 p x t q that are not from C e p x t q , i.e.
[Equation]
ﬁ ﬂ (3)
In our referential games, since the sampling procedure is independent Bernoulli without replacement, the proportion of objects not from C e p x t q in C e ` 1 p x t q is then simply 1 ´ | C
e p x t q| | X | “ 1 ´ | C | | X | (i.e. smaller contexts are more unpredictable, since contexts on successive trials are more likely to differ in their composition).
More details about the above discussion are provided in Appendix B.
3. M EASURING E XPRESSIVITY IN L ANGUAGE G AMES
Following the intuition that the different types of games require messages to convey different amount of information about input space X , MI may seem to be a good candidate measure for expressivity.
However, we found that even for two emergent languages that are both deterministic and bijective mapping functions, i.e. they have the same MI value log p| X |q , their expressivity can still be different.
Further discussion about MI can be found in Appendix D.
Since it is more challenging to directly quantify the expressivity of emergent languages, we instead focus on the partial orders between them in this work.
Considering that the purpose of the emergent communication is to complete tasks, we therefore propose to measure the partial order between expressivity based on game performance.
This leads us to a common practice in the transfer learning community (e.g. Pan & Yang, 2009), i.e. train models on source domain but evaluate them on different target domains.
Inspired by this practice, here we propose a deﬁnition of the partial order between the expressivity of two emergent languages L A and L B , i.e. E L A and E L B :
Deﬁnition 3.1 ( E GL A ą E GL B ) .
Let g be an instance of language game, and G be a set of language game instances.
Then, given G , if the generalisation performance of L A is better than L B on some language games G 1 with statistically signiﬁcant difference, i.e. @ g P G 1 , P g L A ą P g L B , where ∅ Ă G 1 Ď G and P denotes the generalisation performance, while there is no statistically signiﬁcant difference between the converged generalisation performance of L A and L B on the remaining games, i.e. P g 1 L A « P g 1 L B @ g 1 P G z G 1 4 , we then say that expressivity of L A is higher than L B on G , i.e.
E GL A ą E GL B .
The metric of generalisation performance could be varied according to different types of games.
Brieﬂy speaking, given a set of games, if one language performs better than the other on some games while they perform approximately the same on the remaining games, we then say the expressivity of that language is higher than the other.
Following this deﬁnition, there are several possible relationships between the expressivity of two languages L A and L B : 1) E G L A ą E G L B , deﬁned as above; 2)
E GL A “ E GL B , if P g L A « P g L B , @
g P G ; 3) E GL B ą E GL A , deﬁned as above; 4) E GL A is incomparable to E GL B ( E L A (cid:1)(cid:1) £ E L A ), if propositions D g 1 P G , s.t.
P g 1 L A ą P g 1 L B and D g 2 P G , s.t.
P g 2 L B ą P g 2 L A are both true.
P REDICTIONS AND E XPERIMENT D ESIGNS 4. P REDICTIONS F ROM O UR H YPOTHESIS
As we illustrated in Section 3.1, higher complexity indicates more similar distractors to targets, thus requires messages conveying more ﬁne-grained features of inputs.
This leads to the following prediction:
higher complexity in games with same contextual unpredictability can facilitate emergent languages having higher expressivity
Meanwhile, following the conclusion from Winters et al.
(2018) that higher contextual unpredictability leads to higher signal autonomy, i.e. more comprehensive information about input space is encoded in the emergent languages, we then make the following prediction:
higher unpredictability in games having same contextual complexity can facilitate emergent languages having higher expressivity;
Following the above two predictions, both higher complexity and higher unpredictability can unilaterally lead to better expressivity.
However, with an input space whose size is ﬁnite, changing context size always affects the two factors in an inverse way, which leads to our Hypothesis 1 and gives the following prediction:
if the size of X is ﬁxed across games, a language which emerges in a game with moderate complexity and unpredictability should have the highest expressivity.
L ANGUAGE T RANSFER E XPERIMENT
To verify the above three predictions and thus our hypothesis, we evaluate the generalisation performance of languages emerging in various games following the Procedure 1.
Brieﬂy speaking, we ﬁrst train a pair of speaker and listener to complete a referential game, i.e. a “source game” g s , and obtain an emergent language.
We then train a new listener in another referential game, i.e. a “target game” g t , with only 90% of the input-message pairs, and use the game performance of this listener with the remaining 10% pairs as the generalisation performance of g s on g t .
Procedure 1: Procedure for the language emergence and transfer experiment Input for every game g in G do
initialise a new speaker and listener for g is , and train them to play g is with the whole X ; 2.
after the agents converge on g is , record L “ tp x , m q| x P X u ; 3.
randomly shufﬂe and split L into 2 disjoint sets L train and L test s.t.
| L train | “ 90% ¨ | L | ; 4.
for every game g jt in G t do 1.
initialise a new listener for g jt ; 2.
train the listener with L train to complete g jt ; 3.
record the accuracy of listener on L test as the generalisation performance of g is on g jt ; end end
Since our hypothesis concerns the size of C , we propose to experiment on a series of referential games with varying candidate set size | C | (“referX” where X “ | C | ).
Meanwhile, as these referential games have both different complexity and unpredictability, we also implement a series of referential games with different | C |
but the batches are ﬁxed during training such that there is no unpredictability of the context (“referXf” where X “ | C | ).
To sum up, in our experiment, G s “ t “refer2/f” , “refer10/f” , “refer100/f” , “refer1000/f” , “refer2500/f” , “refer5000/f” , “refer7500/f” , “refer10000” u 5 .
Since language games with ﬁxed context are not good simulation of human communication , we keep only the ones with varying batches from G s in G t .
As a naming convention to allow us to refer easily to the factors that inﬂuence a language’s expressivity, we refer to emergent languages by the parameters of their source game, e.g. ‘refer10’ 6 represents a language which emerges in referential games whose | C | “ 10 and context varies across epochs.
More details about the naming of games and languages can be found in Appendix E.1.
R ESULTS & D ISCUSSION
To alleviate the effect from randomness as much as possible, all the following results are obtained with 6 different random seeds.
All values are averaged across the seeds, and the corresponding standard deviation (SD) are also provided (either in this section or appendices).
5. H IGHER C OMPLEXITY ` S AME U NPREDICTABILITY Ñ H IGHER E XPRESSIVITY
To verify the Prediction 1, we compare the generalisation performance of languages from ﬁxed context, i.e. the unpredictability of source games’ context are the same, on G t , and the results are shown in Figure 2.
Languages which emerge in source games whose contexts have higher complexity consistently perform better on all games than those which emerge in source games with lower-complexity contexts, since all lines keep growing with the increase of values on x -axis (i.e. complexity).
That is, the generalisation performance is a monotonically increasing function of | C | if there is no unpredictability in the context, which matches with the prediction of our hypothesis.
[Figure]
5. H IGHER U NPREDICTABILITY ` S AME C OMPLEXITY Ñ H IGHER E XPRESSIVITY
To verify the Prediction 2, we compare the difference between the generalisation performance of language with varying context (‘referX’) and their corresponding ﬁxed-context twins (‘referXf’).
As indicated by that all points in Figure 3 are positive, languages from context with varying objects always perform better on all target games.
Since all “referXf” has lower unpredictability of context than the corresponding “referX” games, this indicates that higher unpredictability can unilaterally lead to better generalisation performance of emergent language across various target games, which matches with the prediction of our hypothesis. [Figure]
[Figure]
A cc u r a c y D i
ff e r e n c e
5. E XPRESSIVITY IS A T RADE - OFF BETWEEN C ONTEXTUAL C OMPLEXITY AND U NPREDICTABILITY
To verify the Prediction 3, we compare performance for all source games with varying context on all target games, and the results are shown in Figure 4 (see also Appendix E.3 for more details).
[Figure]
As shown in Figure 4, on whichever type of game, the peaks of all curves are always among “refer1000”, “refer2500”, or “refer5000”, which means that the best expressivity always emerges in these 3 types of games.
Considering that the complexity of context in referential game is a monotonically increasing function of the | C | | X | while the contextual unpredictability is a monotonically decreasing of it, the results shown in Figure 4 support our hypothesis, i.e. the expressivity is indeed a trade-off between the contextual complexity and unpredictability.
To be speciﬁc, although context in “refer2”, “refer10”, and “refer100” have high unpredictability, their complexity is low thus the languages have lower expressivity.
On the other hand, context of “refer7500” and “refer10000” have high complexity, but expressivity of neither language is the highest due to the high predictability of the context in these games.
This supports our argument that unpredictability of context incentivises the agents to encode more information into the messages such that the messages could overcome the uncertainty of contexts and be interpreted in various contexts.
Therefore, the three languages (‘refer1000’, ‘refer2500’, and ‘refer5000’) from context with both moderate complexity and unpredictability perform better than the others.
To make it more straightforward to compare the expressivity between these languages, we also provide another view of the above results in Figure 9 in Appendix E.3, which shows that ‘refer1000’ (cid:1)(cid:1) £ ‘refer2500’ « ‘refer5000’ ą ‘refer7500’ « ‘refer10000’ ą ‘refer100’ ą ‘refer10’ ą ‘refer2’.
To eliminate the possibility that the above phenomena are caused by the hyperparameters we used in our games, we also run the language transfer experiment with varying communication channel capacity and agent capacity (i.e. the size of hidden layers).
As shown in the ﬁgures in Appendix F, both ‘refer1000’ and ‘refer2500’ perform better than ‘refer10000’ across varying hyperparameters, which shows that the trade-off phenomenon is consistent over game settings and agent capacity.
5. T HE COLLAPSE OF MESSAGE TYPES
As we use a contrastive loss instead of the conventional referential loss applied by (e.g. Havrylov & Titov, 2017; Ren et al., 2020), we also compare the behaviours of both loss functions on G 1 s “
G 1 t “ t “refer2” , “refer10” , “refer100” u .
We discover that the number of message types is some times greatly less than the size of input space.
We refer such phenomenon as “message type collapse”, and the mappings from several different inputs to a same message type as “degeneracy components”.
While comparing the two loss functions, we observe that contrastive loss is more efﬁcient on removing degenerate components in large-scale games (where | C | ě 100 ), and can better avoid the collapse of the number of message types.
To be speciﬁc, as shown in Figure 5a, the numbers of message types collapses to less than 500 on games with conventional referential loss.
They, however, would only decrease to roughly 5 , 000 on ‘refer10’ and ‘refer1000’ with contrastive loss as shown in Figure 5b.
The contrastive loss can also make the training of referential game more stable, as illustrated by the smaller variance in Figure 5b.
Our results indicate that it is necessary to establish a referential game with candidate sets containing enough many distractors to guarantee that the emergent language will be relatively unambiguous.
As far as we know, the existing works do not investigate the effects of the size of candidate sets, e.g. Lee et al.
(2018) set | C | “ 2 , Li & Bowling (2019) set | C | “ 5 , and | C | ď 20 is used by Lazaridou et al.
Given our results, our heuristic is that | C | should be greater than either 1 , 000 for large input spaces or 110 | X | for smaller ones.
We also explore more about the degeneracy degree (as well as structure degree) of the mappings constituting different languages, and the results and discussion are provided in Appendix C.
[Figure]
C ONCLUSION & F UTURE W ORK
To our knowledge, this is the ﬁrst work exploring the expressivity of emergent languages from language games played by DL-based agents that have different context size.
We ﬁrst proposed a hypothesis about the factors inﬂuencing the expressivity of emergent languages, and deﬁned a partial ordering between expressivity of different emergent languages based on the generalisation performance across a set of games.
Through a series of experiments on referential games with varying size of context, we then validated the three predictions of our hypothesis, which furthers indicates that expressivity is indeed a trade-off between the complexity and unpredictability of context in language games.
To overcome the memory inefﬁciency issue caused by the conventional referential loss function, we introduced a contrastive loss into our implementation.
In the comparison between this contrastive loss and the referential loss, we found the message type collapse problem, and also showed that using this contrastive loss can alleviate it.
Since this is a ﬁrst step towards deﬁning and understanding the expressivity of emergent languages, there are still some questions unanswered by this work.
For example, some properties of our emergent languages are surprising given the simplicity of some of our referential games.
In the simplest setting ( | C | “ 2 ), we see emergent languages with more than 2 , 000 message types, whereas an optimal strategy requires a language consists of only 40 unique messages (since there are only 40 different values for all properties in X and two items in the context always differ on at least one property).
Thus, it shows that further investigation about how redundancy helps during the communication between agents is necessary.
More importantly, we lack understanding of the structure of mappings constituting a language, especially when the languages are generated or encoded by neural networks.
We believe that the expressivity of languages still needs further exploration in order to facilitate languages that are universally effective across various tasks. A CKNOWLEDGEMENT
This project has also received funding from the European Research Council (ERC) under the European
Union’s Horizon 2020 research and innovation programme under grant agreement No. 681942, held by K. S.
A M ORE D ETAILS ABOUT C OMPARISON BETWEEN D IFFERENT L OSS F UNCTIONS
In this section, we give more details about the comparison between the referential loss used by previous works and the contrastive loss we use in Section 2.2.
Following the same notations used in Section 2.2, the referential loss used by (e.g. Havrylov & Titov, 2017) can be written as:
[Equation]
where h L m is the embedding for the message m obtained from the sequence encoder of listener L , and f Lcenc p¨q is the candidate encoding function of L .
From the above equation, we can see that if a candidate has an embedding closer to message m , the probability mass for predicting it would then be higher.
Hence, the predicated probability mass is inversely proportional to the distances, e.g. the dot product in the above equation, between the embeddings of m and all candidates.
The contrastive loss function proposed by Chen et al.
(2020) with τ “ 1 is given as follow:
[Equation]
where I is an indicator function.
By comparing the above loss function with Equation 4, we ﬁnd that the aim of them is the same, i.e. make the distance between positive pairs (targets and their corresponding messages) closer while keeping the embeddings for negative pairs (distractors and the messages) as far apart as possible.
While negative sampling has been deeply investigated in the self-supervised learning (SSL) community, (e.g. Chen et al., 2020; Grill et al., 2020), it has not been discussed in the emergent communication community yet, thus we simply use all randomly sampled distractors as negative examples.
As mentioned in Section 2.2, we need to make the whole input space as candidate set, thus the above contrastive loss which has lower space complexity is preferred.
In the following, we give a bit further derivation to show that Equation 1 is computationally equivalent to Equation 5.
By substituting k in the denominator of Equation 5 with j , we can see that:
[Equation]
B M ORE D ETAILS ABOUT C ONTEXTUAL C OMPLEXITY /U NPREDICTABILITY
In this section, we give more details about the deﬁnition of context and its complexity/unpredictability.
B. C ONTEXT
As deﬁned in Section 3.1, the context for a target object x t is the space of samples involved in the calculation of its loss.
We know that the prediction of the listener is calculated based on the distance between the message received ( m p x t q ) which is a function of the target input ( x t ), i.e. softmax p d 1 p m p x t q , x d qq where x d is a distractor and d 1 is a distance function for embeddings.
Cross entropy loss is then calculated based on softmax function.
Thus, we can also write the loss function as L p m p x t q , t x d u ; θ q .
As shown in the equation, the calculation is based on samples from candidate set C .
Therefore, in referential game, C p x t q “ t x t u Y t x d u “ C Ď X Ă O .
B. C OMPLEXITY OF CONTEXT
As deﬁned in Section 3.1, the k -th degree neighbour set is N k p x t q “ t x j : d p x t , x j q ď k u where k P Z ` .
As described in Section 2.1, there are 4 features in our synthetic data set and each of them can take 10 different values.
Therefore, for a speciﬁc target x t , the k in the deﬁnition of neighbour set can take only the following 4 values:
• k “ 1 : samples in N 1 p x t q differ only in 1 feature from x t .
For the only different feature, there are only 9 possible values a distractor can take.
Thus, | N 1 p x t q| “ 4 ˆ 9 “ 36 .
• k “ 2 : samples from N 1 p x t q and samples that differ at exactly 2 features from x t .
For the later case, there are ﬁrst ` ˘ “ 12 possible combinations of such two features, and there are 9 ˆ 9 “ 81 possible samples for each combination.
Thus, | N 2 p x t q| “ 12 ˆ 81 ` 36 “ 1008 .
• k “ 3 : samples from N 2 p x t q and samples that differ at exactly 3 features from x t .
For the later case, there are ﬁrst ` ˘ “ 4 possible combinations of such three features, and there are 9 ˆ 9 ˆ 9 “ 729 possible samples for each combination.
Thus, | N 3 p x t q| “ 4 ˆ 729 ` 1008 “ 3924 .
• k “ 4 : all samples in X except x t are in this neighbour set, thus | N 4 p x t q| “ 10000 ´ 1 “
Since we use independent Bernoulli sampling, for each draw, the probability to get a sample from
N k p x t q is simply N k p x t q | X |´ 1 where | X | ´ 1 is due to excluding x t from X .
Meanwhile, since we deﬁne
C p x t q as the candidate set in referential games.
Therefore, for | C p x t q| “ | C | draws, the probability of having at one sample from N k p x t q is:
[Equation]
B. U NPREDICTABILITY OF CONTEXT
In this section, we ﬁrst derive our measure of unpredictability following Winters et al. In their experiments, the input space consists of 16 referents. There are 2 different attributes, colour and shape , and each attribute has 4 different values. By setting the size of candidate set to 4 , they deﬁne the following two types of context:
• shape-different : all the 3 different distractors have identical colour to the target, but different shapes.
Since there are only 4 shapes and candidate set size is also 4 , in this setting, the distractors for a given target will never change during the games.
This is exactly the same as our “ﬁxed-batch” games deﬁned in Appendix E.1.
• mixed : all the 3 different distractors are sampled uniformly at random from the input space, thus they may have different shapes as well as different colours to the target.
In this case, the distractors for a given target vary across different trials during the games.
By comparing the two above cases, it is straightforward to see that it is more possible to see different distractors for a given target in the games with mixed context.
Following this heuristic, we proposed to measure the unpredictability of context by the expected chance of observing different distractors for targets across training epochs, which further leads us to the deﬁnition given in Equation 3.
Note that there is also complexity (under our deﬁnition) involved in the above two types of context, we focus only on the variability part of the context in our deﬁnition of unpredictability since the complexity part has been taken into consideration by the neighbourhood set deﬁned in Section 3.1.
We then give further explanation about Equation 3 and its simpliﬁed form.
Since we use Bernoulli sampling for each individual object in candidate set, the overall sampling of candidate set is thus a binomial procedure.
As the probability of getting a sample from C e p x t q in X is | C
e p x t q| | X | , the overall proportion of objects from C e p x t q in C e ` 1 p x t q is the same, i.e. | C e p x t q| | X | .
Therefore, the probability of the complementary event, i.e. getting an object not from C e p x t q in C e ` 1 p x t q , is 1 ´ |
C e p x t q| | X | .
C M ORE R ESULTS AND D ISCUSSION ON E XPRESSIVITY
In this section, we provide two more intermediate properties of emergent languages that are also inﬂuenced by complexity and unpredictability of context, and thua may inﬂuence the expressivity.
C. D EGREE OF D EGENERACY A FFECTS BUT D OES N OT D ETERMINE THE E XPRESSIVITY
Inspired by Kirby et al.
, the ﬁrst intermediate factor we explore is the degree of degeneracy (i.e. the degree of ambiguity) of the languages.
As a language is deﬁned as a mapping function
L : X ÞÑ M , the degree of its degeneracy could be measured by the number of distinct message types.
As shown in the Table 1, in referential languages, the degeneracy degree decreases rapidly with the increase of the contextual complexity of the source games, and reaches a ceiling around | D | ě 1 , 000 .
There is a positive correlation between the number of message types and expressivity for the languages with low complexity, which indicates that degree of degeneracy could be an intermediate factor that is correlated with the expressivity of these languages.
However, the fact that the average number of message types on refer10000 is higher than refer1000, while the expressivity of ‘refer1000’ is higher than ‘refer10000’, i.e. emergent languages having lower level of degeneracy does not necessarily have higher expressivity, indicates that degeneracy degree of one language cannot determine its expressivity.
[Table]
C. S TRUCTURE DEGREE OF MAPPINGS OF DIFFERENT LANGUAGES
Since the degenerate degree is not the only factor determining expressivity, another factor to be considered is how the mappings of a language L are structured after the training procedure of agents on source games.
According to the cardinality of the mappings of L , we can divide them into two disjoint sets: i) one-to-one mappings, i.e. an input is mapped to a unique message; and ii) many-to-one mappings (a.k.a degenerate components), i.e. several inputs are mapped to the same message type.
Between this two types, the structure of one-to-one mappings has been explored under the view of compositionality in the previous works, e.g. (Brighton & Kirby, 2006; Andreas, 2019).
However, as illustrated by Chaabouni et al.
(2020), for emergent languages from DL-based agents, a good generalisability does not require a good compositionality, which means that measurements for compositionality are not good predictors for expressivity either.
In this work, we focus on only the many-to-one mappings.
Given the objectives of games, for the degenerate components in an emergent language, a message m being mapped to several closely-related inputs could imply that it captures common features among these inputs.
Suppose that several inputs X K “ t x 1 , . . .
, x K u are mapped to the same message m in a converged language L , we hypothesise that the average distances between every pair of inputs , i.e.
1 | X K | ř i,j Pt 1 ,...,
K u d p x i , x j q , would become smaller after training, compared with their random initial values.
Therefore, we compare the averaged distances between meanings of degenerate components as well as the frequency of degenerate components at the beginning and end of training.
To make the plots more readable, we only plot the statistics about the top- 10 most frequent ambiguous message types, where the frequency of a message type is deﬁned as the number of inputs that are mapped to it in a languages.
Results on ‘refer1000’ (one of the most expressive language in our experiments) are shown in Figure 6.
By comparing the data of beginning and end of training in Figure 6b, we can easily observe that the averaged distance decreased throughout the training, which indicates that the degenerate components actually became more although not fully structured. Another observation we have limited understanding of is that ‘refer1000’ becomes more ambiguous after training (shown by that the red bars F r e q u e n c y {+beginningend (a) Frequencies of degeneracy components         
+}
D i s t a n c e beginningend
[Figure]
representing the statistics at the end of training are lower than the statistics the beginning training), although its accuracy does improve to almost 100% .
However, limited by the scope of this work, we will leave further exploration of it to future works.
D M UTUAL INFORMATION IS INSUFFICIENT
In Section 3.2, we mentioned that MI cannot completely reﬂect the expressivity of an emergent language.
Here, we provide more detailed explanations.
On one hand, a higher MI value does not necessarily mean more task-relevant information, e.g. encoding colour in messages is unnecessary for tasks involving only the shapes of inputs although it provides more information.
On the other hand, not only the determinacy of the mappings matters for generalising them, but also the structures, e.g. it would be easier for humans to extrapolate compositional languages to novel concepts than holistic ones although they are both unambiguous and therefore have the same MI.
In the following paragraphs, we will give more reasoning and empirical evidence to support our claim.
Following the mapping function deﬁnition of emergent languages illustrated in Section 2.1, suppose that we get a message type collection M Ă M in some training epoch, the calculation of the mutual information between M and X is given as follows:
[Equation]
p x , m q log p p x , m q p p x q p p m q p p x , m q “ 1 | X | “ ÿ x P X ÿ m P M | X | log p p x , m q p p x q p p m q “| X | ÿ x P X ÿ m P M log p p x , m q p p x q p
p m q “| X | ÿ x P X ÿ m P M p log p p x , m q ´ log p p x q ´ log p
p m qq p p x q “ 1 | X | “
N ÿ x P X ÿ m P M ˆ log| X | ´ log | X | ´ log p
p m q ˙ “| X | ÿ x P X ÿ m P M p´ log p p m qq “ ´ ÿ m P M log p
p m q
“ ´ ÿ m P M log | L p m q| | M | “ ´ ÿ m P M p log | L p m q| ´ log | M |q
“ ÿ m P M p log | M | ´ log | L p m q|q where p p x q “ 1 | X | as x follows a uniform distribution over the input space, p p x , m q “ 1 | X | since every x is mapped to a message, and | L p m q| is the appearance number of m in one language.
[Figure]
By observing the above equation, it is straightforward to see that the MI value is decided by the number of message types, i.e. the degeneracy degree of one language.
Therefore, if two emergent languages have approximately the same number of messages types (as shown in Table 1 and degeneracy degree, the MI values for these two languages should also be the same.
However, as we illustrate in
Section C.1 that degree of degeneracy cannot determine the expressivity of emergent languages, we can then deduce that MI cannot fully measure the expressivity of emergent languages.
To provide more empirical evidence to support our claim, we tracked the MI values over training epochs on a subset of G s , and the results are shown in the following Figure 7.
As we can see, there is no statistically signiﬁcant difference between the MI values of ‘referX’ (where X ě 1 , 000 ) while it has been shown in Section 5.3 that their expressivity is different.
[Figure]
[Table]
E L ANGUAGE T RANSFER E XPERIMENT R ESULTS E. N AMING OF G AMES AND L ANGUAGES
In this section [Table] we give more details about the naming of games and corresponding languages we used in Section 5 as follows:
• “referX”: this refers to the referential games with candidate sets whose size is X (which itself is an integer), note that we use double quotation marks to indicate that this is a speciﬁc type of game.
For example, “refer2500” refers to referential games with 2 , 500 candidates for listeners.
• ‘referX’: this refers to a emergent languages from “referX”, note that we use apostrophes here in order to differentiate it from the notations of the corresponding game.
For example, ‘refer2500’ refers to emergent languages from referential games with 2 , 500 candidates for listeners.
• “referXf”: this refers to the referential games with candidate sets whose size is X, but the partition of batches is identical over epochs, thus giving the game a ﬁxed context.
For example, “refer2500f” refers to referential games with 2 , 500 candidates for listeners, but the batches of data are kept identical through the whole training procedure.
• ‘referXf’: this refers to the emergent languages from “referXf”.
For example, ‘refer2500f’ refers to emergent languages from referential games with 2 , 500 candidates for listeners with ﬁxed context.
Regarding the meaning of “ﬁxed-batch” or “ﬁxed context”, we give an example below where the whole input space is t a, b, c, d u and batch size is 2 : – varying batch/context: t a, b u , t c, d u is the partition for the 1 st epoch; then t a, c u , t b, d u is used for the 2 nd epoch; and random partitions are used in the following epochs; – ﬁxed batch/context: whichever epoch it is, t a, b u , t c, d u is always used as the partition of context; • “referX/f”: this equals a “referX” and a “referXf”, both of them have same complexity but different unpredictability.
To be more speciﬁc, contextual unpredictability of “referX” is higher than “referXf”.
• G s : the universal set of source games which is deﬁned as t “refer2” , “refer10” , “refer100” , “refer1000” , “refer2500” , “refer5000” , “refer7500” , “refer10000” , “refer2f” , “refer10f” , “refer100f” , “refer1000f” , “refer2500f” , “refer5000f” , “refer7500f” , u . • G t : the universal set of target games which is deﬁned as t “refer2” , “refer10” , “refer100” , “refer1000” , “refer2500” , “refer5000” , “refer7500” , “refer10000” u .
E. R AW R ESULTS OF C OMPLEXITY E XPERIMENT
In this section
[Table]
we provide the raw data of Figure 2 in the following Table 3.
Not that although “refer1000f” performs the best on “refer10” than all other languages
[Table]
we argue this outlier doesn’t inﬂuence conclusion of our prediction considering the simplicity of “refer10”
[Table]
E. M ORE R ESULTS OF T RADE - OFF E XPERIMENT
In this section, we ﬁrst provide the complete version of Figure 4, i.e. Figure 8.
Since the performance of ‘refer2’, ‘refer10’, and ‘refer100’ are too bad on complex target games, it is very clear that their expressivity are much lower that the rest.
Therefore, to emphasise the difference between the other languages, we limit the range of y -axis to r 0 .
s in Figure 8.
[Figure] We also illustrate the partial order between the expressivity of different languages in Figure 9.
It is very clear that the ‘refer2’, ‘refer10’, and ‘refer100’ performs worse than the others on referential games with more candidates than their sources.
Combining this fact with the fact that ‘refer7500’ and ‘refer10000’ perform worse than ‘refer1000’, ‘refer2500’, and ‘refer5000’ shows that increasing the size of candidate set | C | would not always beneﬁt the expressivity.
As for ‘refer1000’, ‘refer2500’, and ‘refer5000’, there is no statistically signiﬁcant difference between their performance on the simpler referential games, and they intersect on some games, thus their expressivity are either incomparable or approximately the same.
At last, we give the precise values of both Figure 9 and 4 in the following Table 4.
All the results are obtained by multiple runs, thus both mean and standard deviation ( σ ) are given.
To better support our claim, we also provide p -values of testing the statistical signiﬁcance between the generalisation performance of two different source games across all types of target games in the following Table 5.
F V ARYING THE CAPACITY OF CHANNEL AND NETWORK
To verify that our ﬁndings are robust to the capacity of the communication channel and agents, we also run some extra experiments on the following different conﬁgurations of referential games:
larger channel capacity : set the length of messages to 8 and size of token inventory to 20 , thus the size of message space becomes 20 8 ; 2.
larger agent capacity : set the hidden layer size to 512 , thus the capacity of both speaker and listener become larger than the original setting ( 256 ); 3.
larger channel&agent capacity : do the above two changes at the same time.
Since our key ﬁndings are about the generalisation performance between refer1000, refer2500, and refer10000, we just run these extra experiments on a game set G extra “ t ‘refer1000’ , ‘refer2500’ , ‘refer10000’ u with multiple runs.
The results of conﬁguration 1, 2, andare given in Figure 10, 11, and 12 respectively.
It can be observed in all ﬁgures that ‘refer1000’ or [Figure] [Table] [Figure]
[Table] [Figure] [Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
‘refer2500’ always perform better than ‘refer10000’ on all kinds of target games, i
[Table]
e. ‘refer1000’ or ‘refer2500’ both have higher expressivity than ‘refer10000’ .
Therefore, we believe the key observation from Figure 9 holds across conﬁgurations of communication channel and capacity of agents, which means that our conclusions are robust to the conﬁguration of language games.
[Table]
[Figure]
[Figure]
[Figure]