Expressivity of Emergent Languages is a Trade-off between Contextual Complexity and Unpredictability

1 INTRODUCTION . Language games were first introduced by Wittgenstein ( 1954 ) to explore the meanings of language utterances . Instantiating this concept with the signalling game design from Lewis ( 1969 ) enables linguists to explore the emergence of linguistic structure ( Kirby , 2001 ; Kirby & Hurford , 2002 ) where artificial languages are represented as symbolic systems . The success of deep learning ( DL ) models on complicated cognitive tasks ( Krizhevsky et al. , 2012 ; LeCun et al. , 2015 ; Silver et al. , 2016 ) then inspired researchers to apply DL-based models to language games for investigating agents ’ ability to invent communication protocols without preset rules . In language games implemented in existing works ( e.g . Lazaridou et al. , 2018 ; Graesser et al. , 2019 ; Zhang et al. , 2019 ; Lee et al. , 2018 ; Yuan et al. , 2020 ) , there are usually two types of agents , speakers that emit messages based on their observations ( i.e . target objects ) and listeners that receive messages and take actions accordingly , as illustrated in Figure 1 . Based on the goals for listeners , we can categorise most of the games into the following three types : 1 ) referential games where listeners need to select the target object observed by the speaker among a set of candidate objects ( candidates for short ) ( e.g . Lazaridou et al. , 2017 ; Ren et al. , 2020 ) ; 2 ) reconstruction games where listeners need to reconstruct the speaker ’ s observation ( e.g . Chaabouni et al. , 2020 ; Kharitonov et al. , 2020 ) ; and 3 ) navigation games where listeners need to go to the location specified by the speaker ( e.g . Lowe et al. , 2017 ; Kajić et al. , 2020 ) . We focus on referential games , illustrated in Figure 1a , which have been well investigated in both the linguistic community ( e.g . Winters et al. , 2018 ; Kirby et al. , 2014 ) and emergent communication community ( e.g . Lazaridou et al. , 2018 ; Li & Bowling , 2019 ) . It is reasonable to assume that listeners in referential games can differentiate the target object from the other distractors in the context as long as the speaker ’ s message encodes some unique feature of the target . Therefore , the speaker ’ s messages must convey different amount of information for listeners to complete their task in games with candidates that are similar in different degree . For example , to distinguish two very similar objects in the candidates set , e.g . blue wide-stripped shirt and blue narrow-stripped shirt , it requires more information about the target to be encoded by the speaker ’ s message than to distinguish a shirt from a cup . Experiments with human participants show that emergent languages are sensitive to the requirements of the communicative tasks for which they are used , with languages developing in which only the necessary discriminatory information is encoded Speaker Listener Message ( a ) Illustration of the referential game we investigate in this work . Observations are concatenations of two 5-digits-long one-hot vectors . The objective of the listener in to select among a set of candidate inputs ( the context ) . Speaker 's Encoder ( MLP ) Speaker 's Message Generator ( LSTM ) Listener 's Message Encoder ( LSTM ) Speaker Modules Listener Modules Listener 's Candidate Encoder ( MLP ) Input target distractor Listener 's Action Module ( Softmax ) Message output distribution Candidate set ( context ) ( b ) Illustration of the architecture of agents we implement for referential games . Blue modules are components of the speaker , and green modules belong to the listener . Orange and grey indicate the vectors generated by agents and input samples respectively . Figure 1 : Diagram of referential game and the architecture of our agents . ( Winters et al. , 2018 ) . Following Kirby et al . ( 2015 ) , we refer to such ability to encode information about input space ( e.g . structure , dimensionality , distribution of samples ) as expressivity . In this work , we explore the factors that influence expressivity in the framework of DL-based language games , and we argue that it is important to understand these factors such that agents could develop languages that are useful in general . Our contribution in this work is threefold . First , we propose and verify a hypothesis about the determining factors of expressivity of emergent languages under the DL-based framework . Following Winters et al . ( 2018 ) , we show that context size ( i.e . the number of candidates the listener must select among when interpreting the speaker ’ s message ) influence both the complexity and unpredictability of that context . Complexity refers to the similarity between the items in the context : in order for the listener to select the target , more information needs to be encoded in the speaker ’ s signal if the context contains more similar distractors for a given target . Unpredictability refers to the extent to which the information that needs to be encoded by the speaker is stable across contexts in successive episodes of communication : in games where the information required by the listener differs from trial ( training epoch for DL agents ) to trial , the speaker needs to encode more information on every trial in order to be sure to encode the information the listener needs.1 As we show in Section 3.1 , complexity and unpredictability are affected differently by the context size : as context size increases , complexity increases but unpredictability decreases . Therefore , we propose and verify the following hypothesis about the determining factors of expressivity : The expressivity of emergent languages is a trade-off between the complexity and the unpredictability of context in language games . [ 1 ] Our second contribution is a novel measure of expressivity based on partial ordering of languages in terms of their generalisation performances across tasks . Although expressivity is related to the amount of information encoded in a language , we illustrate that mutual information ( MI ) is not an appropriate measurement for expressivity in Section 3.2 . Considering that one of our objectives is to facilitate a universally useful emergent language , we propose to measure expressivity of a language based on the generalisation performance of agents trained with it across different games . Since it is more challenging to quantitatively measure it , we focus on the partial ordering between languages in this first work towards understanding it . Our final contribution is the discovery of message type collapse , i.e . the number of unique messages is significantly less than the size of input space in relatively easy games , which further leads to highly ambiguous emergent languages . To overcome the technical limitations imposed by GPU memory size on large-scale referential games , we introduce the contrastive loss proposed by Chen et al . ( 2020a ) in referential games . While comparing the behaviour of the contrastive loss and the loss function by previous work ( e.g Havrylov & Titov , 2017 ; Li & Bowling , 2019 ; Ren et al. , 2020 ; Guo et al. , 2020 ) , we find that this contrastive loss could greatly alleviate the collapse of message types , thus lead to more disambiguous emergent languages . 1Context , complexity , and unpredictability are defined and discussed in depth in Section 3 . 2 GAMES , AGENTS AND LEARNING METHODS . 2.1 GAME ENVIRONMENT AND ARCHITECTURE OF AGENTS . Environment Components As shown in Figure 1b , the environment in our referential games have the following components : 1 ) input space X from which the speaker ’ s observations are drawn , which consists of 10 , 000 possible inputs2 where each x P X is a concatenation of 4 one-hot vectors whose length is 10 ; 2 ) message space M consists of 106 6-tokens-long sequences m ( called as a message type ) where the size of the token inventory is 10 ; note that the tokens are initially meaningless , and meaning emerges through the negotiation of a communication protocol between speaker and listener ; 3 ) candidate set C which contains a target input xt and |C| ´ 1 distractors txd1 , xd2 , . . . , xd|C|´1u sampled uniformly at random from X without replacement . Agents The architecture of the agents we implemented is also illustrated in Figure 1b : 4 ) speaker S consists of both a multi layer perceptron ( MLP ) for encoding the 40-digits-long target input xt onto a 256-digits-long embedding hS , and a decoder for generating a message m based on long-short term memory ( LSTM ) network ( Hochreiter & Schmidhuber , 1997 ) . 5 ) listener L is constituted by an LSTM-based sequence encoder which encodes m into a 256-digits-long embedding hLpmq , and an MLP for encoding the inputs which can be denoted by fLcencp¨q . Definition of an emergent language : following e.g . Ren et al . ( 2020 ) , we can then define an emergent language L as a mapping function from the input space X to the message space M , i.e . L : X ÞÑ M , thus it can be represented as tpxi , Lpxiqq| @ xi P X u . Note that L could be a noninjective mapping function , i.e . it is possible for two different inputs to be mapped to an identical message thus the number of message types might be less than the number of unique inputs . 2.2 LEARNING METHODS OF AGENTS . In the previous works ( e.g . Havrylov & Titov , 2017 ) , the action module of L is usually a softmax function that would produce a categorical distribution over the candidates based on the distance between hLpmq and embeddings of each candidate fLcencpxiq obtained through another MLP-based candidate encoder . By comparing the loss function used in Havrylov & Titov ( 2017 ) and the contrastive loss function proposed by Hadsell et al . ( 2006 ) , we find that the aim of both functions is the same , i.e . to make the distance between positive pairs ( xt and m ) closer while keeping the embedding for negative pairs ( xd and m ) as far apart as possible . To effectively validate our hypothesis , we need to make the whole input space X as candidate set C in which case the batch is too large to be stored in GPU memory ( since the space complexity is Op|X |2q ) . Therefore , to overcome this issue , we adapt the contrastive learning method proposed by Chen et al . ( 2020a ) to referential games . Suppose that we have a batch of input samples B “ tx1 , x2 , . . . , xi , . . . , x|B|u and the corresponding messages from the speaker are tm1 , m2 , . . . , mi , . . . , m|B|u which would be further encoded into thLm1 , h L m2 , . . . , h L mi , . . . , h L m|B| u by the listener L. Then the loss function for a sample xi is defined on a softmax function : ` i “ ´ log exp ´ hLmi J ¨ fLcencpxiq ¯ ř|B| j “ 1 exp ´ hLmi J ¨ fLcencpxjq ¯ . ( 1 ) In this way , the number of candidates |C| is exactly the batch size |B| during training . More details about the comparison between referential loss and this contrastive loss can be found in Appendix A . As for updating the parameters , we use the Adam algorithm introduced by Kingma & Ba ( 2014 ) , and the learning rate is set to 10´4 . To allow the gradients being propagated through the discrete channel to overcome the sampling issue of messages , we apply the Gumbel-Softmax trick proposed by Jang et al . ( 2017 ) , and the temperature hyper-parameter τ is set to 1.0 . Our implementation3 of games bases on the framework EGG developed by Kharitonov et al . ( 2019 ) . The experiments across 6 random seeds , 18 source games , 11 target games took 4 , 216 hours in total , on Nvidia Tesla P100 . Definition of “ source/target games ” are given in the following Section 4.2 . 2According to the results from Chaabouni et al . ( 2020 ) , an input space containing 10 , 000 possible inputs is large enough for neural models to achieve ą 99 % generalisation accuracy . 3Codes are provided in the supplementary materials .

Summary: This paper studies the expressivity, complexity and unpredictability of emergent languages in referential games. The authors demonstrate that the expressivity of emergent languages is a trade-off between the complexity and unpredictability of the context that the languages are used in. They also introduce a contrastive loss based training method for referential games that alleviates the collapse of message types often seen when using other standard training methods.