Expressivity of Emergent Languages is a Trade-off between Contextual Complexity and Unpredictability

1 INTRODUCTION . Language games were first introduced by Wittgenstein ( 1954 ) to explore the meanings of language utterances . Instantiating this concept with the signalling game design from Lewis ( 1969 ) enables linguists to explore the emergence of linguistic structure ( Kirby , 2001 ; Kirby & Hurford , 2002 ) where artificial languages are represented as symbolic systems . The success of deep learning ( DL ) models on complicated cognitive tasks ( Krizhevsky et al. , 2012 ; LeCun et al. , 2015 ; Silver et al. , 2016 ) then inspired researchers to apply DL-based models to language games for investigating agents ’ ability to invent communication protocols without preset rules . In language games implemented in existing works ( e.g . Lazaridou et al. , 2018 ; Graesser et al. , 2019 ; Zhang et al. , 2019 ; Lee et al. , 2018 ; Yuan et al. , 2020 ) , there are usually two types of agents , speakers that emit messages based on their observations ( i.e . target objects ) and listeners that receive messages and take actions accordingly , as illustrated in Figure 1 . Based on the goals for listeners , we can categorise most of the games into the following three types : 1 ) referential games where listeners need to select the target object observed by the speaker among a set of candidate objects ( candidates for short ) ( e.g . Lazaridou et al. , 2017 ; Ren et al. , 2020 ) ; 2 ) reconstruction games where listeners need to reconstruct the speaker ’ s observation ( e.g . Chaabouni et al. , 2020 ; Kharitonov et al. , 2020 ) ; and 3 ) navigation games where listeners need to go to the location specified by the speaker ( e.g . Lowe et al. , 2017 ; Kajić et al. , 2020 ) . We focus on referential games , illustrated in Figure 1a , which have been well investigated in both the linguistic community ( e.g . Winters et al. , 2018 ; Kirby et al. , 2014 ) and emergent communication community ( e.g . Lazaridou et al. , 2018 ; Li & Bowling , 2019 ) . It is reasonable to assume that listeners in referential games can differentiate the target object from the other distractors in the context as long as the speaker ’ s message encodes some unique feature of the target . Therefore , the speaker ’ s messages must convey different amount of information for listeners to complete their task in games with candidates that are similar in different degree . For example , to distinguish two very similar objects in the candidates set , e.g . blue wide-stripped shirt and blue narrow-stripped shirt , it requires more information about the target to be encoded by the speaker ’ s message than to distinguish a shirt from a cup . Experiments with human participants show that emergent languages are sensitive to the requirements of the communicative tasks for which they are used , with languages developing in which only the necessary discriminatory information is encoded Speaker Listener Message ( a ) Illustration of the referential game we investigate in this work . Observations are concatenations of two 5-digits-long one-hot vectors . The objective of the listener in to select among a set of candidate inputs ( the context ) . Speaker 's Encoder ( MLP ) Speaker 's Message Generator ( LSTM ) Listener 's Message Encoder ( LSTM ) Speaker Modules Listener Modules Listener 's Candidate Encoder ( MLP ) Input target distractor Listener 's Action Module ( Softmax ) Message output distribution Candidate set ( context ) ( b ) Illustration of the architecture of agents we implement for referential games . Blue modules are components of the speaker , and green modules belong to the listener . Orange and grey indicate the vectors generated by agents and input samples respectively . Figure 1 : Diagram of referential game and the architecture of our agents . ( Winters et al. , 2018 ) . Following Kirby et al . ( 2015 ) , we refer to such ability to encode information about input space ( e.g . structure , dimensionality , distribution of samples ) as expressivity . In this work , we explore the factors that influence expressivity in the framework of DL-based language games , and we argue that it is important to understand these factors such that agents could develop languages that are useful in general . Our contribution in this work is threefold . First , we propose and verify a hypothesis about the determining factors of expressivity of emergent languages under the DL-based framework . Following Winters et al . ( 2018 ) , we show that context size ( i.e . the number of candidates the listener must select among when interpreting the speaker ’ s message ) influence both the complexity and unpredictability of that context . Complexity refers to the similarity between the items in the context : in order for the listener to select the target , more information needs to be encoded in the speaker ’ s signal if the context contains more similar distractors for a given target . Unpredictability refers to the extent to which the information that needs to be encoded by the speaker is stable across contexts in successive episodes of communication : in games where the information required by the listener differs from trial ( training epoch for DL agents ) to trial , the speaker needs to encode more information on every trial in order to be sure to encode the information the listener needs.1 As we show in Section 3.1 , complexity and unpredictability are affected differently by the context size : as context size increases , complexity increases but unpredictability decreases . Therefore , we propose and verify the following hypothesis about the determining factors of expressivity : The expressivity of emergent languages is a trade-off between the complexity and the unpredictability of context in language games . [ 1 ] Our second contribution is a novel measure of expressivity based on partial ordering of languages in terms of their generalisation performances across tasks . Although expressivity is related to the amount of information encoded in a language , we illustrate that mutual information ( MI ) is not an appropriate measurement for expressivity in Section 3.2 . Considering that one of our objectives is to facilitate a universally useful emergent language , we propose to measure expressivity of a language based on the generalisation performance of agents trained with it across different games . Since it is more challenging to quantitatively measure it , we focus on the partial ordering between languages in this first work towards understanding it . Our final contribution is the discovery of message type collapse , i.e . the number of unique messages is significantly less than the size of input space in relatively easy games , which further leads to highly ambiguous emergent languages . To overcome the technical limitations imposed by GPU memory size on large-scale referential games , we introduce the contrastive loss proposed by Chen et al . ( 2020a ) in referential games . While comparing the behaviour of the contrastive loss and the loss function by previous work ( e.g Havrylov & Titov , 2017 ; Li & Bowling , 2019 ; Ren et al. , 2020 ; Guo et al. , 2020 ) , we find that this contrastive loss could greatly alleviate the collapse of message types , thus lead to more disambiguous emergent languages . 1Context , complexity , and unpredictability are defined and discussed in depth in Section 3 . 2 GAMES , AGENTS AND LEARNING METHODS . 2.1 GAME ENVIRONMENT AND ARCHITECTURE OF AGENTS . Environment Components As shown in Figure 1b , the environment in our referential games have the following components : 1 ) input space X from which the speaker ’ s observations are drawn , which consists of 10 , 000 possible inputs2 where each x P X is a concatenation of 4 one-hot vectors whose length is 10 ; 2 ) message space M consists of 106 6-tokens-long sequences m ( called as a message type ) where the size of the token inventory is 10 ; note that the tokens are initially meaningless , and meaning emerges through the negotiation of a communication protocol between speaker and listener ; 3 ) candidate set C which contains a target input xt and |C| ´ 1 distractors txd1 , xd2 , . . . , xd|C|´1u sampled uniformly at random from X without replacement . Agents The architecture of the agents we implemented is also illustrated in Figure 1b : 4 ) speaker S consists of both a multi layer perceptron ( MLP ) for encoding the 40-digits-long target input xt onto a 256-digits-long embedding hS , and a decoder for generating a message m based on long-short term memory ( LSTM ) network ( Hochreiter & Schmidhuber , 1997 ) . 5 ) listener L is constituted by an LSTM-based sequence encoder which encodes m into a 256-digits-long embedding hLpmq , and an MLP for encoding the inputs which can be denoted by fLcencp¨q . Definition of an emergent language : following e.g . Ren et al . ( 2020 ) , we can then define an emergent language L as a mapping function from the input space X to the message space M , i.e . L : X ÞÑ M , thus it can be represented as tpxi , Lpxiqq| @ xi P X u . Note that L could be a noninjective mapping function , i.e . it is possible for two different inputs to be mapped to an identical message thus the number of message types might be less than the number of unique inputs . 2.2 LEARNING METHODS OF AGENTS . In the previous works ( e.g . Havrylov & Titov , 2017 ) , the action module of L is usually a softmax function that would produce a categorical distribution over the candidates based on the distance between hLpmq and embeddings of each candidate fLcencpxiq obtained through another MLP-based candidate encoder . By comparing the loss function used in Havrylov & Titov ( 2017 ) and the contrastive loss function proposed by Hadsell et al . ( 2006 ) , we find that the aim of both functions is the same , i.e . to make the distance between positive pairs ( xt and m ) closer while keeping the embedding for negative pairs ( xd and m ) as far apart as possible . To effectively validate our hypothesis , we need to make the whole input space X as candidate set C in which case the batch is too large to be stored in GPU memory ( since the space complexity is Op|X |2q ) . Therefore , to overcome this issue , we adapt the contrastive learning method proposed by Chen et al . ( 2020a ) to referential games . Suppose that we have a batch of input samples B “ tx1 , x2 , . . . , xi , . . . , x|B|u and the corresponding messages from the speaker are tm1 , m2 , . . . , mi , . . . , m|B|u which would be further encoded into thLm1 , h L m2 , . . . , h L mi , . . . , h L m|B| u by the listener L. Then the loss function for a sample xi is defined on a softmax function : ` i “ ´ log exp ´ hLmi J ¨ fLcencpxiq ¯ ř|B| j “ 1 exp ´ hLmi J ¨ fLcencpxjq ¯ . ( 1 ) In this way , the number of candidates |C| is exactly the batch size |B| during training . More details about the comparison between referential loss and this contrastive loss can be found in Appendix A . As for updating the parameters , we use the Adam algorithm introduced by Kingma & Ba ( 2014 ) , and the learning rate is set to 10´4 . To allow the gradients being propagated through the discrete channel to overcome the sampling issue of messages , we apply the Gumbel-Softmax trick proposed by Jang et al . ( 2017 ) , and the temperature hyper-parameter τ is set to 1.0 . Our implementation3 of games bases on the framework EGG developed by Kharitonov et al . ( 2019 ) . The experiments across 6 random seeds , 18 source games , 11 target games took 4 , 216 hours in total , on Nvidia Tesla P100 . Definition of “ source/target games ” are given in the following Section 4.2 . 2According to the results from Chaabouni et al . ( 2020 ) , an input space containing 10 , 000 possible inputs is large enough for neural models to achieve ą 99 % generalisation accuracy . 3Codes are provided in the supplementary materials .

Summary: The authors investigate what aspects of scenario design affect the resultant communication protocol in emergent communication. Specifically, the authors investigate whether scenario complexity (in terms of having many similar distractors) and scenario unpredictability (in terms of future batches not containing items to previous batches) affect the expressivity (measured in terms of learnability) of the resulting protocol.  Further, the authors show that defining a softmax loss over the entire batch of possible references (dubbed the 'contrastive' loss) outperforms the 'referential' loss used in previous works.