Linking Emergent and Natural Languages via Corpus Transfer

1 INTRODUCTION . The remarkable recent progress in NLP ( Devlin et al. , 2018 ; Radford et al. , 2019 ; Brown et al. , 2020 ) benefits from huge text corpora , on which powerful models learn rich statistical patterns tabula rasa and adapt to downstream tasks . But in stark contrast , human language acquisition ( De Villiers et al. , 1978 ) does not start with passive and complex corpora like Wikipedia . Instead , children learn language through a cumulative series of interactions with other people ( Bruner , 1985 ; Barton , 1994 ) grounded in the physical and social world ( Harnad , 1990 ; Vogt , 2002 ; Alomari et al. , 2017 ) . Incorporating such cognitive insights with recent machine learning advances holds great promises towards more functional ( Wittgenstein , 1958 ; Wagner et al. , 2003 ) and generalizable ( Lake et al. , 2017 ) language agents . The study of emergent communication ( EC ) ( Cangelosi & Parisi , 2002 ; Kirby , 2002 ; Wagner et al. , 2003 ; Lazaridou & Baroni , 2020 ) is one promising direction toward this motivation , where a communication protocol is shaped through multi-agent interactions with perceptual grounding . A typical setup is the image referential game ( Figure 1 ( a ) ) , where a speaker generates a discrete sequence of tokens based on an input image , a listener is challenged to select the input out of distractors based on the message , and both networks are optimized jointly via game success signals . By studying these games , researchers are interested in the emergence of desirable properties resembling natural 1In this work “ emergent language ” means generated messages from an EC game speaker . The term also refers to children ’ s language development in psychology ( Bohlmann & Downer , 2016 ; Zajicek-Farber , 2010 ) . Looking down on fountains A view of skyline A group of students from plant 13 28 39 555 2 … ( d ) Emergent - > Natural Translation 49 28 39 555 2 … ( b ) Language Modeling Captioned Images Emergent Messages ( c ) Image Captioning ✓ ( a ) Emergent Communication Game … Listener 13 28 22 99 122 Pre-train . LM These people are Fine-tune fromLM Pre-train . 13 34 33 4 .. IC Fine-tune A view of skyline IC 13 87 213 3 22 … 49 28 39 2 843 … Speaker Emergent Message Speaker Figure 1 : Our framework . ( a ) We train a referential game on natural images , and use the trained speaker to generate a corpus of emergent messages ( bold box ) to pre-train for ( b ) language modeling and ( c ) image captioning . We also propose to ( d ) translate emergent language ( yellow ) into corresponding natural language ( blue ) with same perceptual grounding to evaluate their closeness . language , such as game success generalization ( Kharitonov & Baroni , 2020 ; Lazaridou & Baroni , 2020 ) and compositionality ( Smith et al. , 2003 ; Andreas & Klein , 2017 ; Kirby et al. , 2015 ; Lazaridou et al. , 2018 ; Li et al. , 2020b ) . However , these properties are mostly defined and analyzed within each individual game framework . For example , generalization is usually measured as game success with novel inputs , but it remains unknown how the emergent coding can be useful beyond the particular game ( e.g . useful for natural language tasks ) . Moreover , compositionality is often approximated by attribute-wise disentanglement in simple environments , which could be drastically different from the structural properties of natural language ( Dankers et al. , 2021 ) . Such gaps pose difficulty in objectively understanding the emergent properties with respect to the full complexities of natural language , and assessing the universality of various claims . In this work , we aim to address these issues by linking emergent languages with natural languages . Concretely , we investigate if modeling an emergent language provides transferable benefits for downstream natural language tasks — language modeling ( Figure 1 ( b ) ) , image captioning ( Figure 1 ( c ) ) — and if downstream performances in turn allow us to understand and analyze properties of emergent language beyond the EC game and toward natural language . The key technique is corpus transfer , i.e . by pre-training a model on a corpus of emergent messages produced by a trained emergent speaker , and fine-tuning the model on downstream tasks with natural language data . This approach integrates inspirations from prior work like Papadimitriou & Jurafsky ( 2020 ) , which proposes the transfer scheme from synthetic to natural corpora , and Li et al . ( 2020b ) , which aims to improve few-shot translation by transferring the trained EC speaker and listener model weights . Through a series of experiments , we find that corpus transfer is helpful when the downstream natural language resource is limited . For example , in a low-resource setup of modeling two million natural language tokens , such a transfer scheme reduces the test perplexity by 24.6 % on average versus training from scratch , across ten different downstream languages . We also establish the non-triviality of such a transfer performance by comparing to other synthetic and natural source corpora , as well as multiple ablation studies on the EC and downstream transfer setups to help understand the transferability of emergent language . Notably , our method of corpus transfer significantly outperforms directly transferring the trained emergent speaker model ( Li et al. , 2020b ) , demonstrating that modeling the emergent language could yield greater usefulness than directly transferring the EC agents . While our transfer experiments demonstrate that emergent languages can be leveraged and understood outside the game and towards real-world tasks , the pre-training and fine-tuning pipeline is computationally expensive as a scalable evaluation scheme for a population of emergent languages . Instead , we are interested in developing a simpler metric to predict the transferability of emergent language , which pertains to properties of natural language and may challenge metrics defined within the game framework ( e.g . game accuracy ) . Therefore , we propose a novel metric for emergent languages based on how easily it can be translated to the corresponding natural language grounded on the same perceptual input ( Figure 1 ( d ) ) . Concretely , we employ the trained speaker to produce emergent messages based on a small set of unseen images with natural language captions , and train a translation model on paired emergent and natural captions . We show that the translation score ( such as ROUGE L ( Lin , 2004 ) ) can better predict ( e.g . Pearson correlation ρ = 0.83 for modeling Hebrew ) the transfer benefits of different emergent languages than two commonly adopted metrics in prior literature – game success with novel inputs ( ρ = 0.74 ) , and topographic similarity ( Brighton & Kirby , 2006 ; Lazaridou et al. , 2018 ) ( ρ = 0.003 ) . These results call for a re-thinking of what properties we should evaluate for an emergent language corpus , and how we should evaluate them . For example , game accuracy may be confounded by the listener performance , and topographic similarity , a metric of attribute disentanglement , can be flawed for measuring the compositional structure of real language ( Kirby , 2001 ; Goldberg , 2015 ; Steinert-Threlkeld , 2020 ) . In summary , our work takes a step toward bridging the gap between emergent and natural languages . We believe that shifting from synthetic setups and in-game evaluations to leveraging downstream natural language tasks and state-of-the-art language models could potentially lead to more general insights and greater practical impact for EC and NLP research . 2 RELATED WORK . Emergent Communication and Evaluation A large body of prior work qualitatively analyzes the evolved message structures when the EC games are set up with synthetic environments ( Mordatch & Abbeel , 2018 ; Lazaridou et al. , 2018 ) or even with real images ( Havrylov & Titov , 2017 ) . Alternative evaluations and examinations include ease of teaching ( Li & Bowling , 2019 ) and symbol usage purity ( Lazaridou et al. , 2016 ) . While these schemes align the emergent language with game success or different aspects of game input , our work evaluates the emergent language outside the game framework , instead via the corpus transfer performance to natural language tasks and the translation performance to natural language . Linking Synthetic and Natural Language Papadimitriou & Jurafsky ( 2020 ) find that recurrent networks pretrained on non-linguistic corpora ( e.g . music , code , regular languages ) facilitate natural language modeling . In an opposite direction , Lu et al . ( 2021 ) observe that language pre-training improves Transformer ’ s performance with several non-language tasks ( logical , vision , protein ) even when most weights are frozen , indicating that learning language structure could yield generally useful computational subroutines . Andreas et al . ( 2017 ) propose to translate a differentiable communication channel in deep multi-agent policies to natural language by collecting human-human communication in the same games . However , their approach focused on interpreting a continuous channel , while we focus on evaluating emergent messages with discrete structures . Use of Emergent Communication beyond games There have been efforts to incorporate communication signals and natural language supervision to better communicate ( Lowe et al. , 2020 ) , avoid language drift ( Lazaridou et al. , 2020 ) , or to improve NLP tasks like vision-language navigation ( Fried et al. , 2018 ) , translation ( Lee et al. , 2018 ) , and image captioning ( Havrylov & Titov , 2017 ) . Different from these setups where the speaker is also grounded on natural language annotations , Li et al . ( 2020b ) propose to separate the EC game , where no natural language is involved , and fine-tuning on a downstream translation task with limited natural language data . Despite being the closest work to ours , such a model transfer approach is fundamentally different from our corpus transfer in several ways . First , while Li et al . ( 2020b ) mainly aim to improve downstream tasks , we envision transfer as a novel means to evaluating and analyzing emergent languages . Also , the emergent language is a representation with properties beyond the parameters of specific EC models ( Section 3.1 ) . Last but not least , corpus transfer has several practical advantages over model transfer ( Section 4.3 ) . 3 BACKGROUND . 3.1 EMERGENT COMMUNICATION ( EC ) GAME . As shown in Figure 1 ( a ) , we consider a typical speaker-listener referential game ( Lazaridou et al. , 2016 ; 2018 ; Li & Bowling , 2019 ) on a set of N image features DI = { I1 , · · · , IN } . At each training step , the speaker takes an input image feature Ii and generates a discrete message Mi ∈ [ V ] T using the Gumbel-Softmax trick ( Jang et al. , 2016 ) , where V is the vocabulary size and T is the sequence length limit . For simplicity denote m = Mi so that mt =Mi , t denotes the t-th token of the message Mi . Then hs0 = Ii , hst = GRUspk ( mt−1 , hst−1 ) ( t > 0 ) , m0 = [ CLS ] , mt = Gumbel-Softmax ( MLPspk ( hst ) ) ( t > 0 ) . ( 1 ) Here hst denotes speaker hidden states . Next , the listener takes the message m , and tries to guess the right image Ii out of a set of K confounding images Ci = { Ij1 , · · · , IjK } ⊂ DI − { Ii } . To do so , it uses another GRU layer to turn the message m into a hidden vector hlT hl0 = 0 , hlt = GRUlsn ( mt , hlt−1 ) ( t > 0 ) . ( 2 ) Based on hlT , the listener assigns a score for each candidate image based on inverse square error ( Lee et al. , 2018 ) , then selects the image by Softmax sampling across the scores . The speaker and listener are jointly optimized by minimizing the cross-entropy loss of image selection : score ( I ) = ||hlT −MLPlsn ( I ) ||−22 , ( 3 ) p ( guess = I ) = softmax ( score ( I ) ) ( I ∈ { Ii } ∪ Ci ) , ( 4 ) LEC = −EIi , CiEMi log p ( guess = Ii ) . ( 5 ) After the game is trained , the speaker can be employed to sample a corpus of emergent language DM = { M1 , · · · , MN } based on input images . Though the emergent language is generated by a particular EC speaker , the representation could be potentially learned and used by new agents , with more general properties beyond the parameters of a particular speaker architecture . Thus , our work focuses on evaluating and analyzing the properties of the emergent language ( e.g . corpus transfer ) rather than the emergent communication models ( e.g . model transfer ) .

Summary: This paper studies whether output from emergent communication systems might be useful as pre-training data for natural language tasks. The authors train an EC agent, generate from it, and use the generated IDs as data for language model pre-training. Compared to pre-training on Spanish wikipedia or a simple bracket language, pre-training on EC-generated data is better when you have 2M examples, but quickly becomes worse as the amount of data increases. In experiments on image captioning, the authors find that pre-training on either EC-generated or natural language captions improve over not pre-training, but that there is not much of a difference between the results when pre-training on EC-generated text and natural language. Finally, the authors propose EC -> natural language translation as a way of evaluating the quality of EC agents.