Linking Emergent and Natural Languages via Corpus Transfer

1 INTRODUCTION . The remarkable recent progress in NLP ( Devlin et al. , 2018 ; Radford et al. , 2019 ; Brown et al. , 2020 ) benefits from huge text corpora , on which powerful models learn rich statistical patterns tabula rasa and adapt to downstream tasks . But in stark contrast , human language acquisition ( De Villiers et al. , 1978 ) does not start with passive and complex corpora like Wikipedia . Instead , children learn language through a cumulative series of interactions with other people ( Bruner , 1985 ; Barton , 1994 ) grounded in the physical and social world ( Harnad , 1990 ; Vogt , 2002 ; Alomari et al. , 2017 ) . Incorporating such cognitive insights with recent machine learning advances holds great promises towards more functional ( Wittgenstein , 1958 ; Wagner et al. , 2003 ) and generalizable ( Lake et al. , 2017 ) language agents . The study of emergent communication ( EC ) ( Cangelosi & Parisi , 2002 ; Kirby , 2002 ; Wagner et al. , 2003 ; Lazaridou & Baroni , 2020 ) is one promising direction toward this motivation , where a communication protocol is shaped through multi-agent interactions with perceptual grounding . A typical setup is the image referential game ( Figure 1 ( a ) ) , where a speaker generates a discrete sequence of tokens based on an input image , a listener is challenged to select the input out of distractors based on the message , and both networks are optimized jointly via game success signals . By studying these games , researchers are interested in the emergence of desirable properties resembling natural 1In this work “ emergent language ” means generated messages from an EC game speaker . The term also refers to children ’ s language development in psychology ( Bohlmann & Downer , 2016 ; Zajicek-Farber , 2010 ) . Looking down on fountains A view of skyline A group of students from plant 13 28 39 555 2 … ( d ) Emergent - > Natural Translation 49 28 39 555 2 … ( b ) Language Modeling Captioned Images Emergent Messages ( c ) Image Captioning ✓ ( a ) Emergent Communication Game … Listener 13 28 22 99 122 Pre-train . LM These people are Fine-tune fromLM Pre-train . 13 34 33 4 .. IC Fine-tune A view of skyline IC 13 87 213 3 22 … 49 28 39 2 843 … Speaker Emergent Message Speaker Figure 1 : Our framework . ( a ) We train a referential game on natural images , and use the trained speaker to generate a corpus of emergent messages ( bold box ) to pre-train for ( b ) language modeling and ( c ) image captioning . We also propose to ( d ) translate emergent language ( yellow ) into corresponding natural language ( blue ) with same perceptual grounding to evaluate their closeness . language , such as game success generalization ( Kharitonov & Baroni , 2020 ; Lazaridou & Baroni , 2020 ) and compositionality ( Smith et al. , 2003 ; Andreas & Klein , 2017 ; Kirby et al. , 2015 ; Lazaridou et al. , 2018 ; Li et al. , 2020b ) . However , these properties are mostly defined and analyzed within each individual game framework . For example , generalization is usually measured as game success with novel inputs , but it remains unknown how the emergent coding can be useful beyond the particular game ( e.g . useful for natural language tasks ) . Moreover , compositionality is often approximated by attribute-wise disentanglement in simple environments , which could be drastically different from the structural properties of natural language ( Dankers et al. , 2021 ) . Such gaps pose difficulty in objectively understanding the emergent properties with respect to the full complexities of natural language , and assessing the universality of various claims . In this work , we aim to address these issues by linking emergent languages with natural languages . Concretely , we investigate if modeling an emergent language provides transferable benefits for downstream natural language tasks — language modeling ( Figure 1 ( b ) ) , image captioning ( Figure 1 ( c ) ) — and if downstream performances in turn allow us to understand and analyze properties of emergent language beyond the EC game and toward natural language . The key technique is corpus transfer , i.e . by pre-training a model on a corpus of emergent messages produced by a trained emergent speaker , and fine-tuning the model on downstream tasks with natural language data . This approach integrates inspirations from prior work like Papadimitriou & Jurafsky ( 2020 ) , which proposes the transfer scheme from synthetic to natural corpora , and Li et al . ( 2020b ) , which aims to improve few-shot translation by transferring the trained EC speaker and listener model weights . Through a series of experiments , we find that corpus transfer is helpful when the downstream natural language resource is limited . For example , in a low-resource setup of modeling two million natural language tokens , such a transfer scheme reduces the test perplexity by 24.6 % on average versus training from scratch , across ten different downstream languages . We also establish the non-triviality of such a transfer performance by comparing to other synthetic and natural source corpora , as well as multiple ablation studies on the EC and downstream transfer setups to help understand the transferability of emergent language . Notably , our method of corpus transfer significantly outperforms directly transferring the trained emergent speaker model ( Li et al. , 2020b ) , demonstrating that modeling the emergent language could yield greater usefulness than directly transferring the EC agents . While our transfer experiments demonstrate that emergent languages can be leveraged and understood outside the game and towards real-world tasks , the pre-training and fine-tuning pipeline is computationally expensive as a scalable evaluation scheme for a population of emergent languages . Instead , we are interested in developing a simpler metric to predict the transferability of emergent language , which pertains to properties of natural language and may challenge metrics defined within the game framework ( e.g . game accuracy ) . Therefore , we propose a novel metric for emergent languages based on how easily it can be translated to the corresponding natural language grounded on the same perceptual input ( Figure 1 ( d ) ) . Concretely , we employ the trained speaker to produce emergent messages based on a small set of unseen images with natural language captions , and train a translation model on paired emergent and natural captions . We show that the translation score ( such as ROUGE L ( Lin , 2004 ) ) can better predict ( e.g . Pearson correlation ρ = 0.83 for modeling Hebrew ) the transfer benefits of different emergent languages than two commonly adopted metrics in prior literature – game success with novel inputs ( ρ = 0.74 ) , and topographic similarity ( Brighton & Kirby , 2006 ; Lazaridou et al. , 2018 ) ( ρ = 0.003 ) . These results call for a re-thinking of what properties we should evaluate for an emergent language corpus , and how we should evaluate them . For example , game accuracy may be confounded by the listener performance , and topographic similarity , a metric of attribute disentanglement , can be flawed for measuring the compositional structure of real language ( Kirby , 2001 ; Goldberg , 2015 ; Steinert-Threlkeld , 2020 ) . In summary , our work takes a step toward bridging the gap between emergent and natural languages . We believe that shifting from synthetic setups and in-game evaluations to leveraging downstream natural language tasks and state-of-the-art language models could potentially lead to more general insights and greater practical impact for EC and NLP research . 2 RELATED WORK . Emergent Communication and Evaluation A large body of prior work qualitatively analyzes the evolved message structures when the EC games are set up with synthetic environments ( Mordatch & Abbeel , 2018 ; Lazaridou et al. , 2018 ) or even with real images ( Havrylov & Titov , 2017 ) . Alternative evaluations and examinations include ease of teaching ( Li & Bowling , 2019 ) and symbol usage purity ( Lazaridou et al. , 2016 ) . While these schemes align the emergent language with game success or different aspects of game input , our work evaluates the emergent language outside the game framework , instead via the corpus transfer performance to natural language tasks and the translation performance to natural language . Linking Synthetic and Natural Language Papadimitriou & Jurafsky ( 2020 ) find that recurrent networks pretrained on non-linguistic corpora ( e.g . music , code , regular languages ) facilitate natural language modeling . In an opposite direction , Lu et al . ( 2021 ) observe that language pre-training improves Transformer ’ s performance with several non-language tasks ( logical , vision , protein ) even when most weights are frozen , indicating that learning language structure could yield generally useful computational subroutines . Andreas et al . ( 2017 ) propose to translate a differentiable communication channel in deep multi-agent policies to natural language by collecting human-human communication in the same games . However , their approach focused on interpreting a continuous channel , while we focus on evaluating emergent messages with discrete structures . Use of Emergent Communication beyond games There have been efforts to incorporate communication signals and natural language supervision to better communicate ( Lowe et al. , 2020 ) , avoid language drift ( Lazaridou et al. , 2020 ) , or to improve NLP tasks like vision-language navigation ( Fried et al. , 2018 ) , translation ( Lee et al. , 2018 ) , and image captioning ( Havrylov & Titov , 2017 ) . Different from these setups where the speaker is also grounded on natural language annotations , Li et al . ( 2020b ) propose to separate the EC game , where no natural language is involved , and fine-tuning on a downstream translation task with limited natural language data . Despite being the closest work to ours , such a model transfer approach is fundamentally different from our corpus transfer in several ways . First , while Li et al . ( 2020b ) mainly aim to improve downstream tasks , we envision transfer as a novel means to evaluating and analyzing emergent languages . Also , the emergent language is a representation with properties beyond the parameters of specific EC models ( Section 3.1 ) . Last but not least , corpus transfer has several practical advantages over model transfer ( Section 4.3 ) . 3 BACKGROUND . 3.1 EMERGENT COMMUNICATION ( EC ) GAME . As shown in Figure 1 ( a ) , we consider a typical speaker-listener referential game ( Lazaridou et al. , 2016 ; 2018 ; Li & Bowling , 2019 ) on a set of N image features DI = { I1 , · · · , IN } . At each training step , the speaker takes an input image feature Ii and generates a discrete message Mi ∈ [ V ] T using the Gumbel-Softmax trick ( Jang et al. , 2016 ) , where V is the vocabulary size and T is the sequence length limit . For simplicity denote m = Mi so that mt =Mi , t denotes the t-th token of the message Mi . Then hs0 = Ii , hst = GRUspk ( mt−1 , hst−1 ) ( t > 0 ) , m0 = [ CLS ] , mt = Gumbel-Softmax ( MLPspk ( hst ) ) ( t > 0 ) . ( 1 ) Here hst denotes speaker hidden states . Next , the listener takes the message m , and tries to guess the right image Ii out of a set of K confounding images Ci = { Ij1 , · · · , IjK } ⊂ DI − { Ii } . To do so , it uses another GRU layer to turn the message m into a hidden vector hlT hl0 = 0 , hlt = GRUlsn ( mt , hlt−1 ) ( t > 0 ) . ( 2 ) Based on hlT , the listener assigns a score for each candidate image based on inverse square error ( Lee et al. , 2018 ) , then selects the image by Softmax sampling across the scores . The speaker and listener are jointly optimized by minimizing the cross-entropy loss of image selection : score ( I ) = ||hlT −MLPlsn ( I ) ||−22 , ( 3 ) p ( guess = I ) = softmax ( score ( I ) ) ( I ∈ { Ii } ∪ Ci ) , ( 4 ) LEC = −EIi , CiEMi log p ( guess = Ii ) . ( 5 ) After the game is trained , the speaker can be employed to sample a corpus of emergent language DM = { M1 , · · · , MN } based on input images . Though the emergent language is generated by a particular EC speaker , the representation could be potentially learned and used by new agents , with more general properties beyond the parameters of a particular speaker architecture . Thus , our work focuses on evaluating and analyzing the properties of the emergent language ( e.g . corpus transfer ) rather than the emergent communication models ( e.g . model transfer ) .

Summary: This paper proposes a new way of using techniques from the emergent communication literature, where agents are trained to develop languages for communication on some shared task, to improve more typical supervised learning tasks in NLP, namely language modeling and image captioning. This is a question that is only just beginning to be addressed in some work (e.g. Lowe et al., 2021; Lazaridou et al., 2020), so this area is ripe for exploration. This paper proposes an interesting and novel method of using EC to improve supervised learning: generate emergent language from an image-based referential game with simple speaker/listener models. Then, using a much larger transformer, pretrain on this corpus before doing fine-tuning on the supervised task of interest, a la Papadimitriou and Jurfasky, 2020.  Both language modeling and image captioning tasks are explored, and results convincingly show that emergent communication corpora are surprisingly effective. Some ablation studies help elucidate where the benefits arise, although there are some questions here (see Weaknesses).  Finally the authors propose a new metric for evaluating the naturalness and usefulness of a corpus of emergent language: if you have natural language messages generated for some task, as well as true human messages, the authors literally just build a model for translating emergent languages into the corresponding human messages. The ROUGE score obtained by such a model is thus a "transferability" metric. This metric is not only useful for identifying whether an EC corpus will be useful for transfer for some task (more similar = more useful), but is possibly also a measure of humanness/compositionaliy of a language in general, and so this contribution will be useful even for those in EC who are not interested in supervised learning.  Overall, I very much like the philosophy of this paper. I think it helps fill an important gap in the literature, which is namely what to do with the recent deluge of emergent communication studies. The big elephant in the room with these studies is whether the languages are actually useful for some reason besides being studies of linguistic evolution with limited generality. This paper proposes a simple and interesting way of using the emergent languages for something more productive and more of interest to people actually working on real NLP applications. I do however see some weaknesses and lack of comparison which I would love to see answered and/or addressed in the final version.