Linking Emergent and Natural Languages via Corpus Transfer
Authors: Shunyu Yao
Mo Yu
Yang Zhang
Karthik R Narasimhan
Joshua B. Tenenbaum
Chuang Gan
Venue: ICLR 2022 Spotlight (2022)
Keywords: Emergent Language
Emergent Communication
Transfer Learning

Abstract: The study of language emergence aims to understand how human languages are shaped by perceptual grounding and communicative intent. Computational approaches to emergent communication (EC) predominantly consider referential games in limited domains and analyze the learned protocol within the game framework. As a result, it remains unclear how the emergent languages from these settings connect to natural languages or provide benefits in real-world language processing tasks, where statistical models trained on large text corpora dominate. In this work, we propose a novel way to establish such a link by corpus transfer, i.e. pretraining on a corpus of emergent language for downstream natural language tasks, which is in contrast to prior work that directly transfers speaker and listener parameters. Our approach showcases non-trivial transfer benefits for two different tasks – language modeling and image captioning. For example, in a low-resource setup (modeling 2 million natural language tokens), pre-training on an emergent language corpus with just 2 million tokens reduces model perplexity by 24.6% on average across ten natural languages. We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images. We find that our translation-based metric highly correlates with the downstream performance on modeling natural languages (for instance $\rho = 0.83$ on Hebrew), while topographic similarity, a popular metric in previous works, shows surprisingly low correlation ($\rho = 0.003$), hinting that simple properties like attribute disentanglement from synthetic domains might not capture the full complexities of natural language. Our findings also indicate potential benefits of moving language emergence forward with natural language resources and models.

Published as a conference paper at ICLR 2022
LINKING EMERGENT AND NATURAL LANGUAGES VIA CORPUS TRANSFER
Shunyu Yao Princeton University
Mo Yu Wechat Al
Yang Zhang MIT-IBM Watson Al Lab
Karthik Narasimhan Princeton University
Joshua B. Tenenbaum MIT
Chuang Gan MIT-IBM Watson Al Lab
ABSTRACT
The study of language emergence aims to understand how human languages are shaped by perceptual grounding and communicative intent. Computational ap- proaches to emergent communication (EC) predominantly consider referential games in limited domains and analyze the learned protocol within the game frame- work. As result, it remains unclear how the emergent languages" from these settings connect to natural languages or provide benefits in real-world language processing tasks, where statistical models trained large text corpora dominate. this work, we propose a novel way to establish such a link by corpus transfer, i.e. pretraining on a corpus of emergent language for downstream natural language tasks, which is in contrast to prior work that directly transfers speaker and listener parameters. Our approach showcases non-trivial transfer benefits for two different tasks languaaee modeling and image captioning. For example, in low-resource setup (modeling 2 million natural language tokens), pre-training on an emergent language corpus with just 2 million tokens reduces model perplexity by 24.6% on average across ten natural languages. We also introduce novel metric to predict the transferability of an emergent language by translating emergent mes- sages to natural language captions grounded on the same images. find that our translation basdd metric highly correlates with the downstream performance on modeling natural languages (for instance = 0.83 on Hebrew), while topographic similarity, popular metric in previous work, shows surprisingly low correlation (p = 0.003). hinting that simple properties like attribute disentanglement from synthetic domains might not capture the full complexities of natural language. Our findings also indicate potential benefits of moving language emergence forward with natural language resources and model
INTRODUCTION
The recent remarkable progress in NLP (Devlin al., 2018 Radford al. 2019 Brown al.
2020) benefits from huge text corpora, on which powerful models learn rich statistical patterns tabula
rasa and adapt downstream tasks. But in stark contrast, human language acquisition (De Villiers
ct al. 1978) does not start with passive and complex corpora like Wikipedia. Instead, children
learn language through cumulative series of interactions with other people (Bruner 1985 Barton
1994) grounded in the physical and social world (Harnad) 1990; Vogt 2002;; Alomari et al. 2017).
Incorporating such cognitive insights with recent machine learning advances holds great promises
towards more functional (Wittgenstenn, 1958 Wagner et al. 2003) and generalizable (Lake et al.
2017] language agents.
The study of emergent communication (EC) (Cangelosi & Parisi 2002 Kirby 2002 Wagner et al.) 2003 Lazaridou & Baroni 2020) is one promising direction toward this motivation, where a commu-
IIn this work "emergent language" means generated messages from EC game speaker. The term also refers to children's language development in psychology (Hohlmann & Downer 2016 Zajicek-Farber, 2010).
"Code at http://ihuuu.ooyyyyyyyyyee
and correspondence to /hunyuy @ princeton.edu Work partly
done during the first author's remote internship MIT-IBM Watson Al Lab.

---

Published as a conference paper at ICLR 2022
(a) Emergent Communication Game
(b) Language Modeling
(d) Emergent -> Natural
Pre-train
Translation
22
Speaker
Speaker
Fine-tune
Energent Message
Looking down on Fourtaino
These people are
LK
fxom
49 28 39
585 2
(c) Image Captioning
13 67
21.3
Pre-train
22
skyline
Listener
of students
39 2
Pine-tune
84.3
plant
view of skyline
Enargent Meeeagge
Capt
Inagee
Our framework. (a) We train a referential game on natural images, and use the trained speaker to generate a corpus of emergent messages (bold box) to pre-train for (b) language modeling and (c) image captioning, We also propose to (d) translate emergent language (yellow) into corresponding natural language (blue) with same perceptual grounding to evaluate their closeness.
Figure I:
nication protocol is shaped through multi-agent interactions with perceptual grounding. A typical setup is the image referential game (Figure Ta)), a)), where speaker generates discrete sequence of tokens based on an input image, a listener is challenged to select the input out of distractors based on the message, and both networks are optimized jointly via game success signals. By studying these games, researchers are interested in the emergence of desirable properties resembling natural language, such as game success generalization (Kharitonov Baroni [2200] Lazaridou & Baroni 2020) and compositionality (Smith et al. 2003 Andreas & Klein| [017] Kirby et | ctt al. 2018 Li et al. (020b). However, these properties are mostly defined and analyzed within cach individual game framework. For example, generalization is usually measured game success with novel inputs, but it remains unknown how the emergent coding can be useful beyond the particular game (e.g. useful for natural language tasks). Moreover, compositionality is often approximated by attribute-wise disentanglement in simple environments, which could be drastically different from the structural properties of natural language (Dankers et al. 202T). Such gaps pose difficulty in objectively understanding the emergent properties with respect to the full complexities of natural language, and assessing the universality of various claims.
In this work, we aim to address these issues by linking emergent languages with natural languages. Concretely, we investigate modeling an emergent language provides transferable benefits for down- stream natural language tasks - language modeling and if downstream performances in turn allow us to understand and analyze properties of emergent language beyond the EC game and toward natural language. The key technique is corpus transfer, i.e by pre-training a model on a corpus of emergent messages produced by a trained emergent speaker, and fine-tuning the model on downstream tasks with natural language data. This approach integrates inspirations from prior work like Papadimitriou & Jurafsky| (2020), which proposes the transfer scheme from synthetic natural corpora, and Li al. (2020b), which aims to improve few-shot translation by transferring the trained EC speaker and listener model weights. Through a series of experiments, we find that corpus transfer is helpful when the downstream natural language resource is limited. For example, in a low-resource setup of modeling two million natural language tokens, such a transfer scheme reduces the test perplexity by 24.6% on average versus training from scratch, across ten different downstream languages. We also establish the non-triviality of such a transfer performance by comparing to other synthetic and natural source corpora, as well multiple ablation studies on the EC and downstream transfer setups to help understand the transferability of emergent language. Notably, our method of corpus transfer significantly outperforms directly transferring the trained emergent speaker model (Li et al. 2020b), demonstrating that modeling the emergent language could yield greater usefulness than directly transferring the EC agents.
[ image captioning (Figure
While our transfer experiments demonstrate that emergent languages can be leveraged and under- stood outside the game and towards real-world tasks, the pre-training and fine-tuning pipeline is computationally expensive a scalable evaluation scheme for population of emergent languages. Instead, we are interested in developing a simpler metric to predict the transfcrability of emergent language, which pertains to properties of natural language and may challenge metrics defined within
2

---

Published as a conference paper at ICLR 2022
the game framework (e.g. game accuracy). Therefore, propose novel metric for emergent languages based on how easily it can be translated to the corresponding natural language grounded on the same perceptual input (Figure [))). Concretely, employ the trained speaker to produce emergent messages based on a small set of unseen images with natural language captions, and train translation model paired emergent and natural captions. We show that the translation score (such as ROUGEL (Lin 2004) can better predict (e.g. Pearson correlation P = 0.83 for modeling Hebrew) the transfer benefits of different emergent languages than two commonly adopted metrics in prior literature- game success with novel inputs (p = 0. 0.74,, and topographic similarity (Brighton Kirby] [006] Lazaridou et al., 2018) (p 0.003). These results call for a re-thinking of what properties we should evaluate for an emergent language corpus, and how we should evaluate them. For example, game accuracy may be confounded by the listener performance, and topographic similarity, metric of attribute disentanglement, can be flawed for measuring the compositional structure of real language (Kirby) 2001 Goldberg 2015
Threekee- 2020).
In summary, work takes step toward bridging the gap between emergent and natural languages. We believe that shifting from synthetic setups and in-game evaluations to leveraging downstream natural language tasks and state-of-the-- of-the--rt language models could potentially lead to more general insights and greater practical impact for EC and NLP research.
2 RELATED WORK
Emergent Communication and Evaluation A large body of prior work qualitatively analyzes the evolved message structures when the EC games are set up with synthetic environments (Mordatch & Abbeel 2018: Lazaridou et al. 2018) or even with real images (Havrylov & Titov] (017). Alternative evaluations and examinations include ease of teaching (Li & Bowling, 2019) and symbol usage purity (Lazaridou et al., 2016). While these schemes align the emergent language with game success or different aspects of game input, our work evaluates the emergent language outside the game framework, instead via the corpus transfer performance to natural language tasks and the translation performance to natural language.
Linking Synthetic and Natural Language Papadimitriou & Jurafsky 2020) find that recurrent networks pretrained on non-linguistic corpora (e.g. music, code, regular languages) facilitate natural language modeling. In an opposite direction, Lu et al. (202T) observe that language pre-training improves Transformer's 's performance with several non langaae tasks (logical, vision, protein) even when most weights are frozen, indicating that learning language structure could yield generally useful computational subroutines. (Andreas et al. (2017) propose to translate differentiable communication channel in deep multi-agent policies to natural language by collecting numan-human communication in the same games. However, their approach focused on interpreting a continuous channel, while we focus on evaluating emergent messages with discrete structures.
Use of Emergent Communication beyond games There have been efforts to incorporate commu- nication signals and natural language supervision better communicate (Lowe al. 2020), avoid language drift (Lazaridou et al. (020), or to improve NLP tasks like languaae navigation (Fried aall,221.: 2018), translation (Lee et 2018), and image captioning (Havrylov Titov 2017). Different from these setups where the speaker is also grounded on natural language annotations, Li et al. (2020b) propose to separate the EC game, where no natural language involved, and fine-tuning on downstream translation task with limited natural language data. Despite being the closest work to ours, such a model transfer approach is fundamentally different from our corpus transfer in several ways. First, while Li et al. (2020b) mainly aim to improve downstream tasks, we envision transfer as a novel means to evaluating and analyzing emergent languages. Also, the emergent language is a representation with properties beyond the parameters of specific EC models (Section 3.1 Last but not least, corpus transfer has several practical advantages over model transfer (Section
3 BACKGROUND
3.1 EMERGENT COMMUNICATION (EC) GAME
As shown in Figure T a). we consider a typical speaker- listener referential game (Lazaridou et al.)
2016 2018
Li & Bowling 2019) set of N image features D, (I,,... .w). At each training
3

---

Published as a conference paper at ICLR 2022
step, the speaker takes input image feature I, and generates discrete message M, [V] using the Gumbel-Softman trick (Jang et (016)) where V is the vocabulary size and T is the sequence length limit. For simplicity denote m M, so that m, Mi, denotes the t-th token of the message M. Then
h80- = Ti hs: GRU spk (m (1 0),
m/ [CLS], m, Gumbel-Softmax (MLP spil (hs:)) (L> 0).
Here hs, denotes speaker hidden states. Next, the listener takes the message m. and tries to guess the right image I, out of a set of K confounding images C, = [
j C Dr - {I.]. To do so,
uses another GRU layer to turn the message m into a hidden vector hly
hlp = 0, hl, = GRU (m², hl,-1) (t> 0).
Based on hly, the listener assigns score for each candidate image based on inverse square error (Lee)
al. 2018), then selects the image by Softmax sampling across the scores. The speaker and listener are jointly optimized by minimizing the cross-entropy loss of image selection:
sCore(I) = ||hlr ML (1)l
(3)
p(guess I] = softmax(score(I)) (I (L))UC))
LEC = ,,,,c, log p(guess = L.).
(5)
After the game is trained, the speaker can be employed to sample corpus of emergent language
DM [M..... ,MN] based on input images. Though the emergent language is generated by particular EC speaker, the representation could be potentially learned and used by new agents, with more general properties beyond the parameters of a particular speaker architecture. Thus, our work focuses on evaluating and analyzing the properties of the emergent language (e.g. corpus transfer) rather than the emergent communication models (e.g. model transfer).
3.2 EVALUATING EMERGENT LANGUAGE
A central question in emergent communication is how to evaluate the quality of the emergent corpus DM. Two commonly used metrics in prior work are game accuracy and topographic similarity.
Game Accuracy with Novel Objects The most straightforward metric is the accuracy of playing the referential game C.M. []guess=I,] Prior work (Cogswell et al. 2019 Li & Bowling 2019) usually considers how well the speaker and listener generalize to novel ipputs, and funds that such generalization ability might not correlate with other properties such as compositionality (Chaabouni let al.) 2020 Kharitonov & Baroni 2020). Note that this metric involves both the speaker and listener, while the emergent language corpus Da is generated solely by the speaker.
Topographic Similarity The topographic similarity (Brighton & Kirby 2006 Lazaridou al. 2018) measures how messages align with the meaning representations. More concretely. let CSS.yy = be the negative cosine similarity of image features I, and I,. and ED.g be the Levenshtein distance (Levenshtein et al. (966)) between messages M, and My. The metric is defined as the Spearman rank correlation of the two distance metrics:
I.
TED,CS = PR(ED) E44, [(R(ED,) - (BCCs) -MR(C)))))))))))))
(6)
O OR(ED;
Here R(-) denotes the rank of the element in the list. This metric intuitively measures disentangle- ment: different attributes of the input should be expressed in different token positions, and each token only describes one attribute (e.g. "big red cube" and "big blue cube"). Such property has been claimed (Lazaridou et al., 2018 Li & Bowling, 2019) as connected to compositionallly, key structural property of natural language. However, this metric is too rigid in definition of compositionality, ignoring aspects like argument structure, context or morphology which play key role in determining the combination of word semantics (Goldberg 2015).
We note that experiments around both these metrics are within the game framework - accuracy is based on game success, while topographic similarity measures how emergent messages align with game input representations. This leaves two questions unsolved: (i) can emergent languages be used outside the game? (ii) if so, would these metrics predict the usefulness emergent languages for downstream tasks? We tackle these two questions in Sections 4 and 5 respectively.
4

---

Published as a conference paper at ICLR 2022
4 EMERGENT CORPUS TRANSFER FOR NATURAL LANGUAGE TASKS
In this section, we present initial evidence that a corpus of emergent language DM can benefit two types of downstream tasks involving natural language: language-only task (language modeling, Section 4.1) and vision-language task (image captioning, Section 4.2). Moreover, we show such benefit most significant when the natural language resources are limited, which indicates that emergent language is potentially valuable for low-resource languages and tasks. We then analyze what contributes to such a transfer via ablation studies in Section 4.3
4.1 LANGUAGE MODELING
Inspired by Papadimitriou Jurafsky 2020). we consider the task of natural language modeling, and explore if pretraining a language model on corpus of emergent messages can reduce the downstream perplexity in a low-resource setup (Figure [b)), and how emergent language compares to other sources of synthetic and even natural language corpora.
Pre-training Corpora We compare three source corpora: (i) a Spanish Wikipedia corpus (es) culled to 50,000 vocabulary size using the pre-processing script from Papadimitriou & Jurafsky (2020), (ii) a regular language of well-balanced brackets (paren-zipf) with the Zipf unigram distribution as es, and (iii) an emergent language corpus (ec). paren-zipf the best-performing source corpus from Papadimitriou & Jurafsky (2020) not created by humans (e.g. natural language, music, or code) with the inductive bias of hierarchical structure, and es should set an upper bound of the transfer performance for both paren-zipf and ec. We vary the source corpus size from 2, 5, 10, 15 to 30 million tokens to understand how the transfer results change with respect to the pre-training scale.
Fine-tuning Corpora We scrape Wikipedia corpora of 10 languages to test downstream trans- fer: Danish (da, IE-Germanic), Basque (eu, Basque), Japanese (ja, Japanese), Romanian (ro, IE-Romancee), Finnish (fi, Uralic), Indonesian (id, Austronesian), Kazakh (kk, Turkic), Hebrew (he, Afro-Asiatic) Urdu (ur, IEdindic), and Persian (fa, IE- Iranian.. They come from diverse linguistic families with different levels of resource richness. Each corpus has million tokens and a vocabulary size of 50,000, and is pre-processed (to remove noise) via the same script to process es.
Implementation We train the EC game and generate ec based on the Conceptual Captions dataset (Sharma al. 2018), using more than 2.8 million natural images in the wild. Note that we do not use the natural language captions in the dataset for EC game training. We take the 512-dim pre-trained ResNet-18 18 (He et al., 2016) features before the classification head as input image features I. Other architecture and training details mainly follow Li et al. (2020b), and by default V = 4035, T 15, K 256. For language modeling, we adopt a Transformer Vaswani et al., 2017) with decoder layers and 6 attention heads, and pre-train on each source corpus for 3,000 steps with batch size 32, input length 1, 1,000, and learning rate 5 x 10-4 For fine-tuning and training from scratch on downstream corpora, the batch size is 8 and learning rate is 10-4, Hyperparameters are chosen after grid search. We report the test perplexity at the best validation loss.
Results As can be seen from Figure 2 pre-training on ec (blue) significantly reduces the perplexity compared with training from scratch (red, dotted line with constant value), with an average reduction of 24.6% with merely 2 million pre-training tokens. This is a positive signal that a corpus of emergent language could be useful beyond the communication game itself.
Comparing two synthetic pre-training corpora, ec performs better than or comparable to paren-zipf throughout different source sizes and downstream languages, with the exception of Basque and Finnish when pre-training size is more than 15 million. Given the strong inductive bias in paren-zipf (hierarchical structure and unigram information from real language), and the fact that ec is generated by just a single-layer GRU speaker without such explicit bias, such a result hints that ec may possess non-trivial structural properties due to the communication signals and perceptual grounding.
On the other hand, the natural language corpus es usually achieve a lower perplexity than both ec and paren-zipf may be expected. Most surprisingly, when pre-training only on 2M tokens, ec consistently beats es in terms of transfer performance, and their performances are comparable on most languages using 5 million source tokens. To offer one plausible explanation, could contain
5

---

Published as a conference paper at ICLR 2022
Persian (fa)
Basque (eu)
Danish (da)
Rom anlan (re)
Japaneee (ja)
1400
250
Parplsity
200
540
1200
200
120
paren-zipe
scratch
1000
200
100
150
Indonesian ((d)
Turkic (kK)
Hebrew (he)
Urdu (ur)
Finnish (1)
500
200
Perplechy
200
250
180
160
200
150
160
140
2 5 10 16 Source Size (Millon)
2 10
2.5 15
10
15
30
Source Size (Milloo)
Source Size (Milioe)
Source Sun Millon!
Source Sizw (Millon)
Figure 2: Test perplexity for language modeling on ten natural languages, when either pre-trained on a corpus of tokens from EC (ec, blue), Spanish (es, orange), well-balanced brackets (paren-zipf, green) or without any pre-training (scratch, red, dotted, constant with respect source sizes).
coco (5k)
coco (50k)
coco (full) +ec
Metries
base
+ec
+nl
base
+ec
+nl
base
+nl
BLEU4
14.1
15.3
18.5
26.3
27.2
28.2
35.8
36.2
36.0
ROUGE_L
39.8
41.0
44.4
50.0
50.7
51.5
56.9
57.0
57.1
CIDEr
30.8
35.7
48.0
78.3
82.8
88.8
117.5
118.1
118.5
Table 1: Results on image captioning transfer using three fine-tuning dataset sizes. base, +ec, +nl denote training from scratch and pre-training using EC captions and NL captions, respectively.
properties closer to everyday conversations smaller vocabulary, simpler structures (generated by one- layer GRU) and repetitive (all about image references) easier for language modeling, compared to the complicated structures and statistical patterns of large text corpora like Wikipedia es. However, the relative simplicity of also prevents the transfer from scaling well on 30 versus 2 million tokens, ec only reduces perplexity by 2.4% on average, while es reduces 23.8%. Such mixed finding points to potential in combining the strengths of simpler emergent corpora (easier to acquire and benefit from) and noisy natural language corpora (more structures and complexities) for more efficient language learning. Such an idea of mixing NLP and EC training has been useful for some NLP (Fried et al.) 2018 Lee et al. (2088) and EC (Lowe et al. 2020 Lazaridou jet al., 2020) tasks. Another important direction would on how to evolve emergent languages with more complexities that resemble and transfer to natural language, where existing approaches to improving and regularizing EC (Mu & Goodman| [021 Guo et al. 2019 Li & Bowling 2019 Dessi
comparing pretraining
ct al. 2021
Luna et al. 2020 could be tested through our transfer scheme.
4.2
IMAGE CAPTIONING
The language modeling transfer experiment leverages the structural properties of the source corpus, but has little to do with the semantics of the source corpus. For example, a regular language like paren-zipf has no real meanings, but still reduces downstream perplexity due to its hicrarchical structure. Therefore, we also consider transfer to image captioning, a classical vision-language task where a model has ground meaning of the words the images. As shown in Figure [[c). pre-train an image captioning model on unlabeled images to generate EC captions, then fine-tune on downstream annotated images generate natural language captions. We note that such a transfer scheme cannot work for other synthetic corpora such as music, code, or paren-zipf as they are not grounded on perceptual stimuli.
Pre-training Data caption M; based on each ResNet- 18 image feature I, in the dataset. As image captioning requires fine-grained information beyond global feature, we extract detection features I, for each image via pre-trained Faster R- R-CNN (Ren et al., 2015) and ResNet-101 (He et al.) 2016), and pre-train an image captioning model generate EC captions based on detection features (I, rー> M,). To set up an upper
We use the BC speaker trained Conceptual Captions to generate EC

---

Published as a conference paper at ICLR 2022
Method
LM (1o)
LM (he)
Metrics
LM (ro)
LM (he)
EC pretrain
198 (4)
375 (14)
Accuracy
0.672
0.737
from scratch
265 (1)
446 (8)
Topographic
0.030
0.003
random speaker
313 (21)
575 (24)
Translation 0.757
0.829
random input
245 (28)
489 (77)
permuted EC
211 (1)
383 (4)
Table 3: Pearson correlations be- tween different metrics (validation ac- euracy, topographic similarity, emer- gent to natural language translation ROUGE_L) and downstream language modeling performance (negated per- plexity).
model transfer (GRU)
655 (40)
1105 (6)
corpus transfer (GRU)
553 (53)
1056 (55)
Table 2: Ablations on EC game, corpus, and transfer setups. Standard deviations in parentheses. The language model is an GRU for last two rows and Transformer for others.
bound for transfer performance, we also use the English captions NL, in the Conceptual Captions dataset for pre-training (Ię ~ NL.).
Fine-tuning Data We use the MS-COCO dataset (Lin et al., (014) for fine-tuning, which consists of around 500,000 pairs of images and English captions. We use the full training set, or subset with 5,000 or 50,000 samples to study the transfer benefit when natural language annotation is limited.
Implementation We use a Transformer with 3 encoder layers and 6 decoder layers as the image captioning model, and perform pre- wre-training and fine-tuning seq2seq framework (Ott et 2019). Notably, we only transfer the encoder weights, and fine-tune the whole network end-to-end with learning rate of 10 as we notice transferring the full weights leads to worse performance, We report BLEUU (Papineni et al. 2002), ROUGE I. (Lin) 2004), and CIDEr (Vedantam et al., 2015) scores for the test split based on best validation loss.
Results As shown in Table I when fine-tuning on 5,000 50,000 samples, pre-training on either EC or NL captions both significantly improve the image captioning performance. For example, with 50.000 samples, EC pre-training can improve the BLEU-4 score by +0.9 points. while NL pre-training further improves +1.0 points. The fact that EC pre-training can provide half the benefit of NL pre-training is surprising, because the former requires no language annotation both training the EC games and generating the EC captions can be done unlabeled images in the wild, while the NL pretraining leverages more than 2.8 million English sentences.
However, when the full MS-COCO dataset is used, even pretraining on 2.8 million English captions not significantly helpful, possibly because Conceptual Captions collect noisy Internet sentences while MS-COCO annotates captions with less diversity (Wang et al. 2020) scratch on the full dataset can capture most of the output patterns. Other vision- langune tasks where harder more limited (e.g. instruction following, navigation) would
model trained from
natural language annotations potentially benefit more from EC pre-training, where we can turn abundant unlabeled visual stimuli into annotations in the emergent language.
4.3
WHAT CONTRIBUTES TO SUCCESSFUL CORPUS TRANSFER?
We have shown that emergent languages provide transfer benefit for natural language tasks, but it still unknown what properties of emergent languages contribute to such benefit. Hence we ablate several key aspects of the EC game training and EC corpus generation, and examine the effect via transfer (with 15 million pre-training tokens) to two example languages, Romanian (ro) and Hebrew (he). We leave ablation studies the image captioning experiment in Section A.3
Communication and Perceptual Stimuli in EC Game The emergent language stems from two elements in the game that human language acquisition relies on: perceptual grounding (input image features) and communication (referential game). For ablation, we first consider untrained GRU speaker with randomized parameters (random speaker), and use to sample tokens conditioned on image features according (T). As Table shows, pre-training such corpus leads to even worse
7

---

Published as a conference paper at ICLR 2022
performance than training from scratch, because an untrained speaker has highly random generations with no clear structure.
On the other hand, to ablate the effect of visual stimuli, we sample a set of N random vectors from (uu(Dr), o (D1)), and replace it for Dj as inputs of the EC game (random input). We then use the trained speaker to generate EC corpus based on the same random vectors. It turns out only one out of three game trials develop a game accuracy > 90%, and other two trials cannot learn an accuracy beyond 5%. On Hebrew, the "successful" trial achieves a transfer perplexity of 388, which is still worse than EC corpus on visual inputs, while other two trials yield significantly worse perplexities (573, 509). So it turns out communicating visual stimuli instead of random inputs more robustly successful, and more helpful for transfer even when communication can be successful.
Sentence Structure in EC Corpus To investigate how structured EC sentences are, we indepen- dently shuffle each EC sentence in Da to form new--eeetrad www-rrrrriiiin corpus (permuted EC), and find in Table[[that it indeed leads to a consistent worse performance, though better than training from scratch. This indicates the speaker develops coding scheme with sentence structures beyond bag-of-words.
Model vs. Corpus Transfer Li et al. (2020b) propose to transfer the EC models for downstream translation task, whereas we leverage a speaker-generated EC corpus to pre-train with architectures that could be more flexible and powerful, which leads to several significant practical advantages. First, EC model architectures might not apply to many downstream tasks (e.g. image captioning with multiple input features), but corpus transfer could still be readily deployed (Section 4.2). On other tasks (e.g. language modeling) where model transfer could be implemented, its performance is still constrained by the simple EC model architecture45] As shown in Table 2 directly fine-tuning the one-layer speaker GRU (model transfer (GRU)) indeed leads to much worse perplexities than corpus transfer with Transformer (EC pretrain), but more interestingly, is also worse than corpus transfer with the same GRU architecture (corpus transfer (GRU)), possibly due to the benefit of self-distillation (Zhang et al., 2019 Allen -Zhu & Li 2020). In summary, our corpus transfer approach better utilizes the emergent communication for downstream tasks than model transfer.
5 EMERGENT-NATURRE LANGUAGE TRANSLATION AS A METRIC
Section presents initial evidence that the quality of emergent language correlates with the transfer performance, by showing ablated emergent languages transfer worse. However, directly using transfer as evaluation scheme for emergent languages has certain deficiencies. First, the pre- training and fine-tuning optimizations are computationally expensive. Second, transfer performance across different downstream tasks might induce inconsistency. Hence we propose a cheaper and simpler evaluation for emergent languages by measuring how easily it translates to the corresponding natural language grounded on the same input, and show that it better correlates with the transfer performance than existing metrics across different tasks.
More concretely, we use a small set of captioned images, where image I, is associated with a natural language caption NL, We also use the trained speaker to generate emergent message M, based on each I. and train translation model to map emergent sentences to natural sentences M, H> NL, Intuitively, higher translation score means the emergent and natural sentences are closer in structure and semantics, similar to how French-English translation might be easier than Chinese-English.
Implementation To estimate the correlation between metrics and transfer performance re- quires population of emergent languages with different speakers. Thus we choose 5 EC game setups with varying vocabulary and sequence length limits ((VTT) E {44035 5), (4035, 15), (4035, 25), (1000, 15). (10000, 25)}), and for each setup we run 4 trials for 2,000 steps. We consider checkpoints every 200 steps, summing to 5 x 10 = 200 checkpoints Due to computational limits, we train EC games on a small subset of 50,000 MS-COCO images, and use another subset of 50,000 MS-COCO image-caption pairs to calculate the translation metric. For translation, use the same seq2seq Transformer for image captioning. but only train for 2 epochs, so that an emergent language more similar to the natural language can perform better with limited
We note that unlike most tasks, EC model architectures might not arbitrarily scale up due optimization challenges with discretization. For example, we have tried 2/3-layer GRUs for FCC, but the transfer is worse.
8

---

Published as a conference paper at ICLR 2022
(a) Metric (Val Accuracy)
(b) Metric (Topographic)
(c) Metric (Ours, ROUGE L)
(d) Perplexity (ro)
100
0.3
200
0.2
0.40
-20
-20
0.36
-200
0
200 400 600 800 1000 Train steps
200 400 600 800 Train steps
1000
200 400 600
800 Train steps
200 400 600 800 1000 1000
Train steps
Figure 3: Different metrics and downstream Romanian perplexities (negated) with respect to training steps, averaged over four trials with vocabulary limit 10000 and sequence length limit 15. Our metric better correlates with downstream performance across time steps.
2004 as the translation metric, and find other metrics with similar results. For downstream language modeling, we use ImageNet (Deng al., 2009) generate a corpus of 15 million tokens and fine-tune on Romanian (ro) and Hebrew (he).
training. We use ROUGE L.
Results We compare with the two existing metrics, validation accuracy and topographic similarity (see Section 3.2 Table 3 shows how different metrics correlate with downstream performance across all checkpoints form all trial:] Surprisingly, we find little correlation between topographic similarity and downstream performance, while our translation metric has better correlation (Pro- 0.757, Piee = 0.829) than validation accuracy (Pro = 0.672, Phe = 0.737).
To better understand the results, take all 4 trials from specific game setup (V = 10000, T and plot in Figure 3 hoo the three metrics (accuracy, topographic, translation ROUGE L) as well as the downstream Romanian performance change with training steps (similar analysis of more setups (vocabulary size, sequence length) in Section B.4). We observe that our metric (Figure 3]c) aligns with the downstream performance (Figure 3d) in that both reach a high level around 300 steps (red
15)
dash line) and maintains a similar level afferwards. The validation accuracy ((Figure ((a)) has a generally similar tendency as these two metries. with the difference being that it only around 50% at step 300 and far from convergence, possibly due to the underdevelopment of the listener. A similar observation was made in Li et al. (2020b). Such a dependency on the listener renders game success unreliable for the evaluating emergent languages produced by the speaker alone.
On the other hand, the topographic similarity (Figure peaks as early as step 300, after which the degradation fails to explain the downstream performance. To illustrate the misalignment, consider the "best" language with respect to topographic, i.e. a fully disentangled coding scheme (e.g. "red/blue cube/sphere"). Such corpus would essentially have maximum possible entropy. pre-training on which should not help with any downstream tasks. Our finding suggests that simple measurements of rigid disentanglement fall short of estimating the compositionality of real language, and we may need to take into account other structural properties of natural language, e.g. stable irregular- ity (Kirby (2001), the role of argument structures, context and morphology (Goldberg 2015 2020).
6 CONCLUSION
While previous research in language emergence tends to investigate different setups and how they affect certain metrics, such paradigm cannot provide robust and valuable insights if the metrics are flawed. Thus, our work calls for a paradigm shift in how we evaluate and analyze language emergence by stepping from metries within each EC framework or over-approximate NL properties, and instead directly linking to NL by transfer or translation. Such a paradigm shift enables several brand new and exciting future directions how to improve regularize EC through the lens of our metrics, how to push EC to be more useful for ] and how to design more fine- mne-grained metrics for specific NL properties. We believe progress in these directions will further develop a synergy between EC and NLP research and bring benefits and insights for both sides.
For topographic, we exclude 9 checkpoints with the metric undefined due to zero variance of edit distances.
See relevant discussions the end of Section 4.1 and Section 4.2

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENTS
This work was supported by MIT-IBM Watson AI Lab and its member company Nexplore, ONR MURI (N00014-13-1-0333), DARPA Machine Common Sense program, ONR (N00014-18-1-2447) and MERL. The information, data, or work presented herein was also funded by the Advanced Research Projects Agency-Enenn
(ARPA-E), U.S. Department of Energy. under Award Number DE-AR0001 210. SY and KN also acknowledge support from the National Science Foundation under Grant No. 2107048. The views and opinions of authors expressed herein not necessarily state or reflect those of the United States Government any agency thereof.
ETHICS STATEMENT
Our work is preliminary step toward aligning the language of machines and humans, which has potential positive social impacts several ways. First, shaping machine communications towards the structure of natural language contributes human and Al alignment in terms of values and understanding of the world. Second, emergent languages could yield transfer benefits for low- resource languages and tasks, contributing to linguistic diversity and inclusion. We could not think of negative ethics impact for now, as the emergent language is still very different from natural language.
REPRODUCIBILITY STATEMENT
Our research in based public codebases and datasets, with implementation details described throughout the paper. We have conducted reasonable hyperparameter searches and ablation studies make experiment conclusions fair. In the Appendix, we further provide links to all used code and data resources, and additional details including hyperparameter search setups and resource requirements for each experiment, We will clean and release the code upon acceptance.
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. arXiv preprint arXiv:2012.09811 2020.
Muhannad Alomari, Paul Duckworth, David Hogg, and Anthony Cohn. Natural language acquisition and grounding for embodied robotic systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
Jacob Andreas and Dan Klein. Analogs of linguistic structure deep representations. arXiv preprint 1rxivv1707.08// 2017.
Jacob Andreas, Anca Dragan, and Dan Klein. Translating neuralese. arXiv preprint arX1v:1704.06966 2017.
Michelle Barton, Input and interaction language acquisition. Cambridge University Press, 1994.
Natalie I. Bohlmann and Jason Downer. Self-regulation and task engagement as predictors of emergent language and literacy skills. Early Education and Development, 27(1):18-37, 2016.
Henry Brighton and Simon Kirby. Understanding linguistic evolution by visualizing the emergence of topographic mappings. Artificial life, 12(2):229-244 2006.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005. 4665, 2020.
Jerome Bruner. The role of interaction formats in language acquisition. In Language and social situations, pp. 31-46. Springer, 1985.
Angelo Cangelosi and Domenico Parisi. Computer simulation: A new scientific approach to the study of language evolution. In Simulating the evolution of language, pp. 3-28. Springer, 2002.
10

---

Published as a conference paper at ICLR 2022
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Compositionality and generalization in emergent languages. arXiv preprint arXiv:2004.09124,
2020.
Michael Cogswell, Jiasen Lu, Stefan Lee, Devi Parikh, and Dhruv Batra. Emergence of compositional language with deep generational transmission. arXiv preprint arXiv: 2019.
Verna Dankers, Elia Bruni, and Dieuwke Hupkes. The paradox of the compositionality of natural language: a neural machine translation case study, arXiv preprint arXiv:2108. 05885, 2021.
Jill G De Villiers, Jill De Villiers, Peter A De Villiers, and Peter A DeVilliers. Language acquisition. Harvard University Press, 1978.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Ieee, 2009.
Roberto Dessi, Eugene Kharitonov, and Marco Baroni. Interpretable agent communication from scratch (with a generic visual processor emerging on the side). arXiv preprint arXiv:2106.0423 04.5888 2021.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.0400 2018.
Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philiges Morency, Taylor Berg- Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. followr models for vision-and- -nn-laggugee navigation. arXiv preprint arXiv:1806.0277 2018.
Adele Goldberg. Compositionality. In The Routledge handbook of semantics, pp. 435-449 Routledge, 2015.
Shangmin Guo, Yi Ren, Serhii Havrylov, Stella Frank, Ivan Titov, and Kenny Smith. The emergence of compositional languages for numeric concepts through iterated learning in neural agents. arXiv preprini arXiv:1910.0522 2019.
Stevan Haraad. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-9):335-346, 1990.
Serhii Havrylov and Ivan Titov. Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. arXiv preprint arXiv:1705.1113 11192 2017.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778, 2016.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611. 1.01 144, 2016.
Eugene Kharitonov and Marco Baroni. Emergent language generalization and acquisition speed are not tied compositionality. arXiv preprint arXiv:2004 20.44400 2020.
Simon Kirby. Spontaneous evolution of linguistic structure-an iterated learning model of the emer- gence of regularity and irregularity. IEEE Transactions on Evolutionary Computation, 5(2): 102-110, 2001.
Simon Kirby. Natural language from artificial life. Artificial life, 8(2):185-211 2002.
Simon Kirby, Monica Tamariz, Hannah Cornish, and Kenny Smith. Compression and communication in the cultural evolution of linguistic structure. Cognition, 141:87-102, 2015.
Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40, 2017.
Angeliki Lazaridou and Marco Baroni. Emergent multi multi-agent communication in the deep learning era. arXiv preprint arXiv:2006.000 02419, 2020.
11

---

Published as a conference paper at ICLR 2022
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. arXiv preprint arxiv:1112.07182, 2016.
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of lin- guistic communication from referential games with symbolic and pixel input. arXiv preprint arXiv:/804.03984, 2018.
Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman. Multi-agent communication meets natural language: Synergies between functional and structural language learning, arXiv preprint arXiv:2005. 07064, 2020.
Jason Lee, Kyunghyun Cho, Jason Weston, and Douwe Kiela. Emergent translation in multi-agent communication. In International Conference on Learning Representations, 2018.
Vladimir Levenshtein et al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pp. 707-710. Soviet Union, 1966.
Fushan Li and Michael Bowling. Easc-of-g and language structure from emergent communi- cation. arXiv preprint 1906:::900.000000 02403, 2019.
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In European Conference on Computer Vision, pp. 121-137. Springer, 2020a.
Yaoyiran Li, Edoardo Maria Ponti, Ivan Vulić, and Anna Korhonen. Emergent communication pretraining for few-shot machine translation. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 4716-4731, 2020b.
Chin-Yer Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 81, 2004.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Ryan Lowe, Abhinav Gupta, Jakob Foerster, Douwe Kicla, and Joelle Pineau. On the interaction between supervision and self-play in emergent communication, arXiv preprint 2rXiv:2002.01093, 2020.
Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Pretrained transformers as universal computation engines. arXiv preprint arXiv:2103.05247 2021.
Diana Rodríguez Luna, Edoardo Maria Ponti, Dieuwke Hupkes, and Elia Bruni. Internal and external pressures on language emergence: least effort, object constancy and frequency. arXiv preprint 000:20004.03868, 2020.
Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations. In Thirty-second AAAI conference on artificial intelligence, 2018.
Jesse Mu and Noah Goodman. Emergent communication of generalizations. arXiv preprint arXiv:2106.0266 2021.
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. arXiv preprint MwXivv:904.0100 2019.
Isabel Papadimitriou and Dan Jurafsky. Learning music helps you read: Using transfer to study linguistic structure in language models. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 6829-6839, 2020.
Kishore Papineni, Salim Roukos, Todd Ward. and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
12

---

Published as a conference paper at ICLR 2022
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances neural information processing systems, 28: 91-99, 2015.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 2556-2565, 2018.
Kenny Smith, Henry Brighton, and Simon Kirby. Complex systems in language evolution: the cultural emergence of compositional structure. Advances in complex systems, 6004):537-558, 2003.
Shane
Toward the emergence of nontrivial compositionality. Philosophy of Science, 87(5):897-909) 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, tukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4566-4575, 2015.
Paul Vogt. The physical symbol grounding problem. Cognitive Systems Research, 3(3):429-45)) 2002.
Kyle Wagner, James A Reggia, Juan Uriagereka, and Gerald Wilkinson. Progress in the simulation of emergent communication and language. Adaptive Behavior, 11(1):37-69 2003.
Zeyu Wang, Berthy Feng, Karthik Narasimhan, and Olga Russakovsky. Towards unique and informa- tive captioning of images. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VII 16, pp. 629-644, Springer, 2020.
Ludwig Wittgenstein. Philosophical investigations. New York: Macmill an Publish, 1958.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the thee---rthhcrrttt natural language processing. arXiv preprint drX1::1910003771, 2019.
Michaela Zajicek-Farbee The contributions of parenting and postnatal depression on emergent language of children in low-income families. Journal of Child and Family Studies, 19(3)2257-269, 2010.
Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 3713-3722, 2019.
13

---

Published as a conference paper at ICLR 2022
A IMPLEMENTATION DETAILS
A.1 EMERGENT COMMUNICATION GAME
We adapt the public code h from Li al. (2020b) and mostly follow their default setups. For training.
we use batch size of 256, and each batch element contains one input images and other 255 distractor images. Since the Conceptual Captions dataset has more than 2.8 million images, random sampling batch of data is computationally costly. So for each batch, we first sample 50,000 images from the whole dataset, then sample input and distractor pairs from this subset. We use an Adam optimizer with learning rate 10 We use soft version of Gumbel-softmax with temperature and have tried hard Gumbel-softmax and found it not further helpful for downstream performance. Each game training only takes less than 12 hours using GeForce RTX 2080 GPU.
A.2 LANGUAGE MODELING
We the public script" from Papadimitriou & Jurafsky (2020) to pre- pre-process Wikipedia corpora of different languages, using the default setup of culling to 50,000 vocabulary size. We hand-pick downstream languages to make sure they represent different linguistic families.
We use the language modeling scrip fine-tuning.
2 from Huggingface (Wolf et al. 2019) for both pre-training and
We have tried grid search for the pre-training learning rate (10-3, 5 x 10-4 10-1) and batch size (4, 32), which checkpoint transfer (1000, 2000, 3000), as well the fine-tuning learning rate (10-4,5 x 10 10 and batch size (8, 32). We find that for all three source corpora (es, ec, paren-zipt), works best pre-train with learning rate 5 x 10-4 and batch size (32), transfer using the checkpoint with 3000 training steps, and fine-tune with learning rate 10 and batch size 8. For training from scratch, we have tried grid search for the learning rate (10 5 10 10 x 10 10-5) and batch size (4, 32), and find that learning rate 10-4 and batch size 8 work best for different downstream languages. An pre-training experiment can finish within one hour using one GeForce RTX 3090 GPU, while fine-tuning training- wer---froorrrrree experiment can finish within one hour using one GeForce RTX 2080 GPU.
A.3 IMAGE CAPTIONING
We use the pre- pre-processed detection features of Conceptual Captions from the codebase of Li et al. (0220)).
For both pre-training and fine-tuning, we use a public codebase 10 for image captioning based on FAIRSEQ (Ott et al. 2019), and mostly follow their default setups. Pre-training Conceptual Captions takes 8 Geforce RTX 3090 GPU for around two days. Fine-tuning takes GeForce RTX 2080 GPU for one hour.
B ADDITIONAL RESULTS
B. LANGUAGE UNIGRAMS
As shown in Figure 4 the es and paren-zipf corpora have a larger vocabulary size (5000) and a larger entropy (6.48). While is set with vocabulary limit 4,035, corpus only uses around 2,500 words with smaller entropy (3.7). The ee corpus with random speaker almost has large entropy (7.98).
wiki corpus
wtg.gaggeooobiiinnnnnnn.....cc.a clm.
14

---

Published as a conference paper at ICLR 2022
and paren-zipf, antropy=6.48
3.7
(random speaker), entropy-7.9e
16-
14-
15-
12
13-
10-
5-
12-
6--
11-
6-
10-
4-
10000 20000
30000
40200
50000
500
1000
1500
2000
2500
3000
2000
3000
4000
Figure 4: Unigram distributions of (1) es and paren-zipf, (2) ec, and (3) ec with random speaker.
CIDEr
CIDEr
CIDEr
sonathh 5000)
Ni.re-tral looco 5000)
NL pre frain tooco 60000)
pre trin (coc0 full
EC
EC 60000)
scratch
1.16
0.4
0.8
1.14
0.3
1.1
0.2
0.4
1.08
1.08
0.1
0.2
1.04
Step
Step
1.02
Step
0
0
3
2
3
Figure 5: The validation CIDEr (Vedantam et al., 2015) score across different fine-tuning epochs, when using 5,000, 50,000, or the all samples of MS-COCU training samples.
B.2 IMAGE CAPTIONING
We visualize the fine-tuning process of image captioning experiments in Figure 5 Interestingly. find that under different natural language resource conditions (5,000, 50,000, or all samples in the MS-COCO (Lin et al.) 2014) training set) the training progress is different. Specifically, with 5,000 samples, BC or NL pre-training and training from scratch first learn similarly well, then gaps gradually appear with more training epochs. In contrast, when more han 50,0000 samples are used, the gap between pre-training methods and training from scratch is most significant when trained for only one epoch, and it starts to diminish with more training epochs. It suggests that even when downstream natural language resources are abundant, pre-training on an EC corpus might still help in a fast adaption setup.
We also perform additional ablation studies to confirm the non-triviality of image captioning results. Due to computation limits, use 450k MS-COCO samples (instead of 3M Conceptual Captions samples used in the main paper) to pre-train (image + emergent caption/ / shuffled emergent caption / random paren-zipf string with the same unigram emergent captions), and the other 50k MS-COCO samples to finetune (image H English caption). As shown in Table 4 pre-training with shuffled emergent captions ungrounded strings leads to worse performance than even pre-training. while pre-training with emergent captions still improves the downstream performance. This helps address the importance of having semantically grounded and structural language for vision-language pre- training. However, we do note that in the case of MS-COCO image captioning, fine-tuning usually does the heavy-lifting and pre-training contributes much less than the case of language modeling. That is why our work focuses analysis on the language modeling experiments.
B.3 CORRELATIONS OF METRICS AND DOWNSTREAM PERFORMANCE
Figure ploos the 200 data points in terms of their metric values and downstream performance on Romanian and Hebrew modeling, respectively. We note that topographic similarity is usually very
15

---

Published as a conference paper at ICLR 2022
Pre-training language
CIDEr
Emergent language
62.5 (1.4)
Emergent language (shuffled)
(0.5)
Paren-zipf
59.5 (0.2)
No pre-training
60.9 (0.5)
Table 4: Image captioning results with different pre-training languages or no pre-training.
Pearson Correlation:: 0.672
Pearson Correlation: 0.757
Pearson Correlation: 0.030
-200
sp9 (ro)
-250
-300
-350
20
40
80
100
0.30
0.35
0.40
0.45
0.0
0.1
0.2
0.3
0.4
800
ROUGE_L
Pearson Correlation: 0.737
Pearson Correlation: 0.829
Pearson Correlation: 0.003
-400
-450
1apl (ho)
-500
-600
-650
20
40
80
100
0.30
0.35
0.40
0.45
0.0
0.1
0.2
0.3
0.4
acc
ROUGE_L
Figure 6: 200 data points with x axis being metrics (validation acuracy, translation ROUGE L, topographic) and y axis being downstream performance (negated perplexity) on Romanian (ro) and Hebrew (he), respectively.
low (less than 0.1), and a higher value does not correlate with better downstream performance. Also, note that data points around the beginning of training (e.g. 200 steps) have very low accuracy translation score, and their downstream performance is worse but significantly more varied than the rest of data points.
B.4 How SETUPS (VOCABULARY SIZE, SEQUENCE LENGTH) EFFECT METRICS AND DOWNSTREAM PERFORMANCES
Figure in Section 5 mainly studies how metrics and downstream performance change with respect the training steps. while controlling the other setups. Following similar logic, in Figure wee take those checkpoints with 1000 training steps and sequence lengths being 15 and plot the downstream performances and metrics with respect to the vocabulary size (1000, 4035, 10000), and Figure take those checkpoints with 1000 training steps and vocabulary size being 4035 and plot the downstream performances and metrics with respect to the sequence length (5, 15, 25).
For the vocabulary size setup (Figure 7 that the performance when vocabulary size is 1000 is worse and has significantly more variance. In contrast, topographic similarity is highest when vocabulary size is smallest (1000), which disagrees with downstream performances. However, all metrics are higher when vocabulary size 4035 instead of 10000, which is the opposite for downstream performances.
we find that both validation accuracy and metric capture
16

---

Published as a conference paper at ICLR 2022
For the sequence length setup (Figure 8), find that metric captures the fact that a larger sequence length leads to better and less varied downstream performance. In contrast, validation accuracy with sequence length being 25 is worse than sequence length being 15, and topographic similarity fails to capture the decreasing trend of performance variance.
In summary, find that larger vocabulary size sequence length within setup choices leads to better downstream performance with less variance, which is best captured by our translation metric.
Morda
(Topograri))
(Ous, ROUSE 1)
Perlooly
Perlodly
0.4
-110
-215
-215
-20
V000
Figure 7: How metrics and downstream performances change with respect to vocabulary size (1000, 4035, 10000). Sequence length is 15 and training step is 1000.
(4) Matro (Val Acouraa)
Metiic (Toosgratecc
(5) Matre (Dues, HOUSE 13)
(8) Pwwpectyyyyyyy [ts
-20
(4)- Perckoiit (he)
0.175
0,20
9429
-113
0.05
Figure 8: How metrics and downstream performances change with respect sequence length (5, 15, 25). Vocabulary size is 4035 and training step is 1000.
B.5 QUALITATIVE EXAMPLES
Figure D iincludes some COCO images (not Conceptual Captions training images for EC) and their natural and emergent captions. Manual check might reveal some patterns, c.g. transportation tools might start with 2340 (a, b, j), indoor scenes might start with two 1777 tokens and indoor home scenes might start with even more 1777 tokens (d, e, k, l). Still, we note that our paper intentionally aims to move away from previous manual-check paradigms in EC papers and tries to establish more standardized, scalable, and robust quantitative metric to enable more progress.
17

---

Published as a conference paper at ICLR 2022
(a) large passenger airplane fly- ing through the air. [2340, 2403, 2308, 320, 1329, 2308, 2308, 1329, 3643, 1512, 4033, 1298,
(c) green street light in between two buildings. [1603, 320, 2422, 1965, 2403, 3098, 319, 1965, 1307, 1526, 1307, 2389, 1526,
(b) delta airplane at the airport on
the runway. [2340, 2422, 2403, 141, 2422, 320, 2340, 2308, 3701, 110, 3701, 397, 1241, 587, 0]
1526, 1526, 0]
1223, 0]
(d) this kitchen has stainless steel appliances and granite counter- tops. [1777. 1777, 1777, 1777, 1298. 1777, 3682, 2823, 3701, 7701,,643, 4033, 1526, 1526, 0]
(c) black white and brown cat looking into toilct bowl. [1777, 1777, 1777, 1043, 1777, 1043, 1976, 1777. 2422, 2550, 2762,
(f) large airplane and truck next to building. [1777, 2403, 2422, 2308, 1965, 814, 2308, 397, 1526.
477, 2422, 3715. 0]
477, 2389, 1526, 2389, 1526, 0J
(g) there are bicycles parked along stone sidewalk. [1777. 1965, 2403, 1329, 1329, 1329, 319, 2277, 1329, 1526, 1526, 3701,
(h) cat sitting next to bunch of bikes parked next to each other.
(i photo of a sidewalk and yard with fire hydrant and signs. [1603, 2422, 319, 1603, 1043, 319, 2762, 320, 2277, 2762, 141, 3701, 2389, 1526, 0]
[2422, 1777, 1043, 320, 1329, 1043, 3643, 3682, 3643, 477, 3701, 2944, 2277, 2389, 0]
1526, 2277, 01
(i) man standing on vehicle with two large wheels. [2340, 1777.
(k) television screen is mounted in
the wall of the bathroom. [1777,
()) restaurant that has many ta- bles set up . [1777, 1777, 3098, 320, 320, 1298, 1965, 2823, 3701, 1298, 1714, 1241, 3701, 4033, 0]
1965. 319, 1329, 3682, 1329, 1329. 3643, 1329, 165, 3643,
1777, 1777, 1777, 1777, 2762,
1965, 2550, 3226. 4012, 2823,
2389, 2389, 0]
2550, 2655, 4033, 0]
Figure 9: Images from COCO and their natural and emergent captions.
18


Reviews:
[{"review_id": "bM1QjyIArTa", "reviewer": "Reviewer_QkKj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- I think this research question is important and interesting, and that the study of what EC agents output and how they might relate to natural language is important for both EC and NLP\n- The setting was made very clear. Figure 1 is excellent.\n- The claims, particularly those about language modeling, are evaluated on a variety of settings\n\nWeaknesses:\n- Some of the claims don't seem entirely justifiable. It's true that pretraining on EC is better than pre-training on ES or parens at 2M tokens, but it's not entirely clear to me that this is an interesting setting because _all_ models perform poorly. It's more interesting to me that ES is similar to parens\n- It's not clear to me how specific to this particular experimental setup the proposed evaluation technique is. It seems to correlate better with LM performance in this setting, but how about others? Using a particular method for evaluation requires some degree of confidence that it holds generally, and I'm not sure that the results here convince me of that.", "summary_of_the_paper": "This paper studies whether output from emergent communication systems might be useful as pre-training data for natural language tasks. The authors train an EC agent, generate from it, and use the generated IDs as data for language model pre-training. Compared to pre-training on Spanish wikipedia or a simple bracket language, pre-training on EC-generated data is better when you have 2M examples, but quickly becomes worse as the amount of data increases. In experiments on image captioning, the authors find that pre-training on either EC-generated or natural language captions improve over not pre-training, but that there is not much of a difference between the results when pre-training on EC-generated text and natural language. Finally, the authors propose EC -> natural language translation as a way of evaluating the quality of EC agents.", "summary_of_the_review": "I think this work addresses an interesting question, but I disagree with a few of their claims based off of the experimental evidence provided. The broader research question is important enough that I feel like even initial work in this direction is perhaps of interest to the ICLR community and worth accepting.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Rl8WUe07Ms_", "reviewer": "Reviewer_wvqW", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper's very clearly written, well-organized, and much of the discussion was grounded in a broad range of references to previous work.  My concerns with this paper are in regards the experimental design, the strength of the conclusions, and how the authors are choosing to interpret them.\n\nPerhaps the best way to discuss this paper is through the authors' goals, to understand: unsolved: \"(i) can emergent languages be used outside the game? (ii) if so, would these metrics predict the usefulness of emergent languages for downstream tasks? We tackle these two questions in Sections 3 and 4 respectively.\"\n\n## can emergent languages be used outside the game?\n\nTo question (1), it is of course important to note that this has already been shown in Li et al. 2020 in the context of pre-training for MT.  There is plenty of opportunity to broaden the scope of their findings, but as it pertains to this question, it no longer needs to be asked and should be presented truthfully in this way.  This language persists throughout, and a reader not aware of Li's work would easily go through most of the paper believing that this paradigm is being attempted here for the first time.  For instance, even at the end of the paper \"We present a new perspective for studying emergent communication by linking the corpora of emergent and natural languages in two ways.\"  One of these seems like an established perspective, and what sets this work apart is primarily the choice of tasks: language modeling and image captioning.  \n\nReturning to the question, can emergent languages be used outside the game for language modeling or image captioning?  First, it is also important to note that this EC setup is only one possibly game, and the properties of the EC are hugely shaped by the properties of the game.  So again, for accuracy, it's important to clearly define the scope of the work as pertaining to this one particular scenario.\n\nFor language modeling, the authors show that indeed the EC pre-training is useful for the downstream language modeling task, but the results are not very convincing that this would ever be useful in a practical sense.  Pre-training on EC data is rarely the best option, often outperformed by simple synthetic language data, and is never the best option as more data is observed.  And since pre-training on Spanish is still the most effective strategy even in low-resource languages, and Spanish is more easily acquirable than EC data, in what situation would one ever opt for using this strategy?  Back to the research question, yes, EC languages *can* be used outside the game, but it's not apparent that it would ever be best choice nor the easiest.\n\nThe case is similar for the captioning task, except the margins (even those between the best choice -- natural language -- and the worst choice baselines) is exceedingly small.  I think it throws into question whether MS-COCO was ever a good testbed for this research question, as (as the authors note) the captions are very regular and structured.  It seems like a reasonable attempt, but it's difficult to find a take-away.\n\nWhen it comes to understanding the properties of the emergent languages, how these properties are translating into useful biases for downstream tasks, it is also left quite open.  The ablation experiments compare against random scenarios, and I personally wasn't able to gain much insight into where the performance comes from.  Personally I would have liked more qualitative analysis into the real correlations and differences of the EC and natural languages as it pertains to the downstream tasks, but even barring that I imagine there are also some purely quantitative metrics which could have been more targetted, and in doing so, be more informative.\n\n## if so, would these metrics predict the usefulness of emergent languages for downstream tasks?\n\nAnother thread of this work is criticism of some existing EC metrics, like topographical similarity, which may be failing to correspond well to the types of structure and compositionality found in NLs.  The authors propose a new method of evaluating ECs by training (briefly) a translation model to translate to natural language captions from EC language generated from the same images.  The metric corresponds better to downstream task accuracy than topographic similarity.  \n\nThe authors argue that \"intuitively, a higher translation score means the emergent and natural sentences are closer in structure and semantics, similar to how French-English translation might be easier than Chinese-English\".  But isn't the purpose of a such a linguistically-focused EC metric to evaluate how well EC reflects the types of structures found in natural (big-L) Language?  i.e., not any specific language.  It seems that in closely tracking the language used in the caption data, it is only natural that this relates to higher downstream task accuracy, but isn't that at the cost of low similarity to another natural language?  Criticisms of existing metrics are likely warranted, but I fail to see how this metric really improves upon it.  An EC learns a Chinese-like structure from a caption dataset, the translation model performs poorly at mapping it to English (vs a German-like EC source language) and now we have a plausible, NL-like EC with low scores under the metric.  I think ideally a good EC metric should score all human languages highly, or one should answer the question of which of two human languages contains more \"human-like\" language structure.  A metric that targets specific aspects of human languages cuts around this issue, but I think in this case the authors are burdened with providing some answer to this question.\n\nSo overall the story of the paper feels like a bit of circuitous reasoning: EC languages might be more helpful to downstream tasks than natural languages if they somehow capture a greater structural similarity to the target (presumably low-resource) language than available natural languages.  But the closer the language to existing natural language, the less useful it is, since it doesn't provide a unique advantage over simply using that natural language.  And in the space of all human languages, only the one closest in structure to the particular fine-tuning task achieves high marks.  So I'm curious how the authors reconcile this?\n\n\n## Conclusion\n\nOverall the work is tackling important problems with an interesting method, but in its current state I believe it's lacking clear insights into the problem, does not provide a practical usefulness, and doesn't dig sufficiently deep in understanding why models behave the way they do.  It also feels almost like two disjoint papers -- a paper could explore pre-training on ECs but with more focus given on understanding the properties of the ECs, and broadening it to more games to ensure lessons are more generally applicable.  Another paper could target metrics like topographical similarity, but in this case it is important to discuss the seemingly obvious failure cases of the proposed measure.\n\nI think this is summed up well in the conclusion:\n\"Our methodology and results open up possibilities of establishing a synergy between the research of language emergence and natural language processing, where we could potentially better understand and improve emergent communication through the lens of natural language, and in turn improve upon low-resource natural language tasks with the help of emergent languages\"\n\nWhile the phrasing is again speculative/potential, I find it hard to view the paper experiments as strongly supporting either of these cases: I don't think we've learned much about the structure of ECs, even the limited examples used here in the paper, and I don't know of situations where EC would reasonably be used for improving low-resource monolingual NLP over NLs.\n\nThat's not to say the paper doesn't have a lot going for it, but each experiment felt like it only scratched the surface of the problem, and more thorough research feels necessary.\n\n\n\nTypos / Grammar:\n((Figure 3(a)))\n\"Our research in based on\"\n\n", "summary_of_the_paper": "This paper explores the use of pre-training models on corpora of emergent languages to improve performance on downstream tasks (in this case language modeling and image captioning).  The authors compare against using other types of pre-training corpora, like synthetic or natural languages.  While not always optimal, the authors demonstrate that there is clearly an advantage to using emergent languages in this way, if one is otherwise considering training from scratch.  They propose a new metric for evaluating whether languages have structure, doing so by training a translation model to map emergent language to natural language on the same data, and validate the measure by showing it corresponds closely to task accuracy. ", "summary_of_the_review": "The paper is well-structured and the experiments seem generally sound, but they're too brief and at too high of a level to provide real insight into EC language structure and how we should identify and measure it.  However, I do call into question really how sound the ablation experiments are, since they are contrasted with such trivial baselines that it again hinders the paper from providing novel insights.  There evaluation measure is I think the closest thing to a novel contribution, and the authors seem correct in pointing out the shortcomings of existing measures, but the proposed solution doesn't seem a suitable replacement (it seems like it violates some core assumptions of what the community would want from an EC metric, and is also language and task specific).  Overall it seems like good preliminary work, and I hope to see a deeper exploration using these same methods and motivations.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "OBWsOt8Lyc5", "reviewer": "Reviewer_G942", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths\n\n- This is a great idea, and one of those ideas that I (and others) will wish they had come up with. The idea of pretraining on emergent corpora neatly brings together new ideas about transfer learning and modern studies of emergent communication.\n- Significant experiments in both grounded (captioning) and ungrounded (LM) settings convincingly demonstrate the efficiency of EC pretraining. The ablation studies are also very useful, but also open up several questions (see Weaknesses).\n- There is a very rich space of future experiments to try in this area. I imagine it will inspire plenty of follow-up work, exploring how different aspects of the EC corpus do or do not result in better transfer. This may also become a typical evaluation w.r.t. how useful/systematic a language corpus is—how useful is it for downstream transfer?\n- The natural language translation metric is extremely clever, and is a natural extension of similar ideas for \"translating\" emergent sentences (Andreas et al., 2018).\n\n# Weaknesses\n\n- Poor comparison to parameter transfer approaches\n    - Perhaps the biggest issue I have with the paper is the lack of comparison to parameter transfer approaches (Li et al., 2020). The GRU transfer line in Table 2 is the right idea, but is woefully insufficient: it makes no sense to compare a 1-layer GRU perplexity to transformer complexity, and there's no reason that you can't generate the EC corpus with transformers and therefore more cleanly compare corpus transfer to parameter transfer. I would be really interested in these results. To be clear, I don't think it's necessary that corpus transfer has to do \"better\" than parameter transfer for the paper to be useful—there are some potential advantages of corpus transfer: (1) learning from EC corpora generated by heterogenous models, (2) cheaper to generate EC corpora from a cheaper model. But I still really want to see this comparison.\n    - I don't think this is a fatal weakness, and perhaps it illustrates follow up experiments that this work inspires, which is to vary the architecture/capability of the EC agents and seeing what kinds of agents produce EC corpora that are more useful for transfer, and how those EC corpora differ using the various tools we have for analyzing emergent languages (e.g. topographic similarity).\n    - In fact, I think this experiment is nonsensical and misleading enough (every other line in the table uses a Transformer), that, without rerunning modified experiments, I would encourage authors to just remove this number entirely.\n- Limited evaluation of non-EC pretraining for image captioning.\n    - The experimental evaluation is overall quite comprehensive, except in the image captioning studies. Authors say \"We note that such a transfer scheme cannot work for other synthetic corpora such as music, code, or paren-zipf as they are not grounded on perceptual stimuli.\" While these other corpora indeed are not grounded in perceptual stimuli, you could still imagine pretraining on the ungrounded data, then finetuning while introducing grounding, or even arbitrarily associating different strings from a corpus with images. I would really like to see this comparison, as part of the strengths of Sec 3.1 are in the careful comparison of EC pretraining w/ other choices of pretraining corpora.\n- Results on permuted EC corpora could be explained further\n    - One outstanding question-issue I have with this paper is the results on permuted EC corpora. As I understand it, this is for the ungrounded language modeling task for ro and he languages. The permuted EC corpus does not seem substantially worse than EC pretraining (e.g. 195 vs 211 perplexity is not a huge difference compared with 211 vs 266), so I'm somewhat suspicious of the author's claim that \"speaker develops a coding scheme with sentence structures beyond bag of words.\"  A significance test would be helpful to measure the effect here. But regardless, it's clear that training on permuted EC results in impressive gains. Then what benefit does the EC corpus actually provide? The permutation experiments show that it's not syntax/sequential structure. And it's not learning a grounding between objects in images and emergent language tokens, since these numbers are for the ungrounded LM task. Maybe it's just token colocation statistics then?\n    - As an aside, I would really love to see the same experiments made with the image captioning task as well (e.g. even if the language ignores sequential structure, the bag of words annotation of relevant objects/features in the grounded input that the permuted EC corpus provides may be just as good as the original EC corpus)\n- Connections to related work.\n    - There have been a few attempts ta combining emergent communication and supervision, e.g. via multitask training ([Lowe et al., 2021](https://arxiv.org/abs/2002.01093), also [Lazaridou et al., 2020](https://arxiv.org/abs/2005.07064)) which are missing from related work. The connection could be made more clear. Why might we expect corpus transfer to do better or worse than multitask training? Perhaps these methods could be combined, e.g. multitask training on EC corpus and real corpus?\n\n# Questions/Minor\n\n- How does changing the vocabulary size and bandwidth of the EC pretraining corpus change its utility for pretraining? There is surely a point at which EC corpora are too simple (e.g., imagine just a single token and a very small vocab). Would be interesting to quantify the benefits gained not just by varying corpus size, but by varying emergent language complexity. Figure 3 partially answers this question (in that untrained EC corpora are not helpful for transfer), but I'd be interested, for example, in seeing a plot of (V, T) and transfer efficacy.\n- One of the central issues in studies of emergent communication is that agents often develop non-compositional, unintuitive communication protocols. The EC setup described in 2.1 doesn't seem to have any of the guards against degenerate languages that more modern studies of EC have investigated, e.g. different ImageNet categories (Lazaridou et al., 2017) or SimCLR views ([Dessi et al., 2021](https://arxiv.org/abs/2106.04258)), or even other ways of regularizing a language (e.g. [Luna et al., 2020](https://arxiv.org/abs/2004.03868)) Yet the EC languages still seem useful for transfer. So questions are, (1) do we still see degeneracy in the languages used in the EC corpora, and (2) if so, does trying to reduce degeneracy/improve systematicity of the language result in better transfer performance?\n- Why are there no standard deviations for EC pretrain/from scratch in Table 2?\n- Table 3 - how many data points actually go into the computation of correlations here? Is it 5 x 4 = 20? I would really love to see the full scatterplots rather than just correlation, e.g. in the appendix, and to actually see how much variance we see in the different EC corpora, as measured by the metrics.\n- The details of the training steps expt in Figure 3 are a little underexplained, and I tried my best to reconstruct, though I invite the authors to clarify if I've misunderstood anything. You take checkpoints of the EC corpora generated from EC agents at 0, 200, 400, ... 1k training steps, and at each step measure the various EC langauge metrics, and also do the pretrain/finetune experiment with transformers, starting with the EC corpora, to get plot d - perplexity. Is that correct?", "summary_of_the_paper": "This paper proposes a new way of using techniques from the emergent communication literature, where agents are trained to develop languages for communication on some shared task, to improve more typical supervised learning tasks in NLP, namely language modeling and image captioning. This is a question that is only just beginning to be addressed in some work (e.g. Lowe et al., 2021; Lazaridou et al., 2020), so this area is ripe for exploration. This paper proposes an interesting and novel method of using EC to improve supervised learning: generate emergent language from an image-based referential game with simple speaker/listener models. Then, using a much larger transformer, pretrain on this corpus before doing fine-tuning on the supervised task of interest, a la Papadimitriou and Jurfasky, 2020.\n\nBoth language modeling and image captioning tasks are explored, and results convincingly show that emergent communication corpora are surprisingly effective. Some ablation studies help elucidate where the benefits arise, although there are some questions here (see Weaknesses).\n\nFinally the authors propose a new metric for evaluating the naturalness and usefulness of a corpus of emergent language: if you have natural language messages generated for some task, as well as true human messages, the authors literally just build a model for translating emergent languages into the corresponding human messages. The ROUGE score obtained by such a model is thus a \"transferability\" metric. This metric is not only useful for identifying whether an EC corpus will be useful for transfer for some task (more similar = more useful), but is possibly also a measure of humanness/compositionaliy of a language in general, and so this contribution will be useful even for those in EC who are not interested in supervised learning.\n\nOverall, I very much like the philosophy of this paper. I think it helps fill an important gap in the literature, which is namely what to do with the recent deluge of emergent communication studies. The big elephant in the room with these studies is whether the languages are actually useful for some reason besides being studies of linguistic evolution with limited generality. This paper proposes a simple and interesting way of using the emergent languages for something more productive and more of interest to people actually working on real NLP applications. I do however see some weaknesses and lack of comparison which I would love to see answered and/or addressed in the final version.", "summary_of_the_review": "Overall, I think this is a creative, technically sound, and very interesting paper with valuable contributions to the emergent communication literature (notably, the big question of how to make EC useful for the rest of the community), and the paper really made me think. It's a shame that there are some unanswered questions remaining in the paper (see Weaknesses). I hope the authors might consider some of these follow-up questions and experiments during the rebuttal phase, but I think even despite these flaws, the paper has enough contributions to stand on its own. If all of the issues in the weaknesses section are addressed I would consider raising my score to a 10.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "3wZWFxBX_pz", "reviewer": "Reviewer_gtt4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Summary\n\nThe authors propose to use corpora generated from _emergent communication_ as a fine-tuning signal for NLP tasks (language modeling and image captioning in particular).  They show that, especially in small-data regimes, pre-training on an emergent language can yield significant performance boosts in both tasks.  (In particular, pre-training on emergent language performs on average better than synthetic hierarchical data, but not quite as well as a different natural language, and all of these pre-training methods do better than training from scratch.)  This is the newest of a small but growing body of literature that seeks to connect emergent communication with genuine NLP tasks.  The results are intriguing and promising and should be of interest both to the emergent communication community as well as to the broader community working on low-resource NLP.\n\nStrengths:\n* Novel method of using emergent language for pre-training (as opposed to transferring an entire artificial agent)\n* Some good ablations to identify what contributes to successful transfer\n* A new evaluation metric (emergent --> NL translation performance) that best correlates with fine-tuning performance\n\nWeaknesses:\n* Some parameter choices and the design of some ablations are not completely justified\n* Some additional related works could be included\n\n\n# Minor comments / questions\n\n* \"However, this metric is too rigid in its definition of compositionality, ignoring aspects like argument structure, context or morphology which play a key role in determining the combination of word semantics (Goldberg, 2015).\"  Steinert-Threlkeld (2020) \"Towards the Emergence of Non-trivial Compositionality\" makes similar points and could be cited here as well.\n\n* Why are the lines for \"from scratch\" flat in Figure 2?  I would have thought this was training an LM on progressively larger portions of the relevant data (being used for fine-tuning the others), in which case I'd also expect a downward trend in perplexity.  Are these rather the ppl resulting from training an LM on the full dataset?  It should be stated more clearly in the text what this means.\n\n* Why does the EC pre-training use |V| = 4035, as opposed to 50 that's used in the other tasks and the fine-tuning corpora?  Do the authors expect better results with a larger vocab size?\n\n* Ablation: the model vs corpus transfer comparison seems unfair to me.  In particular, you are comparing a small GRU LM to a larger transformer LM, where the latter is, as you mention, a much more powerful model.  While it is also true that corpus transfer _enables_ this divergence in model types, to really test whether transferring the whole model versus using the corpus works or not, I would think you would want to compare fine-tuning the sender GRU on the LM data vs. starting from scratch _with a GRU of the same type_ and pre-training on the EC corpus before fine-tuning.  Did you do an experiment of that type?\n\n* Related work: I would also include a discussion of this Lazaridou et al paper where they compare ways of combining EC with non-EC learning signals (e.g. image caption training): https://aclanthology.org/2020.acl-main.685.pdf\n\n* Ethics statement: I appreciate this statement and agree with the possible positive impacts.  Do the authors see any potential negative impacts?  If not, that should also be explicitly stated.\n\n\n# Typographic comments\n\n* p 1, \"the input out of detractors\" --> \"the input out of distractors\"\n \n* p 2: \"transferable benefits for downstream natural language tasks\" the single hyphens surrounding the subsequent list should be em dashes (three hyphens in TeX)\n\n* p 3: \"uses another GRU layer to decode the message m into a hidden vector hl\" I would use \"encode\" instead of \"decode\" here, since text-->representation is usually what an encoder does\n\n* p 3: \"The most straightforward metric is the accuracy of playing the referential game p(guess = Ii).\"  Given the way p(guess=Ii) is used above, I think this should be more like E[argmax(p(guess=Ii)) = i].  Or if they are measuring the probability assigned to the true image and not just accuracy, the name shoudl be changed from accuracy.\n\n* p 4: \"es should set a upper bound\" --> \"es should set an upper bound\"\n\n* p 5: \" ec perform better than or \" --> \" ec performs better than or \"", "summary_of_the_paper": "The authors propose to use corpora generated from _emergent communication_ as a fine-tuning signal for NLP tasks (language modeling and image captioning in particular).  They show that, especially in small-data regimes, pre-training on an emergent language can yield significant performance boosts in both tasks.  (In particular, pre-training on emergent language performs on average better than synthetic hierarchical data, but not quite as well as a different natural language, and all of these pre-training methods do better than training from scratch.)  This is the newest of a small but growing body of literature that seeks to connect emergent communication with genuine NLP tasks.  The results are intriguing and promising and should be of interest both to the emergent communication community as well as to the broader community working on low-resource NLP.", "summary_of_the_review": "The paper presents a novel method and evaluation of using emergent communication for pre-training models on real NLP tasks.  This has the potential to help in low-resource settings, and demonstrates the value of emergent communication for NLP in addition to theoretical interest in language evolution.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores ways in which *emergent communication* (EC) methods from representation learning can be evaluated extrinsically, by hooking them into downstream NLP tasks. Reviewers agree that the paper is thorough, and finds encouraging results.

This paper is borderline, and difficult to evaluate, even after very substantial discussion (some of it private). From my reading of the reviews and pieces of the paper, I'm very sympathetic to wvqW's concern that none of the present-day applications under study seem likely to benefit from this kind of emergent communication pretraining: *Natural* language pretraining, even transferring across natural languages, is for too strong a baseline, and it's not even conceptually clear how one could substantially outperform that baseline. I'm very concerned that the results in this paper will be—misleadingly—cited as proof that EC research is already contributing to downstream progress in NLP.

However, the narrow claims in the paper itself seem to be sound, and two confident reviewers whom I trust argue strongly that the ideas results here are surprising and novel, and that the paper could be the starting point for productive discussion and future work in this area. I'm recommending spotlight presentation in the hope that the paper will provoke a nuanced discussion in that setting.
Decision: Accept (Spotlight)