Interpretable agent communication from scratch (with a generic visual processor emerging on the side)
Authors: Roberto Dessi
Eugene Kharitonov
Marco Baroni
Venue: NeurIPS 2021 Poster (2021)
Keywords: emergent communication
self-supervised learning
interpretability
multi-agent
deep neural networks

Abstract: As deep networks begin to be deployed as autonomous agents, the issue of how they can communicate with each other becomes important. Here, we train two deep nets from scratch to perform realistic referent identification through unsupervised emergent communication. We show that the largely interpretable emergent protocol allows the nets to successfully communicate even about object types they did not see at training time. The visual representations induced as a by-product of our training regime, moreover, show comparable quality, when re-used as generic visual features, to a recent self-supervised learning model. Our results provide concrete evidence of the viability of (interpretable) emergent deep net communication in a more realistic scenario than previously considered, as well as establishing an intriguing link between this field and self-supervised visual learning.

Interpretable agent communication from scratch (with a generic visual processor emerging on the side)
Roberto Dessi Facebook AI Research Universitat Pompeu Fabra rdessi cetssi@fb.com com
Eugene Kharitonov Facebook Al Research khari thavvfo.. com
Marco Baroni Facebook AI Research Universitat Pompeu Fabra ICREA mbaroni@fb.. com
Abstract
As deep networks begin to deployed as autonomous agents, the issue of how they can communicate with each other becomes important. Here, we train two deep nets from scratch to perform large-scale referent identification through un- supervised emergent communication. We show that the partially interpretable emergent protocol allows nets to successfully communicate even about object classes they did not see at training time. The visual representations induced as a by-product of our training regime, morcover, when re-used as generic visual features, show comparable quality to recent self-supervised learning model. Our results provide concrete evidence of the viability of (interpretable) emergent deep net communication in more realistic scenario than previously considered, well as establishing an intriguing link between this field and self-supervised visual learning.
Introduction
As deep networks become more effective at solving specialized tasks, there has been interest in letting them develop language-like communication protocol so that they flexibly interact address joint tasks [I] One line of work within this tradition focused on what is arguably the most basic function of language, namely to point out, refer to, objects through discrete symbols. Such ability would for example allow deep-net-controoleed agents, such as self-driving cars, to inform each other about the presence and nature of potentially dangerous objects, besides being basic requirement to support more advanced capabilities (c.g., denoting relations between objects).
While discreteness is not necessary prerequisite for agent communication 3 practical and ethical problems might arise if communication incomprehensible humans. discrete code analogous to language is certainly easier to decode for us, helping us to understand the agents' decisions, and ultimately contributing to the larger goal of explainable AI
In this paper, we study emergent discrete referential communication between two deep network agents that trained from scratch the task. We observe that the referential discrimination task played by the networks is closely related to pretext contrastive objectives used in self-supervised visual representation learning (5. We exploit this insight to develop a robust end-to-end variant of communication game. Our experiments confirm that, in our setup: i) the nets develop set of discrete symbols allowing them successfully discriminate objects in natural images, including novel ones that were not shown during training; ii) these symbols denote partially interpretable categories, so that their emergence can be seen a first step towards fully unsupervised image annotation; iii) the visual representations induced as a by-product can be used as high high-quality general general-purposss purpose features,
'Code: htt.d///peee...k.ooo.ooooooottte
_s__ssl
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

whose performance in various object classification tasks is not lagging much behind that of features induced by a popular sule-supe representation method specifically designed for this task.
2
Background
Deep net emergent communication There has recently been interest in letting deep nets com- municate through learned protocols. This line of work has addressed various challenges, such as communication in a dynamic environment how interface the emergent protocol with natural language (see [T] for a survey). Probably the most widely studied aspect of emergent communication is the ability of deep net agents use the protocol to refer objects in their environment [e.g.. 16)
The typical setup is that of a referential, or discriminative, communication game, In the simplest scenario, which we adopt here, an agent, the Sender, sees one input (the target) and it sends a discrete symbol to another agent, the Receiver, that sees an array of items including the target, and has to point to the latter for communication to be deemed successful. Importantly, task success is the only training objective; the communication protocol emerges purely as a by-product of game-playing without any direct supervision on the symbol- trabbbl-tannmiision channel.
In one of the earliest papers in this line of research, Lazaridou et al. used images from ImageNet [17] as input to the discrimination game; Havrylov and Titov [1]] used MSCOCO [1]] and Evtimova et al. used animal images from Flickr. While they relied on natural images, all these studies were limited to small sets of carefully selected object categories. Moreover, in all these works, the agents processed images with convolutional networks pretrained on supervised object recognition. While this sped up learning, it also meant that all the proposed systems de facto relied on the large amount of human annotated data used for object recognition training. Lazaridou et al. [3] and Choi et al. [14] dispensed with pre-trained CNNs, but they used synthetically generated geometric shapes as inputs.
Results on the interpretability of symbols in games with realistic inputs have generally been mixed. Indeed, Bouchacourt and Baroni 19 showed that, after training Lazaridou et al. 's networks on real pictures, the networks could use the learned protocol to successfully communicate about blobs of Gaussian noise, suggesting that their code (also) denoted low-level image features, differently from the general semantic categories that words in human language refer to. In part for this reason, recent work tends to focus on controlled symbolic inputs, where it is easier to detect degenerate communication strategies [e.g.. [10 15 [6].
Learning discrete latent representations with variational autoencoders Like emergent commu- nication, variational autoencoders and related techniques have been used to induce discrete variables without direct supervision (2).22). There are however important differences stemming from the fact that this research line is interested in inducing latent representations to be used in tasks such image generation, whereas the goal of emergent communication is to induce discrete symbols for agent coordination. As result, for example, discrete representations derived with variational autoencoders are typically of much higher dimensionality, making a direct comparison in terms of interpretability diffieult. Still, it is remarkable that the goal of learning discrete representations in unsupervised way independently arose in different fields, and future work should explore connections between these ideas.
Self-supervised representation learning Self-supervised learning of general-purpose visual fea- tures has received much attention in recent years. The main idea is to train network on pretext task that does not require manual annotation. After convergence, the net is used to extract high-quality features from images, be applied in various "downstream" tasks of interest. This is often done by training a simple classifier on top of the frozen trained architecture 263.
Early models used image-patch prediction as the proxy task 8 27
Recent work has instead focused
on an instance-level contrastive discrimination objective 7 28 Two symmetric networks encode
different views of the same input images obtained through stochastic data augmentation pipeline. Optimization is done with variants of the InfoNCE loss (27 29). that maximizes similarity among representations of the same image while minimizing similarity of different ones.
Interestingly, the contrastive pretext task very close to the one of identifying a target image among distractors, as in the standard emergent communication referent discrimination game. The

---

2
3
SENDER
RECEIVER
symbol499
IMAGE SCORER
GNN
SYMBOL GENERATOR
SYMBOL
Target
Candidates
2
3
Figure 1: Game setup and agent architectures. Image sources: https: pttpps:/unsppash....
com com
influential SimCLR model proposed by Chen al. [ is particularly similar our setup. It uses two twin networks with shared convolutional module optimizing the (diilsimil of sets of target/distractor images. The main conceptual differences are that there no discrete bottleneck imposed on "communication" between the networks, and there is no asymmetry, so that both networks act simultaneously as Sender and Receiver (both networks produce continuous "message" that must be discriminative for the other network as possible).
We rely here on the connection with self-supervised learning two ways. First, we import the idea of data augmentation from this literature into the communication game, showing how it helps in evolving more semantically interpretable protocol. Second, evaluate the discrimination game as a self-supervised feature extraction method. We find that the visual features induced by the CNNs embedded in our agents are virtually as good as those induced by SimCLR, while the emergent protocol is better for communication than the one obtained by adapting SimCLR to the discrete communication setup.
Setup
3.1
The discrimination game
Sender network receives as input a targer picture, and produces as output one of [V] symbols. Receiver network receives in input this symbol, as well list of n pictures, one of them (randomly placed the ith position the list) being the same target presented to the Sender. Receiver produces probability distribution of cardinality interpreted as its guess over the position of the target. The guess is correct iff Receiver concentrates the largest probability mass on the ith position, corresponding to the target slot.
Agent architecture Agent architecture and game flow are schematically shown in Fig. ] Sender reads the target image through convolutional module, followed by one-layer network mapping the output the CNN onto V dimensions and applying batch normalization [30], obtain vector Following common practice when optimizing through discrete bottlenecks, then compute the Humbel-Softman continuous relaxation B2 which was shown to also effective in the emergent communication setup I1. At train time, Sender produces approximation
one-hot symbol vector with each component given by m, Σ, exp [(,, +t.)7) where s; random sample from Gumbel(0.1 1) and dimension of v. The approximation is controlled by temperature parameter as T approaches O, the approximation approaches a one-hot vector, and as approaches +o0, the relaxation becomes closer uniform. Importantly, at test time the Sender's output is generated by directly argmax-ing v, so that it is discrete one-hot vector indexing one of [V] possible symbols.
exp exp((s=t)))/71
Receiver passes each input image through its visual module (a CNN architecture), followed by a two-layer MI.P with batch normalization and ReLU after the first layer (33] then computes
temperature-wwiigteed cosine scores for the linearly embedded symbol compared to each image

---

It is important to stress the different roles that SimCLR comparison model will play in the experiments below. When playing the communication game (Section 4.1 our discretized Sim- CLR method must be simply seen as an interesting baseline, the system was not designed for discrete communication the first place (it is indeed interesting that it performs as well as it does). When performing protocol analysis (Section 4.2), the discrete clusters explicitly built SimCLR representations act as a challenging comparison point for the categories implicitly induced by our system through game playing. Finally, when evaluating the visual features learnt by the two systems (Section 4.3), the roles are inverted, with SimCLR being a standard method developed for these by-product of representation. The resulting vector of cross- modal (symbol-image) similarities is transformed into a probability distribution over which image is the likely target by applying the softmax operation.
For both Sender and Receiver, we use ResNet-50 34 as visual module. As they are different agents, that could (in future experiments) have very different architectures and interact with further agents, the most natural assumption is that each of them does visual processing with its own CNN (no weight sharing). We consider however also a setup in which the CNN module is shared (closer to earlier emergent-commmnicatto work, where the agents relied on the same pre-trained CNN).
Optimization Optimization is performed end-to-end and the error signal, backpropagated through Receiver and Sender, is computed using the cross-entropy cost function by comparing the Receiver's output with a one-hot vector representing the position of the target in the image list.
SimCLR as comparison model Given the similarity between the referential communication game and contrastive self-supervised learning in SimCLR [5], we use the latter comparison point for our approach. Fig. 2iscemaattttalllly shows the SimCLR architecture. The crucial differences between SimCLR and our communication game are the following: i) In SimCLR, the agents are parameterized by the same network, that is, the visual encoder and transformation modules in the two branches of Fig. 2 are instances of the same net. ii) The setup is fully symmetric. Like our Receiver, both agents get a set of images in input, and, like our Sender, both agents can be seen as producing "messages" representing each input image. iii) Instead of (a probability distribution over) symbols, the exchanged information takes the form of continuous vectors (s in the figure). iv) The loss is based directly comparing embeddings of these continuous vectors (z in the figure), maximizing the similarity between pairs representing the same images (positive examples in contrastive-loss terminology) and minimizing that of pairs representing different images (negative examples). This differs from our loss, that maximizes the similarity of the Receiver embedding of the Sender-produced discrete symbol with its own representation of the target image, while minimizing the similarity of the symbol embedding with its representation of the distractors.
purposes, whereas in our setup the emergence of good visual representations communication-be based model training.
linear 2_P
visual encoder
linear + relu
egg
similarity baaeed alignment
BU0 q
visual encoder
linear + relu
linear
h_g
z_9
Figure 2: The SimCLR architecture. aug_p and aug_g are outputs of the stochastic augmentation pipeline used to generate two views of the same image.
Data augmentation In the original discrimination game proposed in [9, the agents are shown exactly the same target image Pl In self-supervised learning, on the other hand, it is common practice to "augment" images in different ways, e.g., by applying different croppings or color perturbations
LLazaridou and colleagues 9 also considered variant of the game in which the agents different pictures of the same category (e.g.. the shared target is dog, but the agents get different dog pictures). This version of the game is however severely limited by the requirement of manual category annotation. Lazaridou et al. [I3] also provide different images to Sender and Receiver, by feeding them different viewpoints of the same synthetically generated objects: again, strategy that will not scale up to natural images.

---

B5 36 In standard contrastive learning frameworks, where all the weights are shared and there is no communication bottleneck, is necessary to create these different views, or else the system would trivially succeed at the pretext contrastive task without any actual feature learning. We conjecture that data augmentation, while not strictly needed, might also be beneficial in the communication game setup: presenting different views of the target to Sender and Receiver should make it harder for them to adopt degenerate strategics based on low-level image information [19]. We follow the same data augmentation pipeline as [5], stochastically applying crop-and-resize, resize, color perturbation, and random Gaussian blurring to every image. Note that, for the experiments reported in the main paper, we do not apply data augmentation at test time. Results with augmentation also applied at test time are reported in Appendix A.4.
Implementation details All hidden and output layers are set to dimensionality 2048 Note that this implies [V] = 2048, more than double the categories in the dataset we use to train the model (see f Section 3.2 below), to avoid implicit supervision on optimal symbol count We fix Gumbel-Softman temperature at 5.0, and Receiver cosine temperature at 0.1. The latter value is also used for the equivalent parameter in the NTXent-loss of our SimCLR implementation.
We train with mixed precision 37 for 100 epochs, with a batch of size 16 x 128 = 2048, divided across 16 GPUs. Rather than sampling distractors from the entire dataset, current device's 's batch, thus playing the communication game with 127 distractor images in all reported experiments We do not share distractors (negative samples) across devices. As in SimCLR, we use the LARS optimizer [38] with linear scaling [39], resulting in an initial learning rate of 2.4. We apply a cosine decay schedule without warmup nor restart [40.. Compute requirements are reported in Appendix A.1. All models are implemented with the EGG toolkit 41
take them from the
3.2 Data
Referential game Training targets and distractors are sampled from the ILSVRC-2012 training set containing 1.3M natural images from 1K distinct categories. We use two image sources for testing. First, we use the ILSVRC-2012 validation set, containing around 50K images from the same categories as the training data. Second, in order to probe the generality of the emergent protocol, we introduce a new ouutof-diitribution" dataset (henceforth, the OOD set). To build the latter, we relied on the whole ImageNet database [17,, exploiting its WordNet-derived hierarchy [43] In particular, we randomly picked (and manually sanity-checked. 80 categories that were neither in ILSVRC-2012 nor hypernyms or hyponyms of ILSVR categories (e.g., since hamster is in ILSVRC-2012, we avoided both rodent and golden hamster). We also attempted to sample categories of comparable degree of generality to those in 1 For each of the categories chosen according to these criteria, we randomly sampled 128 images from ImageNet. Examples of included categories are eucalyptus, yellowtail, and drawer. The OOD set categories are not tremendously different from the ones in 1LSVRC-2012, belonging to similar high-level domains, such as plants, fish and furniture. This is on purpose. As our agents are limited to single-symbol communication, we do not expect them to be able to denote completely unrelated test categories by compositional means. Rather, the function of the OOD set is to check that the symbols of emergent protocol do not overfit the very specific classes of the training set, but are general enough to be usable for somewhat related categories (c.g., that symbols developed for other training-sod set fish categories can also denote yellowtail)
[2].
Linear evaluation of visual features on downstream tasks Following standard practice in self- supervised learning [c.g.. 15 23 44, we evaluate the visual features induced by the CNN components of our models by training linear object classifier on top of them. We use four common data sets:
"This is the same size used in the original SimCLR paper, except for the nonlinear projection head. For the latter, number of sizes were tested, and the authors report that they do not impact final performance. We 2048 for direct comparability with our setting.
'Results with different vocabulary sizes are reported in Appendix A.6
$ RRssults with different numbers distractors are reported in Appendix A.5.
'Paths to the ImageNet images in the OOD set and the corresponding categories are available at https: http///ccommeceeeeeeeeeeeeeeeeee as__a_____________ee_e_e ass_sss000___e___ txt
//gi //gtthub. http:///..abbbbeeeeee////////o/e
5

---

ILSVRC-2012, Places205 455, 1Naturalis22018 and VOCO7 Evaluation is carried out with the VISSL toolkit [46] adopting the hyperparameters in its configuration files without changes.
Experiments
4.1 Referential communication accuracy
We start by analyzing how well our models learn to refer to object-depicting images through learned protocol As an interesting baseline, let the trained SimCLR model play the referential game by argmax-ing its layer into discrete "symbol" (SimCLRaise). By looking at the SimCLR diagram Fig. 2 should be clear that constitutes the equivalent of the communication layer, with functioning as symbol embedding layer. Discretizing h, in any case, led to lower game-playing performance
Accuracy is given by the proportion of times in which the Receiver "picks the right image", that is, assigns the largest hambbboommbbcddiigggagggerrsssss distractors (chance = 0.8%).
similarity the target compared 127
Results in Table The ILSVRC-val column shows that all models can play the game well above chance when tested on new images of the same categories encountered during training. The next column (OOD set) shows that the models also play the game well above chance with input images from new categories, although mostly with a drop in performance. All variants of our model are considerably more robust than the SimCLR baseline (which, however, does remarkably well at this discrete communication game it was not designed for).
ILSVRC-val OOD set
Gaussian Blobs
SimCLR
56.9%
47.4%
0.8%
Communication Game
-augmentations -shared
91.2%
90.8%
43.4%
-augmentations +shared
92.8%
92.7%
84.7%
+augmentations -shared
81.5%
72.0%
0.8%
+augmentations +shared
82.2%
73.7%
0.8%
Table 1: Game-playing accuracy. = augmentations marks whether the game was trained with data augmentations not. + shared indicates whether there was CNN weight sharing or not between Sender and Receiver.
Looking at model variants, sharing CNN weights or not makes little difference (an encouraging first step towards communication between widely differing agents, that will obviously not be able to share weights). On the other hand, data augmentations apparently harm performance. However, turns out that the better performance of the non-augmented models is due to opaque communication strategy in which the agents are referring to low-level aspects of images (perhaps, specific pixel intensity levels?), and not the high-level semantic information they contain (ideally, object categories).
To show this, we replicated the sanity check from 19 play the communication game with blobs of Gaussian noise targets and distractors. We use 384 batches of 128 224x224-sized random images whose pixels are drawn from the standard Gaussian distribution N 1), for total of 49152 items (a size comparable to that of ILSVRR-val), Results are in the last column of Table [ The +angmentations models and SimCLR fully pass the sanity check, with performance exactly the 0.8% chance level. The -augeentation models, the other
We freeze the trained models and let them
http::/placcs..co ceail. wataal..i.... mit. wwweedddddesseeeeeeeeeee html http://host.rooee ox. ex.aa ac
https://visal... ai/
AAppendix A.2 reports this and all following experiments repeated with 5 distinct initializations of our most representative model (*augmentations -shared). It shows that variance across runs is negligible. We did not repeat the check for the remaining models due to time and resource constraints (see Appendix A.1).
We also compared to an architecture from earlier work in emergent communication with realistic visual inputs, that is, the one of [9 [9]. However, as shown in Appendix A.3, its performance is considerably lower than that of SimCLR

---

hand, are able to use the symbols they learned from sane input to communicate about the Gaussian blobs, showing that they developed an opaque protocol. The -augmentations +shared model. in particular, is hardly affected by the switch to noise data.
4.2 Protocol analysis: emergent communication as unsupervised image annotation
The Gaussian blob test suggests that the +augmentations models do not fall into the trap of degenerate low-level protocol. However, it is not sufficient to conclude that they learned to associate symbols with human-meaningful referents. To test whether this is the case, we exploit the fact that, as we are working with ImageNet data, we have labels denoting the objects depicted in the images. We use this information in two ways. We compute the normalized mutual information (nMI) between the ground-truth labels of target images and the symbols produced for the same images by the trained Sender. The nMI of two variables obtained by dividing their MI by their average entropy, and ranges between O and 1. We also compute a normalized similarity measure based on the shortest path between two categories in the WordNet is-a taxonomy. Our WNsim score is the average shortest- shortest-path similarity of the ground-trum -ruth categories of all target pairs that share the same Sender symbol, and it also ranges between and 1. WNSim is more nuanced than nMI, it will penalize less a Sender using the same symbol for similar categories (e.g., cats and dogs) than one using the same symbol for dissimilar ones (cats and skyscrapers). WNsim is computed with NLTK 47
We again take SimCLRaise as a comparison point, where a "symbol" is simply the dimension with the largest value on a certain layer. To give this approach its best chance. we evaluated the h, s and layers (see Fig. 2). and report statistics for & (CNN output), as it produced the best overall scores across data sets. We also run k-means clustering on the h layer (SimCLRemmann.). Cluster centroids were estimated on a random 10% of the training set, and then used to group the test images into clusters (treated as equivalents the symbols produced by our systems). We tried clustering with k = 1000 (ground- -rround-truth truth class cardinality) and k 2048 (same as vocabulary size of our models). We report the significantly better results we obtained with the second choice.
ILSvRC-val
OOD set nMI
|P|
nMI
WNsim
|P|
WNsim
SimCLR SimCLR kmeane
1489
0.49
0.11
1069
0.46
0.19
2035
0.59
0.18
1519
0.54
0.30
Communication Game
-augmentations -shared
2044
0.50
0.08
1921
0.45
0.11
-augmentations +shared
2048
NS
NS
2025
NS
NS
+augmentations -shared
2042
0.58
0.18
1752
0.53
0.32
+augmentations +shared
2046
0.56
0.15
1765
0.51
0.25
Table 2: Protocol analysis. IPI is the observed protocol size, that is, the number of distinct symbols
actually used at test time. We mark as NS the cases where the obtained scores were not significantly different from chance according to permutation test with 0.01.
Looking at Table 2 we first observe that, consistent with the Gaussian blob sanity check, there is no significant sign of symbol-category association for the -augmentations +shared protocol. All other models show some degree of symbol interpretability (with significantly above-chance nMI and WNsim scores). Even when there is no data augmentation during training. using different visual modules (-augmentations -shared) leads to some protocol interpretability. coherently with the fact that this configuration was less able than its +shared counterpart to communicate about noise. We were morcover surprised to find that simply argmaxing the SimCLR visual feature layer produces meaningful "symbols", which suggests that information might be more sparsely encoded by this model than one could naively assume.
Importantly, the game protocols derived with data augmentation have particularly high nMI and WNsim scores. Impressively, the scores achieved by our models, and the +augmentations -shared setup in particular, are very close to those obtained by clustering SimCLR visual features. Recall that, unlike our models, whose protocol independently emerges during discriminative game training, SimCLRemmans runs clustering algorithm on top of the representations produced by the SimCLR
Appendix A.7 further reports analysis of symbol frequency distributions for representative models.
7

---

Symbol
Symbo 2
Sumbol
9CSS
Syrbol
Symbol 5
Pemboll
Symbot
Symool
Syntas 9
Figure 3: Randomly selected ILSVRC-val images triggering the +augmentations -shared Sender to produce its most frequent symbols.
Symook
Symoot
Figure 4: Randomly selected OOD set images triggering the +augmentations -shared Sender to produce the 3 most frequent VLSl symbols (cf. Fig. 3).
visual encoder with the express goal to discretize them into coherent sets, thus constituting a hard competitor reach.
Beyond the quantitative results, a sense how good our symbols are as unsupervised image labels can be gauged by qualitatively inspecting images sharing the same assigned symbol. Fig. shows a random set of such images for the symbols most frequently produced by the +augmentations -shared Sender in ILSVRC-val, without hand-picking [2 Some symbols denote intuitive categories, although, interestingly, ones that do not correspond to specific English words (birds on branches, dogs indoors...). Other sets are harder characterize, but they still share clear high-level "family resemblance" (Symbol 2: objects that glow in the dark: Symbol 3: human artifacts with simple flat shapes, etc). Frequency imbalance in input super-categories, together with the fact that the agents allowed to use a large number of symbols, leads to partially overlapping clusters (Symbol 9 might denote living things the grass, whereas Symbol seems to specifically refer to mammals in the grass).
The fact that symbols do not exactly denote ILSVRC categories plays the agents' advantage when they must communicate about OOD images. While this set, by construction, does not contain ILSVRC categories, Fig. 4 shows, it still contains birds tree, glowing objects and artifacts with
We excluded 25% of symbol-5 images before sampling, they depicted people. Consistent with this symbol's "theme", the latter mostly show people with dark backgrounds.

---

CLSv
Places205
INaturalis20018
VOCO7
Supervised SimCLR
76.5%
53.2%
46.7%
87.5%
60.6%
49.0%
31.8%
78.7%
Communication Game
+augmentations -shared
59.0% 60.2%
47.9% 49.1%
30.8% 31.3%
77.0% 78.8%
+augmentations +shared
Table 3: Linear evaluation object classification. Reported scores are mAP for VOCO7, top-1 accuracy elsewhere. Supervised results are from 24
flat shapes, so that the agents can successfully refer to them by using the "spurious" but interpretable symbols denoting these concepts that they induced from ILSVRC.
4.3 Downstream object classification: emergent communication self-supervised visual feature learning
Finally, we evaluate the features produced by the Sender CNN trained on the communication game as out-of-the-box visual representations. We follow the standard protocol for training linear classifier on the output of the frozen CNN trunk on various object classification data sets (see Section 3.2 above). We focus on Sender because the features produced by the two agent networks are always highly correlated [13 We further exclude the -augmentations models that, having learned a degenerate strategy, reach extremely poor classification performance on ILSVRC-val (below 5% accuracy).
As reasonable upper bound, Table 3 reports the fully-supervised object classification results from [24]. As more direct point of comparison, we also report the performance of our SimCLR implementation The table shows that the features developed game are of comparable quality to those of SimCLR, method developed specifically for visual feature learning. This is extremely promising first step towards employing emergent communication as form of self supervision. Many ideas from the self-supervised literature (c.g., new data augmentation pipelines, the use of memory banks for distractor sampling or variants of the similarity- based pretext task) could straightforwardly be integrated into our setup, hopefully leading to the emergence of even better visual features and, perhaps, an even more transparent protocol.
by-product of the communication
Conclusion
Deep agent coordination through communication has recently attracted considerable interest. Ref- erential games are natural environment to test the agents' emergent communication strategies. Past approaches. however, relied on relatively small image pools processed with pretrained visual networks, or artificial input. We showed here that deep agents can learn to refer to high number of categories depicted in large-scale image datasets, while communicating through discrete channel and developing their visual processing modules from scratch. Performance on referential games with two distinct test sets (one with categories not presented training), along with protocol analysis, shows that the agents' protocol is effective and partially interpretable.
key ingredient success was input data augmentation. We borrowed this idea from recent approaches to self-supervised visual learning. Conversely, we showed how the agents' visual networks emerging from discriminative game playing produce high-quality visual features. Further integration of emergent communication and self-supervised learning methods should be explored in the future.
Our work constitutes just a small step in the right direction, and has important limitations to be addressed in future work. Our agents communicate through single symbol, but the true expressive
Acrosss setups and data sets, the Sender/Receiver correlation between all pairwise visual representation similarities was never below 0.96.
"It is difficult to compare our SimCLR ILSVRC-val val performance precisely
reported in the original
paper since, coherently with the communication game setup, we use per-GPU batch size of 128 without sharing negatives across GPUs. By Jooking at the leftmost bars of Fig. 9 in (5), we note that the performance we report is within the range of their results for the number of training epochs (100).

---

power of human language comes from the infinite combinatorial possibilities offered by composing sequences of discrete units 48]. Allowing longer messages and probing whether this results the development of compositional code is our next priority. Additionally, while in our experiments distractors are selected at random, this is obviously not the case in real-life referential settings (dogs will tend to occur near other dogs or humans, rather than between whale and a space shuttle). Dealing with realistic category co-occurrence is thus another important future direction. Finally, although we argued that discrete protocol should be more interpretable than continuous one, and we provided preliminary quantitative and qualitative evidence that the agents' protocol is indeed reasonably transparent, whether the achieved degree of interpretability is good enough for human-in- on-the-loop scenarios remains to be experimentally investigated.
Much recent work in the field has moved towards theoretically-oriee oriented understanding of deep agent communication in symbolic and artificial setups. We went back instead to the original motivation behind the study of emergent language, as path towards the development of autonomous Als that can interact with each other realistic environment. While we are still far from real-life- deployable interpretable machine-machine interaction, we hope that our work will stimulate more studies pursuing this ambitious goal.
Acknowledgments
We would like thank the NeurlPS area chair and reviewers, well Gemma Boleda, Rahma Chaabouni, Emmanuel Chemla, Simone Conia and Lucas Weber for feedback; Priya Goyal for technical support with VISSL; Mathilde Caron and the participants in the EViL meeting, the FAIR EMEA-NLP meetup and the Trento/ haato///nttedamarreeeeee
(TAB) meeting for fruitful discussions. We also want thank Jade Copet for sharing an early version of the code used in this work.
Funding transparency statement
The authors did not receive any form of third-part party funding or support. They do not have any financial relationship with entities that could be considered broadly relevant the work, except those declared in their affiliations.
References
[1] Angeliki Lazaridou and Marco Baroni. Emergent multi-agent communication in the deep learning era. htttps:/arxiv.org//
02419 2020.
[2] Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. Learnnng multiagent communication with backpropagation. In Proceedings of NIPS, pages 2244 2252,, Barcelona, Spain, 2016.
[3] Nur Geffen Lan, Emmanuel Chemla, and Shane Keeinet--Thrlkell. On the Spontaneous Emergence Discrete and Compositional Signals. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4794-4800, Online, July 2020. Association for Computational Linguistics.
[4] Ning Xie, Gabrielle Ras, Marcel van Gerven, and Derek Doran. Explainable deep learning: A field guide for the uninitiated, https://arxiv.orr abtprg//bs//000.. 14545 2020.
[5] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple frame- work for contrastive learning of visual representations. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597-1607. PMLR, 13-18 Jul 2020.
[6] Kaiming He, Haoqi Fan. Yuxin Wu, Saining Xie. and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[7] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent
10

---

a new approach to self-supervised learning. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 21271-21284, Curran Associates, Inc., 2020.
[8] Carl Doersch, Abhinav Gupta, and Alexei A. Efros. Unsupervised visual representation learning by context prediction. In 2015 IEEE International Conference Computer Vision (ICCV), pages 1422-1430, 2015.
[9] Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. In Proceedings of ICLR Conference Track, Toulon, France, 2017. Published online: httpps//ppenrevlees.ntt.ggn Iot. cc/211111../// conference
[10] Satwik Kottur, José Moura, Stefan Lee, and Dhruv Batra. Natural language does not emerge 'naturally' in multi-agent dialog. In Proceedings of EMNLP, pages Denmark, 2017.
Copenhagen,
[11] Serhii Havrylov and Ivan Titov. Emergence of language with multi-agent games: Learning communicate with sequences of symbols. In Proceedings of NIPS, pages 2149-2159 Long Beach, CA, 2017.
[12] Katrina Evtimova, Andrew Drozdov, Douwe Kiela, and Kyunghyun Cho. Emergent communi- cation in a multi-modal, multi-step referential game. In Proceedings of ICLR Conference Track, Vancouver, Canada, 2018. Published online: https://ooennreview net get//rroppid-ICLn ccc/011//Cofeerence
[13] Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of linguistic communication from referential games with symbolic and pixel input. In Pro- ceedings of ICLR Conference Track, Vancouver, Canada, 2018. Published online: https //openreview. net/ group?id=ICLk http///.........////////e Connfernncee
[14] Edward Choi, Angeliki Lazaridou, and Nando Freitas. Compositional obverter communi- cation learning from raw visual input. In Proceedings of ICLR Conference Truck, Vancouver, Canada, 2018. Published online: https://openrevier. oag.pprrrriiiee.eetrrooooaaaaaaa cc/20188
Conference
[15] Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, and Marco Baroni. Anti-efficient encoding in emergent communication. In Proceedings of NeurIPS, Vancouver, Canada, 2019. Published online: htttss// wwwp..cc/ppe.... cpppscccpppeer/111
[16] Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Compositionality and generalization in emergent languages. In Proceedings of ACL, pages 4427 4427-4442, virtual conference, 2020.
[17] Jia Deng, Wei Dong, Richard Socher, Lia-Ji Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of CVPR, pages 248-255, Miami Beach, FL. 2009.
[18] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zimick. Microsoft COCO: Common objects in context. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision ECCV 2014, pages 740-755, Cham, 2014. Springer International Publishing.
[19] Diane Bouchacourt and Marco Baroni. How agents see things: On visual representations in an emergent language game. In Proceedings of EMNLP, pages 981-983 Brussels, Belgium, 2018.
[20] Jason Rolfe. Discrete variational autoencoders. In Proceedings of ICLR Conference Track, Toulon, France, 2017. Published online: https::/openreviiee wttperreviiweeeeeeeeeeeeeeeeeeee group 2017/conference
[21] Aaron van den Oord, Oriol Vinyals, and Koray Kavukeuoglu. Neural discrete representation learning. In Proceedings of NIPS, pages 6306-6315, Long Beach, CA, 2017.

---

[22] Ali Razavi, Aaron van den Oord, and Oriol Vinyals. Generating diverse high- sish-fideiity images with VA-V2 In Proceedings of NeurIPS, Vancouver, Canada, 2019, Published online: httpps:/pappers nips. cc/ wwtppppppe//000/e 2209
[23] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski. and Armand Joulin. Emerging properties in self-supervised vision transformers. arXiv preprint arXiv:2104. 14294, 2021.
[24] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 9912-9922299 2924... Curran Associates, Inc., 2020.
[25] Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisser- man. With little help from my friends: Nearest-neighbor contrastive learning of visual representations. CoRR, fiss21004145453 2021.
[26] Jure Zhontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. CoRR, abs/2103.03230, 2021.
[27] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. CoRR, abs/1 01800077788 2018.
[28] Xinlei Chen and Kaiming He. Exploring simple Siamese representation learning. CoRR, abs/201 1.10500 2020.
[29] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: new estimation principle for unnormalized statistical models. In Yee Whye Teh and Mike Titterington, editors, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages 297-304, Chia Laguna Resort, Sardinia, Italy, 13-15 May 2010. PMLR.
[30] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Leurning, volume 37 of Proceedings of Machine Learning Research, pages 448 456, Lille, France, 07-09 Jul 2015. PMLR.
[31] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparamcterization with Gunbel-Softmax, In Proceedings of ICLR Conference Track, Toulon, France, 2017. Published online: hhtpps //openreview.o net. gropp/id=IIIR... cc c012017 co0f77//ofeeecceeee
[32] Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv: 1611:00611.00011 2016.
[33] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of AISTATS, pages 315-323, Fort Lauderdale, FL, 2011.
[34] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Patlern Recognition (CVPR), pages 770-778, 2016
[35] Philip Bachman, Devon Hjelm, and William Buchwalter. Learning representations by maximiz- ing mutual information across views. In Proceedings of NeurIPS, Vancouver, Canada, 2019. Published online: https://paper:: nips.cc. pttp//pppp///1199
[36] Mang Ye, Xu Zhang, Pong Yuen, and Shih-Fu Chang. Unsupervised embedding learning via invariant and spreading instance feature. In Proceedings of CVPR, pages 6210-6219, Long Beach, CA, 2019.
[37] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory F. Diamos, Erich Elsen, David Gareía, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. Mixed precision training. CoRR, abs/1710.03740, 2017.
12

---

[38] Yang You, Igor Gitman, and Boris Ginsburg. Scaling SGD batch size to 32k for imagenet training. CoRR, abs/1708.03888 2017.
[39] Priya Goyal, Piotr Dollár, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangging Jia, and Kaiming He. Accurate, large minibatch SGD: training imagenet in 1 hour. CoRR, fel//706.06677... 2017.
[40] Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with restarts. CoRR, abs//608..em 030883 2016.
[41] Eugene Kharitonov, Roberto Dessi, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni. EGG: a toolkit for research on Emergence of lanGuage in Games, https://gi hhtbb/tttttuut... com/
facebookrresearch/E66 2021.
[42] Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition challenge. International Journal of Computer Vision, 115(3):211-25,, 2015.
[43] Christiane Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA, 1998.
[44] Junnan Li, Pan Zhou, Caiming Xiong, and Steven Hoi. Prototypical contrastive learning of unsupervised representations. In Proceedings of ICLR, Online event, 2021. Proceedings at: https://openreviexx netn wttpiewwwweetttteeeeeeeee /tt/ ccc/02///oofereencccc
[45] Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. Learning deep features for scene recognition using Places database. In Proceedings of NIPS, Montreal, Canada, 2014. Published online: https::/
[46] Priya Goyal, Quentin Duval, Jeremy Reizenstein, Matthew Leavitt, Min Xu, Benjamin Lefaudeux, Mannat Singh, Vinicius Reis, Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Ishan Misra. VISSL. https:
comm//tceebooireeeerrrai/ssss 2021.
[47] Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: analyzing text with the natural language toolkit. "O' O'Reilly Media, Inc.", 2009.
[48] Robert Berwick and Noam Chomsky, Why Only Us: Language and Evolution. MIT Press, Cambridge, MA. 2016.
13


Reviews:
[{"review_id": "N7ub0WPjevn", "reviewer": "Reviewer_DKJ7", "summary": "The model aims to train a pair of two ConvNets in the setup of the so-called discrimination game -- one ConvNet (sender) can pass discrete information to the other ConvNet (receiver) to identify the same image presented to the Sender among a set of images. Generating such discrete (bottlenecked) information is learned in an end-to-end manner, which is claimed to provide a human-interpretable inter-ConvNet communication. Compared to SimCLR (Chen et al., ICML 2020), the proposed method shows better or matched performance in terms of discrimination game-playing accuracy and downstream image classification task.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper needs to be more organized and focused. As the main task that the paper aims to solve is the communication between two deep neural networks, I would expect to see a more thorough analysis of the communication. E.g. (i) why |V| is chosen as 2048 and what would happen with larger or smaller values, (ii) sparsity or entropy of V, and/or (iii) what information V is learning. \n\nMoreover, the proposed model is mainly compared with SimCLR, a self-supervised image representation learning method, but such a comparison does not look fair. SimCLR is mainly trained to make closer the latent representations of the same image of different views, thus simply argmax-ing SimCLR’s internal representations for computing game-playing accuracy (Table 1) seems unfair.\n\nPlus, as summarized in Section 2.1, it looks like there is some prior work for the deep net emergent communication task, but surprisingly none of them is compared. Some prior work has already proposed to use the discrimination game and to use discrete communication protocol. As I am not closely following this field, it is very difficult to judge its novel contributions and their significance. For me, the contribution looks like training the existing model in an end-to-end manner. I would expect to see the author’s response on this.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper does not address the limitations and societal impact of their work. I would suggest adding a discussion section to describe the potential limitations of the proposed model.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "I--qI2iff08", "reviewer": "Reviewer_rUAU", "summary": "This paper studies the visual features emerging from a discrete referential game between networks. A sender network processes the input image into a discrete symbol; the receiver is tasked to identify the input image from discrators images using the encoded discrete symbol. The authors study the emerging features based on their referential accuracy, interpretability, and quality as unsupervised pre-trained features.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "=== Strengths ===\n\n+ The paper is well written and the presentation is easy to follow\n\n+ I appreciate the analysis on interpretability in section 4. The WNsim score computed using WordNet taxonomy shows that the emergent features are semantically meaningful. The analysis on the sparsity of SimCLR features, demonstrated by its interpretability scores, is particularly useful.\n\n+ The downstream object classification results in Table 3 are competitive against the strong baseline SimCLR. This shows great potential for the proposed referential as an alternative self-supervised pre-training method.\n\n=== Weaknesses ===\n\n- One of the main contributions the authors claim is that the visual features emergent from the referential game is of high quality. The authors argue this by showing competitive numbers against SimCLR in downstream classification, both with frozen features. I would like to also see the comparisons on fine-tuned features, which measure the usefulness of the emergent features more directly.\n\n- I am confused on why there is no data augmentation during testing in the referential game (L181-182). This means the exact same image is present in the candidate images. Wouldn’t this mean that any deterministic network acting as both the sender and receiver, even with completely randomized weights, can achieve 100% accuracy, assuming there are no ties?\n\n- The authors claim that this paper provides evidence that communication protocols emerge in a realistic setting (L8-10). In my opinion (my opinion only), this is slightly sensationalist.\nFirstly, I disagree that the emergence feature can be considered as “communication protocols”, as they merely describe the implicit class of the input image. Admittedly, they are emergent and discrete. However, prior work has already shown that it is possible to learn compact discrete latent variables of images, without any supervision [1,2]. Secondly, I disagree that the described referential game is a realistic setting, as the receiver agent is only tasked to identify **the exact same image** that the sender encodes. \n\n\n[1] Neural Discrete Representation Learning, van den Oord et al., 2017\n\n[2] Generating Diverse High-Fidelity Images with VQ-VAE-2, Razavi et al., 2019\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Discussed in the paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "8oQFg039vdx", "reviewer": "Reviewer_gFT2", "summary": "This paper investigates a large-scale emergent communication task, where a speaker and listener agent learn to communicate in a referential game, and draws connections between emergent communication setups and recent advances in self-supervised learning (e.g. SimCLR). \n\nThe authors show that it is possible to train emergent communication agents to such a degree that the learned communication protocol is interpretable, generalizes to unseen object categories, and most surprisingly, learns visual representations that are comparable to SimCLR.\n\nOne of the key insights enabling such a result is borrowed from the contrastive learning literature: the authors propose using standard data augmentation techniques to make the speaker and listener see different \"views\" of the same image, to prevent overfitting to low-level visual features present in the input. This is a common concern of emergent communication systems. They convincingly show via Gaussian blob experiments that such augmentation applied at scale prevents degenerate communication schemes, and only with such an augmentation are such high-quality visual representations learnable.\n\nThe setup is a fairly typical one used in the emergent communication literature: speaker and listener are parameterized by (possibly seperate) convolutional networks, and the speaker learns to embed the identity of a target image into a one-hot vector, which is given to a listener which must identify the referred target. The entire system is end-to-end differentiable.\n\nOverall, this paper, with its striking finding that the visual representations learned by emergent protocols are as good as SimCLR, makes an exciting step towards more seriously considering  emergent communication-based setups as an alternative way to learn good visual (and linguistic) representations for a variety of downstream tasks.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n\n- Overall, this paper is very good: it presents a clear and highly scalable emergent communication setup, at a larger scale that has been attempted before, and shows that agents are able to communicate with an interpretable protocol that generalizes to unseen categories. There are a lot of experimental details given in the paper which should facilitate reproducibility of the presented results, and the released codebase will hopefully make this easy.\n- The most striking finding is that the visual features learned by agents in this game are able to perform on par with SimCLR (though this is a reimplementation, which doesn't achieve quite as high performance as the best model in the original paper).\n- Theoretically, I appreciate the new connections between recent self-supervised learning techniques (SimCLR) and emergent communication-based setups. I especially like the authors idea of casting SimCLR as a two-player game. Drawing these connections allow us to bring many of the insights from self-supervised learning to bear on the emergent communication problems.\n- Along these lines, one of the notable borrowed insights is the idea of using data augmentation to define different \"views\" of an image given to speaker and listener. This greatly increases robustness of the learned communication and visual representations, which is a very common concern in the emergent communication literature (Bouchacort and Baroni 2018, Kottur et al. 2017), and (as noted in Footnote 1) has typically been ameliorated with handcrafted, non-scalable solutions. The Gaussian blobs experiments are very astute inclusions here.\n- Even the naive casting of SimCLR as a zero-shot \"emergent communication\" baseline is interesting (e.g. that the argmax is meaningful).\n\n## Weaknesses\n\n- The paper does not have much significant technical novelty; it seems to be a fairly straightforward extension of a now-standard emergent communication setup in the literature. I don't see this as a crucial weakness, however. The increased scale (and the interesting tricks that enable scalable training) are drawn from the papers biggest contribution, which is the connections to the self-supervised learning literature.\n- It's unclear how truly OOD the collected OOD images are, and how difficult generalization to a few held-out ImageNet categories is in this setup. At least from Figure 4, the OOD images do not seem *that* OOD to me (we still have birds for Symbol 1, for example). A figure depicting sampled OOD images and their closest in-domain WordNet classes might be useful here. More interesting would be to compare generalization outside ImageNet (e.g. any of the adversarial ImageNet datasets, or other datasets entirely, e.g. the ones used for transfer in Table 3).\n- The paper could compare and contrast emergent communication-based setups to other unsupervised learning methods. For example, especially in the shared CNN setting, an emergent communication task looks very similar to a discrete autoencoder (which can be used as an \"unsupervised image annotation\" tool, whose features can be reused for supervised learning, etc). What are the benefits of casting this as a referential game - presumably that it is much easier/more tractable to learn to distinguish between images, rather than reconstruct ones (which is a similar purported advantage of contrastive learning)?\n- It may be better to report adjusted mutual information (Vinh et al., JMLR 2010; [https://scikit-learn.org/stable/modules/generated/sklearn.metrics.adjusted_mutual_info_score.html#sklearn.metrics.adjusted_mutual_info_score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.adjusted_mutual_info_score.html#sklearn.metrics.adjusted_mutual_info_score)) rather than normalized mutual information in Table 2, as the former is adjusted for chance.\n\n## Questions\n\n- One of the interesting findings of the Bouchacort and Baroni paper is that, even in the referential game with different images presented to both speaker and listener, the agents are still able to generalize to gaussian blobs. What is different, then, about the augmentation+ strategy presented in this paper that makes it unable to generalize to gaussian blobs? Is it simply increased scale?\n- Have authors explored the effect of vocabulary size?\n- What about the number of distractor images in each game?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "**No**: In L457 of the checklist, I would caution against authors claiming that a discrete channel for multi-agent communication is *inherently* \"interpretable\" and will unequivocally \"have a positive impact in terms of explainable AI.\" In fact there are many cases in the literature (e.g. Kottur et al., 2017) where such discrete protocols are distinctly uninterpretable, and care must be taken not to naively assume otherwise. Obviously encouraging more interpretable protocols, e.g. via the data augmentation methods described here, is a good step, but does not guarantee interpretability and positive impact. I'd encourage the authors to consider this more carefully either in the main text or in the supplement.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Although ratings were quite divergent, I find myself in agreement with the reviewer arguing for acceptance. There are several interesting empirical findings in this work, and I find the connections being established between emergent communication and contrastive learning very interesting. I would have liked to see referential games with message lengths greater than one and experiments varying the way distractors were sampled. Though, I do appreciate these being called out explicitly as future work. Please incorporate the additional results and clarifications from your rebuttal/discussions into the camera ready version of the paper. A couple of small suggestions on my side: I think it could be smart to reduce the size of Figure 3 and use the extra space to walk the reader through the abstract connection you are making more thoroughly. The core of the argument and the setup for why comparing with SimCLR makes sense appear to be things that readers could misunderstand, and perhaps the authors can guard against this with a bit more discussion of them. Finally, thank you for your engagement with the reviewers during the rebuttal period.
Decision: Accept (Poster)