PatchGame: Learning to Signal Mid-level Patches in Referential Games
Authors: Kamal Gupta
Gowthami Somepalli
Anubhav Gupta
Vinoj Jayasundara
Matthias Zwicker
Abhinav Shrivastava
Venue: NeurIPS 2021 Poster (2021)
Keywords: emergent language
referential games
self-supervised learning
mid-level patches

Abstract: We study a referential game (a type of signaling game) where two agents communicate with each other via a discrete bottleneck to achieve a common goal. In our referential game, the goal of the speaker is to compose a message or a symbolic representation of "important" image patches, while the task for the listener is to match the speaker's message to a different view of the same image. We show that it is indeed possible for the two agents to develop a communication protocol without explicit or implicit supervision. We further investigate the developed protocol and show the applications in speeding up recent Vision Transformers by using only important patches, and as pre-training for downstream recognition tasks (e.g., classification).

PatchGame: Learning to Signal Mid-level Patches in Referential Games
Kamal Gupta kampta@umd.. edu
Gowthami Somepalli gowthami @umd edu
Anubhav Gupta anubhav@umd. edu
Vinoj Jayasundara vinoj @umd edu
Matthias Zwicker zwicker@umd. edu
Abhinav Shrivastava abhinav8cs umd edu
University of Maryland, College Park
Abstract
We study referential game (a type of signaling game) where two agents commu- nicate with each other via discrete bottleneck achieve a common goal. In our referential game, the goal of the speaker is to compose message or symbolic representation of "important" image patches, while the task for the listener is match the speaker's message to different view of the same image. We show that it is indeed possible for the two agents to develop a communication protocol without explicit or implicit supervision. We further investigate the developed protocol and show the applications in speeding up recent Vision Transformers by using only important patches, and pre-training for downstream recognition tasks (e.g.. classification).
Introduction
The ability to communicate using language is a signature characteristic of intelligence [60]. Language provides structured platform for agents to not only collaborate with each other and accomplish certain goals, but also to represent and store information in compressible manner. Most importantly, language allows us build infinitely many new concepts by the composition of the known concepts. These qualities are shared by both the natural languages used in human-human communication and programming languages used in human-machine communication. The study of the evolution of language can hence give us insights into intelligent machines that can communicate [54].
Our goal in this paper is to develop and understand an emergent language, i.e.. language that emerges when two neural network agents try to communicate with each other. Clark [14] argued that supervised approaches that consist of single agent learning statistical relationships among symbols don't ton't capture the functional relationships between the symbols i.e., the use of symbols leading to an action an outcome. Krishna al. [43] argued the same viewpoint in the context of images. We, therefore, resort to the recent works in emergent language [5, 30, 42, 45, 46, 74] which show that communication protocol can be developed or learned by two or more cooperative agents trying to solve a task. The choice of the task is quintessential since the language derives meaning from its use [78]. We choose a task where two agents, speaker, and a listener, play a referential game, type of signaling game first proposed by Lewis [48]. The speaker agent receives target image and sends a message to the listener. The message consists discrete symbols or words, capturing different parts of the image. The listener receives another view of the target image, and one or more distractor
Code available at https://kamptt: wwtggi..../pateeeeee wwt.pticppphhhpeee game..
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

grundruthh
Speaker S
Ouock
Qvédin
Listener Lig
Speaker view X
Listoner
X Sp
Figure Overview of Referential Game generate two random views every image in the given batch of images. The speaker takes one of the views as the input and generates a sequence of symbols (or message). The listener takes the message sent by the speaker and the second view of the image, and projects both into an embedding space. Both the speaker and listener agents learn by minimizing the constrastive loss (see Eq. between between the views in this embedding space.
images. The goal of the speaker and the listener agents is to maximize the agreement between the message and the target image. Fig. illustrates the overview of the proposed referential game.
In computer vision, a number of attempts have been made to represent images as visual words [38], with focus on low-level feature descriptors such as SIFT [49], SURF [6], etc.. Recent works in deep learning have attempted to describe the entire image with fixed number of discrete symbols [58, 59, 63]. however, we postulate that large images contain a lot of redundant information and good visual representation should focus on only the "interesting" parts the image. discover what constitutes the interesting part of the image, we take inspiration from the works mid-level patches [18, 37, 70], the patches in the image that are both representative and discriminative [28, 70]. This means they can be discovered in large number of images (and hence representative), but simultaneously they should also be discriminative enough to set an image apart from the other images in the dataset. Hence, the speaker agent in paper focuses computing symbolic representation in terms of these mid-level patches, as opposed to the entire image.
To summarize, propose PatchGame, a referential game formulation where given an image, the speaker sends discrete signal in terms of miv-level patches, and the listener embeds these symbols to match them with another view of the same image in the presence of distractors. Compared to previous works [22, 30, 45], we make the following key changes:
Agents in the some of the prior works [22, 30, 45] have access pre-trained network, such as AlexNet [44] or VGG [69], for extracting features from images. In this work, the agents rely on training on large scale image dataset, and invariance introduced by various image augmentations, to learn the language in a self-supervised way [53].
We propose novel patch-based architecture for the speaker agent, which comprises of two modules: (1) PatchSymbol, a multi-layered perceptron (MLP) that operates at the patch-level and converts given image patch into sequence of discrete symbols, and (2) PatchRank, ConvNet that looks at the complete image and ranks the importance of patches in a differentiable manner.
We introduce novel mansformer-basee architecture for the listener agent, consisting of two modules: (1) a language module that projects the message received from the speaker to a latent space, and (2) vision module that projects the image into the latent space. We a contrastive loss in this latent space to train both the speaker and the listener agents simultaneously.
We propose new protocols to evaluate each of the speaker and listeners' modules.
We assess the success of PatchGame via qualitative and quantitative evaluations of each of the proposed component, and by demonstrating some practical applications. First, we show that the speaker's PatchRank model does indicate important patches in the image. We use the top patches indicated by this model to classify ImageNet [16] images using pre-trained Vision Transformer [19] and show that we can retain over 60% top-1 accuracy with just half of the image patches. Second, listener's vision model (ResNet-18 18) can achieve upto 30.3% Top-1 accuracy just by using K-NN (k = 20) classification. This outperforms other state-of-the-art unsupervised approaches [28, 63) that learn discrete representations of images by 9%. Finally, we also analyze the symbols learned by our model and the impact of choosing several hyperparameters used in our experiments.

---

2 Related Work
Referential games. Prior to the advent of deep learning, sigmificant research in the field of emergent communications has shown that a communication protocol can be developed or learned by agents by playing a language game [2-5, 39, 42, 74, 77]. However, the agents employed these works were typically located in synthetic world and made several assumptions about the world such as the availability of disentangled representations of objects with discrete properties. More recent works [15, 23,36 36, 45-47, 73, 75] have employed deep learning methods to develop discrete language for communication between the agents. Lazaridou et al. [45] used neural network agents represented by MLP to communicate concepts about real-world pictures. They used fixed-sized message composed of a large vocabulary for their communication. Bouchacourt and Baroni [9]. Evtimova et al. [22], Havrylov and Titov [30] relax this assumption and allow communication via variable- sequences. Havrylov and Titov [30] allows the speaker agent to use an LSTM [33] to construct variable-length message. Havrylov and Titov [30], Lazaridou et al. [46] show that even when we allow agents to use variable-length sequences to represent a message, they tend to utilize the maximum possible sequence achieve the best performance (in terms of communication success).
The idea of using a Gumbel-softmax distribution [34, 51] with the trraightthroough trick [7] for learning language in multi-agent environment was concurrently proposed by Mordatch and Abbeel [56] and Havrylov and Titov [30]. They show that we can achieve more stable and faster training by using this technique compared to reinforcement learning used in several other approaches.
Evaluating emergent communication. Evaluating the emergent language turns out to be an equally challenging research problem. Existing approaches use the successful completion of the task or the correlation between learned language and semantic labels as evaluation metrics. Lowe et al. [50] and Keresztury and Bruni [40] show that simple task success might not be a good or sufficient metric for evaluating the success of game. They discuss heuristics and advocate measuring both positive signaling and positive listening independently to evaluate agents' communication. Andreas [1] provides a way of evaluating compositional structure in learned representations.
Discrete Autoencoders. In parallel to the works on emergent communication, there is a large body of research on learning discrete representations of images using some form of autoencoding or reconstruction [20, 28, 29, 57, 58, 62, 63, 65] without labels. The focus of VQ-VAE [58] and VQ-VAE-2 [63] is learn a discrete bottleneck using vector quantization. Once we can represent any image with these discrete symbols, powerful generative model such as PixeICNN [58, 66].
or transformer [61, 76] learned on top of these symbols to sample new images. Patch VAE [28] achieves the same using Bumbel-Softmar and imposes an additional structure in the bottleneck of VAES. We argue that because of mismatch in the objectives of reconstruction and visual recognition tasks, each of these models trained using peconstruction-basel losses do not capture meaningful representations in the symbols.
Self-supervised learning in vision. Self-supervised learning (SSL) methods, such as [10-13, 26, 32, 55, 81] have shown impressive results in recent years on downstream tasks of classification and object detection. Even though the bottleneck in these methods is continuous (and not discrete symbols), these methods have been shown to capture semantic and spatial information of the contents of the image. Unlike SSL methods, neural networks representing the two agents in our case, do not share any weights. Also, note that the continuous nature of representations learned by SSL techniques is fundamentally different from the symbolic representations used in language. And indeed, we show that a k-Nearest Neighbor classifier obtained from the continuous representations learned by SSL methods can perform better than the one obtained using Bag of Words (or symbols). However, to the best of our knowledge, our work is one of the first attempts to make representations learned in a self-supervised way more communication- language-oriented.
Comparison with Mihai and Hare [52, 53]. [52, 53] extends Havrylov and Titov [30] by training speaker and listener agents end-to-end without using pre-trained networks. However, the prior works [30, 52, 52] use a top-down approach to generate discrete representation (or a sentence) for an image, i.e., they compute an overall continuous embedding of the image and then proceed by generating one symbol of the sentence at time using an LSTM. The computational cost of LSTMs is prohibitive when length of sentence is large, which is needed to describe complex images. The transformers, on the other hand, require constant time for variable length sentences at the cost of increased memory (in the listener agent). However, generating the variable length sentences with the
3

---

speaker agent using transformers is non- trivill.. To solve this, we propose bottom-up approach, i.e., we first generate symbols for image patches and combine them to form a sentence. This approach allows for computationally efficient end-to-end training. Further, it allows the speaker to compose symbols corresponding different parts of the image, instead of deducing from pooled 1D representation of the image.
3 PatchGame
We first introduce various notations and the referential game played by the agents in our work. We provide further details of architectures of the different neural network agents, as well as the loss function. We also highlight the important differences between this work and prior literature. Code and pre-trained models to reproduce the results are provided.
3.1 Referential Game Setup
Fig. shows the overview of our referential game setup. Given dataset of N images {x, )x, we formulate a referential game [48] played between two agents, speaker Sn and listener L& as follows: As in the setting of Grill et al. [26]. we generate two "random views" for every image. A random view is generated by taking a 224 x 224 crop from randomly resized image and adding one or more of these augmentations color jitter, horizontal flip, Gaussian blur and/or solarization. This prevents the neural networks from learning a trivial solution and encourages the emergent language to capture invariances induced by the augmentations. Given a batch of B images, we refer to the two views as } and {x) each iteration during training, one set of views is presented to the speaker agent and another set of the views is shown to the listener agent.
The speaker So encodes each image x; independently into variable length message m,, Each message m is represented by a sequence of one-hot encoded symbols with maximum possible length and fixed size vocabulary V. The space of all possible messages sent by the speaker is of The the order |V/4. The input to the listener T, is the batch of messages [m from the speaker, and the second set of random views of the batch of images. The listener consists of language module
Siext to encode messages and a vision module @vision to encode images. The goal of the listener is to match each message to its corresponding image.
Specifically, for batch of B (message, image) pairs, Sq and Lg are jointly trained to maximize the cosine similarities of B actual pairs while minimizing the similarity of (B2 - B) incorrect pairs. For target message my. the image x/J (augmented view of xj) acts the target image while all the other (B - 1) images act distractors. And vice versa, for the image x the message my (encoded by the speaker So (xx)) acts as the target message while all the other (B - 1) messages act as distractors. We use the following symmetric and contrastive loss function, also sometimes referred InfoNCE loss in previous metric-learning works [59, 72].
B
exp exp (ep(@rex(mm))) Ovision (x,)/T)
j=1 Liest -
Σ
log
(1)
exp(Prcatt @vision ( (2/))/ (x/)/T)
13
exp(drext (m+) dvision \/T
Lvision Σ log
(2)
Σラ=1 exp( drexx (m,) @vision /T/
k=1
C= == + Livision)
where 7 is constant temperature hyperparameter.
The game setting used in our work is inspired from Lazaridou al. [45] and Havrylov and Titov [30], but there are important differences. Both our speaker Sş and listener Lg agents are trained from scratch. This makes the game setting more challenging, since agents cannot use the pre-trained models which have been shown to encode semantic and/or syntactic information present in natural language or images. Our training paradigm, where we show different views of the same image the speaker and listener, is inspired by the recent success of self-superviser learning in computer vision. Empirically, we observe that this leads to a more stable training and prevents neural networks from learning degenerate solutions. However, in contrast with such self-supervised approaches, our goal is to learn a discrete emergent language as opposed to continuous semantic representations. We discuss the differences in the architecture of the two agents in the following sections.

---

campe
patzhes
eatwor
Pruck
1.1
8.9
6.8
mossacO II
imaga
Figure 2: Speaker agent architecture. The upper branch represents the PatchSymbol, 0ymb module and the lower branch represents PatchRank, Orank module. Each of the modules take the raw image the input. PatchSymbol computes message for each patch of pre-defined size, using fixed size vocabulary and message length. PatchRank ConvNet compute weights the importance of each patch. Note that the symbols for patch are context independent, however the importance a patch depends on the context. The speaker agent combines the output of each of the models and sends a variable length message to the listener.
3.2 Speaker agent architecture
A desirable characteristic of the speaker agent is that it should be able to encode "important" com- ponents of images with a variable length sequence of discrete symbols. Previous works [9, 22, 30] have achieved this by first converting the image into continuous deterministic latent vector and then using an LSTM network [33] to generate a sequence of hidden states, and sample from this sequence of hidden state until special end of sequence token (or maximum length) reached. As observed by [30, 46], in order to achieve the minimum loss, the model ends up always using the maximum allowable length. In our experiments as well, we observed that having an LSTM makes the training slow and does not achieve the objective of encoding images in variable length sequences. We propose leverage separate modules in the speaker agent Sş to circumvent this problem the first module called PatchSymbol (0symb) is 2-layer MLP that computes patch-level embeddings for the image, the second module called PatchRank (frank) is small ConvNet that computes rank importance of each patch in the image.
PatchSymbol, 6symb- The idea of encoding image at the patch level is inspired by the works on discovering mid-level patches [18, 37, 70] in images. We use a simple 2-hidden layer MLP, to encode each Rxs dimensional image patch Xpatch to 1 vectors of log of probabilities [log pl..... log pl....... [logp/..... p/....,..ogg. log ply] image or patch,
Here C' is the number of (color) channels in the input the spatial dimension of square patch, V the size of the vocabulary used to represent single symbol, and / is the number of symbols used to encode each patch. Hence an image of size x XW can be encoded using K of log probabilities allow to sample from categorical distribution of V categories, with continu- relaxation by using the Gumbel-softmal trick [34, 51]. For given vector [log log pi-), we draw i.i.d samples of from the Gumbel(0. 1) distribution [27] and get differentiable approximation of arg max as follows:
#W
patches, each consisting of symbols. The vectors
s
[[og pl. .l...., log l,,.....,
plo.p...... log p = MLP MLLP(xpunt))
(4)
V
exp((logr) + + g.)/Ta))
y = one leoo_not(arrge mox_
.jj
(5)
kml exp((logr) + g
i=1
0symbl (Xpatch) = {
6
where т, controls how close the approximation is to arg max. The final output of the 0sybb network
for the entire image x is L one-hot encoded V- didimensional symbols, where L x In all our experiments, we fix V = 128 and 1.
HW
8
PatchRank, frank- An image might have a lot of redundant patches encoded using the same symbols.
The goal the 0ran network to give an importance score to each patch. Since importance of patch depends on the context and not the patch alone, we use small ResNet-9 [31] to compute an importance weight for each of the K weights is to simply normalize them between (0, 1) and repeat the
WW patches. One possible way to use these importance trick to sample
5

---

important patches. The listener network Lg would see only the message consisting of "important" patches. However, we empirically observed that simple min- max or L2-normalization allows the network to assign high weights to each patch and effectively send the entire sequence of length T. to the listener. Instead, we propose to use differentialle ranking algorithm by Blondel al. [8] to convert the importance weights to soft-ranks (1..... K} in O log K) time. This method works by constructing differentiable operators as projections onto the convex hull of permutations. Once we have the vector of soft-ranks r E R* we normalize the ranks and sample binary values again using special case of the Gumbel-softmax trick for Bernoulli distributions [34, 51] as
r(x) arg max (CNN(x), pm) REE
Orank (x) ~ Bern
R RT(x²)
(8)
where CNN(x) are the importance weights obtained by applying a ResNet-9 to the image x, 2 is the set of all K! permutations, and Pr are the ranks corresponding to the permutation и E Σ. We refer the reader to [8] for a detailed description of the soft-sort algorithm. Therefore, the final symbols encoded by the speaker agent, Sa, is given by:
So(x) = 0xym((x) Orank (x)
3.3 Listener agent architecture
As discussed in $3.1, the listener agent Lg consists of language module Uwxt and vision module @vision- We implement trex using small transformer encoder [76]. We prepend a (CLS) token the beginning of each message sequence received from the speaker [17, 29], and use the final embedding of (CLS) to compute the loss described in Eq. 3. We implement @vision using small vision transformer [19], and follow a similar procedure as in the text module to obtain the final image embedding. Both the text and vision modules use similar transformer encoder architecture (no weight sharing) with 192 hidden size, 12 layers and attention heads. Following [11, 13, 81], we add high dimensional projection at the end of the last layer before computing the loss function.
3.4 Training
Each of the weights of the speaker and listener agents {0-ymb: Grank, Vvision Dweu) are optimized jointly during the training. We use 2-layer MLP for Osymb. ResNet-9 [31] for Orank. a ResNet-18 [31] for @vision. and a small transformer encoder (hidden size 192, heads, 12 layers) for érext- All the experiments are conducted on the training set of ImageNet [16]. which has approximately 1.28 million images from 1000 classes. We create a training and validation split from the training set by leaving aside 5% of the images for validation. After obtaining the final set of aaperr-prameters. we retrain on the entire training set for 100 epochs. We use Stochastic Gradient Descent (SGD) with momentum and cosine learning rate scheduling. In order to train the stochastic component of the Speaker, we use the straigh-tthouuhh trick [7]. We reiterate that the speaker and listener do not share weights, and the only supervision used is the InfoNCE loss defined in Eq. 3. Please refer to the appendix and code attached in the supplementary material for more details.
Experiments
We evaluate the success of communication in the referential game and impact of various hyper- parameters on the success. Also, following the work of Lowe et al. [50], we evaluate the emergent communication in two primary ways. In section 4.1, we measure positive signaling, which means that Sa sends messages relevant to the observation. In section 4.2, we measure positive listening, which indicates that the messages are influencing the Ls agent's 's behavior.
4.1 Positive Signaling- Visualizing Patch Ranks from Orank
We first visualize the output of PatchRank module as heatmap overlayed on the original image in Fig. 3. Most important patches are colored towards 'red' and the least important ones are colored towards 'blue". The figure shows that our PatchRank can capture important and discriminative parts of the images. In case of images of various animals and plants, the model assigns the highest important

---

Abacus
African Chamelzon
African Crocodila
Mushroom
Remota Control
Dishrag
Revoiver Gun
Rhinoceros Beatle
Rhodesian Ridgeback
Table Lamp
Thunder Snake
Sombrero
Hard
House Finch
Soup
Figure 3: Heat maps based Patch Importance. Red represents the most important patches and Blue the least important. Labels listed for better understanding and not used during training. We have used 224 224 images and 32 32 patches to get 49 non-overlapping patches per image. Model successfully separates primary object in an image the "device" the "Abacus" image, "rabbit" in the "hare" image etc. (as shown by higher concentration of yellow-to- torded patches on these areas). Some failure cases are shown as well on the right.
80
spach
epoch
Top-1 Accuracy (%)
100
Top-1 Accuracy (96)
epoch 100
40
40
20
20
40
80
100
120
140
0
20
40
180 patch
100
120
140
180
(a) 32 x 32 patches
(b) 16 16 patches
Figure Impact number patches used during evaluation a pre-trained VIT [19]. The performance of a VIT drops if we provide fewer patches during inference. However a PatchRank model (which is small ResNet-9) can provide important patches to VIT during inference with minimal loss in Top-1 accuracy.
discriminative body parts. For the inanimate objects such as abacus or revolver the model is able to distinguish between the foreground and the background. Note that although approaches such GradCam [67] can provide pixel-level importance heatmaps, they require extensive supervision. Our method on the other hand is self- supervised. We also show some of the failure cases when discriminative patches the image cover majority the image on the rightmost 2 columns of Fig. 3.
4.2 Positive Signaling Image Classification with subset of patches provided by frank
Recently proposed Vision Transformers (ViT) by Dosovitskiy et al. [19] have gained popularity because of their simplicity and performance. These models treat an image as a sequence of N x N patches, use an MLP convert the patch into an embedding, and finally use these set of patches to perform classification. We first note that both the inference time and memory consumption of VIT depend largely on the length sequence, because the ( self-attention operation. Secondly, the performance of the Vision Transformers drops if instead of using all patches, we only use subset of patches during inference. This provides us simple way to evaluate the frank module. We artificially constrain the number of patches available to ViT during inference. We measure the Top-1 Accuracy of ViT using only & allowed patches. The selection of the patches is done by the Orank model.
We consider two different pre- pre-trained VIT [19] models, one trained using 32 x 32 patches, on the images of size 384 384 (so the number of patches an image is 144), while the second model is trained using 16 16 patches on the images of size 224 x 224 (196 patches per image). In Fig. 4a and 4b, we show the Top-1 accuracy obtained by the pre-trained ViT models using important patches predicted by 0rnk at different values of k. At the t epoch (with random weights), the performance of ViT drops almost linearly as we lower the patch count k. At the 100th epoch, at the end of the training. we observe that performance of the ViT does not drop drastically.
7

---

12
18
42
90
91
97 110
113
118
119
123
TFE
lakAn
Sto
grass
scales
branches
text
eyes
lexture
Laces
fur
soil
eyes
Figure Visualizing patches corresponding to various symbols. The number in top row corresponds to of 128 vocabulary ids. representative patches corresponding to each patch are shown. The bottom row corresponds to our interpretation of what concept that symbol might be capturing.
Table 1: Performance of Listener's Vision module
Table 2: Performance of Listener's Vision module
@vision Downstream classification accuracy for
@vssion Downstream mean Average Precision for Pascal VOC
ImageNet dataset using K-NN (k=20)
Method
Top 1 (%)
Top-5 (%)
Method
mAP-50 (%)
MoCo-v2 [32] (R18)
36.8 (#: 0.2)
60.3
MoCo-v2 [32] (R18)
65.8
VQ-VAE2 [59]
17.2 (2: 0.6)
30,5
PatchVAR [28] (S = 16)
52.2
PatchVAB [28] (RI8, 8 = 16)
16.4 0.6)
28.5
PatchVAE [28] (R18, 32)
(1
36.2
VAt [28] = 32)
54.2
Ours (R18,5 8 16)
27.6 0.6)
46.2
Ours 16)
58.9
Ours (R18,
32)
30.3 0.5)
49,9
Ours (S 32)
61.3
4.3 Positive Signaling- Visualizing Oyymb symbols
As mentioned earlier, we are using a vocabulary V = 128 and patch size S = 32 in our base model. This means Oyymh has map each 32 x 32 patch one of the 128 available symbols. A natural way to analyze what the symbols are encoding to see is to visualize their corresponding patches. While V 128 is far too small vocabulary interesting patterns in Fig. 5. Many of the symbols seem to adhere to specific concepts repeatedly. We observed that symbols have lesser preference for color, but more preference for texture and shape. We discovered several symbols corresponding to textures such as grass, branches, and wood. We also noticed many symbols firing for the patches corresponding to text, eyes, and even faces. There can be multiple symbols representing single concept, e.g., both symbol 91 and 123 both fire in case of eyes.
describe all possible patches, observe some
4.4 Positive Listening
Next, evaluate the vision module, Pvision of the listener. We follow the protocol employed by various approaches in self-supervised learning literature. We consider the features obtained at the final pooling layer of the @vision (which is R in our case). Next, we run a k-NN classification with & = 20 on the validation dataset. Table shows the Top-1 and Top-5 % accuracy obtained on ImageNet using the listener's vision module and the baselines approaches. Although our method outperforms VQ-VAE-2 and Patch (methods that learn discrete image representation) we observe that there is still gap between representations learned by these models as compared the representations learned by continuous latent models such as MoCo [32]. Note that, because of the resource constraints, all results reported in the table are obtained by training ResNet- 18 for only 100 epochs. The results for both MoCo-v2 and our approach continue to improve if we continue the training beyond 100 epochs (as also noted by He et [32]). Further, we use the listener's vision module as pre-trained network for Pascal VOC dataset [21]. Our results are shown in the Table 2. Again, results are not competitive as compared to self-supervised counterparts such as MoCo-v2 but we outperform models with discrete bottleneck such as Patch VAE. We find that the convergence of models with discrete bottlenecks (such as this work, and PatchVAB) is slow and hence, improving the training efficiency of this class of models is interesting future direction.
4.5 Ablation study
communication iteration is successful if the receiver is able to match the message sent by the speaker to the correct target image. In our experiments, we use an effective batch size of 512 (split over 4 GPUs), so the chance accuracy of success of the speaker accuracy for each image of the batch and average it over validation data at the end of epoch. In Fig. 6a, we observe that too high or too low learning rates can be detrimental the success of communication. We fix the learning rate at 0.0001 which shows the best validation performance empirically. We train
1.99. We measure the Top-1
8

---

100
Top-1 Acturacy (1)
0.00
Top-1 Accuracy (N)
Top-1 Accuraty (N)
0.0011
8
60
%
60
20
28
15
60
80
100
62
100
apoch
epoch
epoch
(a) Learning rate
(b) Length single patch
(c) Vocabulary
Figure 6: Impact of various hyperparameters on the success of communication. Top-1 accuracy denotes the percentage of messages in batch that model able to match to the corresponding image. Having higher message length and vocabulary helps the model learn faster.
our models at different vocabulary sizes, and different message lengths for 100 epochs. From Fig. 6b and 6c, observe that having either a large vocabulary larger message length allows the model to reach high accuracy faster. This is intuitive since the larger the space spanned by the messages is, the easier it is for receiver to distinguish between the images. A large message length increases the possible number of messages exponentially, at the cost much larger computation cost since self-attention [17, 76] is O(N²) operation. A large vocabulary also increases the both the span of messages and computation cost linearly.
Discussion
Summary. In this work, we have shown that two cooperative neural network agents can develop variable length communication protocol to solve a given task on a large scale image dataset with only supfrsuuer To the best of our knowledge, this is the first work to develop an emergent communication via mid-level patches without using any pre-trained models or external labels. We have introduced a novel nidtle based architecture for the speaker agent, in order to represent only the "interesting" parts of image with discrete symbols. The listener agent learns to align the messages and images using language and image encoders speaker and listener agents are trained end-to-end jointly to capture invariances induced by various data augmentations
common embedding space. The
order to solve constrastive task. We propose a number of quantitative and qualitative measures to analyse and evaluate the emerged language. We also show two major applications of the developed approach (1) extract representative and discriminative parts of the image (2) transfer learning in image classification tasks.
Limitations and Future Work. There are few limitations of our approach. Firstly, one of the applications discussed in Section 4.1 is using fewer patches for inference in VIT. Although, using fewer patches reduces the memory cost of ViT, the overhead of using another neural network for predicting the important patches means our gain is minimal. In the future, we would like to explore even faster architectures to have a bigger impact on classification speed. Second, our method only works with fixed-size square patches. Discovering arbitrary sized mid-level patches is a challenging task that we would like address future work. Third, the language emerged from our current model is not grounded in natural language and requires human intervention for interpretation. Going forward, we would like to ground this emerged language in the natural language.
Broader Impact. Like most self-supervised approaches, our approach is data hungry and trained using a large amount of unlabeled data collected from the Internet. Our model in its current form prone to learning and amplifying the biases present in the dataset, especially if the dataset in question not carefully curated. While the data collection and labeling has been discussed in the original paper [16]. the community has focused towards imbalance and privacy violation in existing image datasets only recently [79]. A recent study [80] shows that 997 out of 1000 ImageNet categories are not 'people' categories. To compare against previous methods and allow future benchmarking, we provide results on the 2012 version the dataset with 1.28 million images in training set and 50000 images in the validation set.
Acknowledgements. We thank Matt Gwilliam, Max Ehrlich, and Pulkit Kumar for reviewing early drafts of the paper and helpful comments. This project was partially funded by DARPA SAIL-ON (W91 INF2020009), and Amazon Research Award AS.

---

References
Jacob Andreas, Measuring compositionality in representation learning. arXiv preprint arXiv: 190.0099,, 2019.
[2] Jacob Andreas and Dan Klein. Analogs of linguistic structure in deep representations. arXiv preprint arXiv:1707.08139 2017.
[3] Jacob Andreas, Dan Klein, and Sergey Levine. Learning with latent language. arXiv preprint 1oXiv:/711.00455 2017.
[4] Andrea Baronchelli. Maddalena Felici, Vittorio Loreto, Emanuele Caglioti, and Lue Steels. Sharp transition towards shared vocabularies in multi-agent systems, Journal of Statistical Mechanics: Theory and Experiment, 2006(06):P0601,, 2006.
[5] John Batali. Computational simulations of the emergence of grammar. Approach to the Evolution of Language, pages 405-426, 1998.
[6] Herbert Bay. Tinne Tuytelaars, Luc Gool. Surf: Speeded up robust features. European conference on computer vision, pages 404-417. Springer, 2006.
[7] Yoshua Bengio, Nicholas Léonard, and Aaron Courville. stochastic neurons for conditional computation. arXiv preprint
Estimating or propagating gradients through 2013.
[8] Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking, International Conference on Machine Learning, pages 950-959. PMLR, 2020.
[9] Diane Bouchacourt and Marco Baroni. Miss tools and mr fruit: Emergent communication agents lcarning about object affordances. arXiv preprint arXiv: /1811, 2019.
[10] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsu- pervised learning of visual features by contrasting cluster assignments. arXiv preprint arXiv:2006.0988) 2020.
[11] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. arXiv preprint arXiv:2104. 104.44444, 2021.
[12] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffre Hinton. A simple framework for contrastive learning visual representations. International conference on machine learning, pages 1597-1607 PMLR, 2020.
[13] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. arXiv preprint arXiv:2011.10366. 2020.
[14] Herbert H Clark. Using language. Cambridge university press, 1996.
[15] Abhishek Das, Satwik Kottur, José MF Moura, Stefan Lee, and Dhruv Batra. Learning cooperative visual dialog agents with deep reinforcement learning. Proceedings of the IEEE international conference on computer vision, pages 2951-2960, 2017.
[16] Jia Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248 445..55. Icce, 2009.
[17] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint 18000008,, 2018.
[18] Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei Efros. What makes paris look like paris? ACM Transactions on Graphics, 31(4,, 2012.
[19] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020.
16x16 words:
[20] Patrick Esser, Robin Rombach, and Björn Ommer. Taming transformers for high-resolution image synthesis. arXiv preprint arXiv:2012.09841, 2020..
10

---

[21] M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, 111(1):98-136. January 2015.
[22] Katrina Eytimova, Andrew Drozdov, Douwe Kiela, and Kyunghyun Cho. Emergent communication in multi-modal, multi-step referential game. arXiv preprint arXiv:/705. 10369, 2017.
[23] Jakob N Foerster, Yannis M Assael, Nando De Freitas, and Shimon Whiteson. Learning to communicate with deep aulti--genn reinforcement learning. arXiv preprint arXiv:1605. 16050660, 2016.
[24] Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, and Biao Li. Axiom-based grad-cam: Towards accurate visualization and explanation of cnns, arXiv preprint arXiv:2008.02311 2020.
[25] Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming Hc. Accurate, large mimibatch sgd: Training imagenet in I hour. arXiv preprint arXiv: / 02677, 2017.
[26] Bran-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires. Zhaohan Daniel Guo, Mohammad Cheshlaghi Azar. et al. Bootstrap your latent: A new approach self-supervised learning. arXiv preprint arXiv:2006.07733, 2020.
[27] Emil Julius Gumbel. Statistical theory of extreme values and some practical applications: a series of lectures, volume 33. US Government Printing Office, 1954.
[28] Kamal Gupta, Saurabh Singh, and Abhinav Shrivastava. Patehvac: Learning local latent codes for recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11746-4755, 2020.
[29] Kamal Gupta, Justin Lazarow, Alessandro Achille, Larry S Davis, Vijay Mahadevan, and Abhinav Shrivastava. Layouttransformer: Layout generation and completion with self-attention In Proceedings of the IEEE/CVF International Conference on Computer Vision. pages 1004-1014, 2021.
[30] Serhii Havrylov and Tvan Titov. Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. arXiv preprint orXiv:/705, 1922 2017.
[31] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778 2016.
[32] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9729-9738, 2020.
[33] Sepp Hachreiter and Jürgen Schmidhuher. Long short-term memory, Neural computation, 1997.
[34] Eric Jang, Shixiang Gu and Ben Poole. Categorical reparameterization with numbel-softmax arXiv preprint 1rXiv:6611.0114,, 2016.
[35] Karen Sparek Jones. statistical interpretation of term specificity and its application retricval. Journal of documentation, 1972.
[36] Emílio Jorge. Mikael Kågebäck, Fredrik D Johansson, and Emil Gustavsson. Learning to play guess who? and inventing a grounded language as a consequence. arXiv preprint 161.v//61..03//1 2016
[37] Mayank Juneja, Andrea Vedaldi, CV Jawahar, and Andrew 7isserman. Blocks that shout: Distinctive parts for scene classification. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 923-930, 2013.
[38] Frederic Jurie and Bill Triggs. Creating efficient codebooks for visual recognition. In Tenth IEEE International Conference Computer Vision (ICCV'05) Volume 1, volume pages 604-610. IEEE, 2005.
[39] Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg, Referitgame: Referring to objects in photographs of natural scenes. In Proceedings the 2014 conference empirical methods in natural language processing (EMNLP), pages 787-798, 2014.
[40] Bence Keresztury and Elia Bruni Compositional properties of emergent languages in deep learning. arXiv preprini arXiv:2001.086 2020.
[41] Diederik P Kingma and Max Welling. Auto autooenncodig variational bayes. arXiv preprint arXiv:1312.6114. 2013.
11

---

[42] Simon Kirby, Natural language from artificial life. Artificial life, 8(2):185-215, 2002.
[43] Ranjay Krishna, lnes Chami, Michael Bernstein, and Li Fei-Fei. Referring relationships. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pages 6867-6876, 2018.
[44] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097-1105, 2012.
[45] Angeliki Lazaridou, Alexander Peysakhowich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. arXiv preprint wrXivv1612.071885 2016.
[46] Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of linguistic communication from referential games with symbolic and pixel imput. arXiv preprint wrXiv:1804.03994 2018
[47] Jason Lec, Kyunghyun Cho, Jason Weston, and Douwe Kicla. Emergent translation in multi-agent communication. arXiv preprint 1rXiv:1710.0692. 2017.
[48] David Lewis. Convention: philosophical study. John Wiley Sons, 1969.
[49] David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2)91-1110, 2004.
[50] Ryan Lowe, Jakob Foerster, Y-Lan Boureau, Joelle Pineau, and Yann Dauphin. the pitfalls of measuring emergent communication. arXiv preprint arXiv: 1903. 0066005668 2019.
[51] Chris Maddison, Andriy Maih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint 1rXiv:1611.00712 2016.
[52] Daniela Mihai and Jonathon Hare. Avoiding hashing and encouraging visual semantics in referential emergent language games. arXiv preprint arXiv: 191.5669 2019.
[53] Daniela Mihai and Jonathon Hare. The emergence of visual semantics through communication games. arXiv preprint arXiv:2101. 10253, 2021.
[54] Tomas Mikolov, Armand Joulin, and Marco Baroni. A roadmap towards machine intelligence. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 29-61. Springer, 2016.
[55] Ishan Misra and Laurens van der Maaten, Self-supervised learning of pretext-invariant representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6707-6711 6617, 2020.
[56] Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent popula- tions. Proceedings of the AAAI Conference on Artificial Intelligence. volume 32, 2018
[57] Charlie Nash, Jacob Menick, Sander Dieleman, and Peter W Battaglia. Generating images with sparse representations. arXiv preprint 103.::21033.3844 2021.
[58] Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. arXiv preprint 171.00171,.00937, 2017.
[59] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint 1.XXv:1807.03744 2018
[60] Steven Pinker. The language instinct: How the mind creates language. Penguin UK, 2003.
[61] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. 2018.
[62] Aditya Ramesh, Mikhail Paylov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. arXiv preprint arXiv:2102. 2021.
[63] Ali Razavi, Aaron van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. arXiv preprint arXiv: prxivv/906.000 /906,00466, 2019.
[64] Jason D Rennie, Lawrence Shih, Jaime Teevan, and David Karger. Tackling the poor assumptions of naive bayes text classifiers. In Proceedings of the 20th international conference on machine learning (ICML-03), pages (266662,2222 2003.
12

---

[65] Jason Tyler Rolfe. Discrete variational autoencoders. arXiv preprint arXiv:1609.02200. 2016.
[66]
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv: 1701.0557, 2017.
[67] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of IEEE international conference on computer vision, pages 618 626, 2017.
[68] Kim Seonghyeon. vq-vae-2-pytorch. https:://gthun wtt.ccco//ooc.cee. hetm//osinaalityvq--ooooo vae- pytorch, 2019.
[69] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv preprint arXiv: 1556, 2014.
[70] Saurabh Singh, Abhinav Gupta, and Alexci A Efros. Unsupervised discovery of mid-level discriminative patches. In European Conference on Computer Vision, pages 73-86. Springer, 2012.
[71] Josef Sivic and Andrew Zisserman. Video google: A text retrieval approach object matching in videos. In IEFE International Conference Computer Vision, 2003.
[72] Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 1857-1865, 2016.
[73] Matthew Spike, Kevin Stadler, Simon Kirby, and Kenny Smith. Minimal requirements for the emergence of learned signaling. Cognitive science, 41(3):623-65 658,, 2017.
[74] Luc Steels. What triggers the emergence of grammar? Society for the Study of Artificial Intelligence and Simulation of Behaviour, 2005.
[75] Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. Learning multiagent communication with backprop- agation. arXiv preprint orXiv:1605.0772 2016.
[76] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. arXiv preprint arXiv:1706.0:333 2017.
[77] Kyle Wagner, James A Reggia, Juan Uriagereka, and Gerald S Wilkinson. Progress in the simulation of emergent communication and language. Adaptive Behavior, 11(1):37-6 69, 2003.
[78] Ludwig Wittgenstein. Philosophical investigations. John Wiley Sons, 1953.
[79] Kaiyu Yang. Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hierarchy. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 547-558, 2020.
[80] Kaiyu Yang, Jacqueline Yau, Li Fei-Fei, Jia Deng, and Olga Russakovsky, study of face obfuscation in imagenet. arXiv preprint arXiv:2103.0619 2021.
[81] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. arXiv preprint arXiv22103.03230, 2021.
[82] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018.
13


Reviews:
[{"review_id": "y5zQTAJMdS8", "reviewer": "Reviewer_Hpoa", "summary": "This paper presents a visual referential game played by a pair of agents. Compared to existing work in this space, the game is a variant of those studied previously as the communication is based directly on encoded patches (rather than a latent vector describing the whole image), and the agent models diverge from prior work in their architectural construction.   \n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality\n-----------\n\nUnfortunately, one of the main claims to originality the paper makes is that prior work in emergence of language using visual referential games all looks at models with pretrained networks for visual feature extraction rather than models in which this inductive bias is removed. There was a paper at the NeurIPS 2019 Emergent Communication workshop that explored exactly this aspect and started to draw out the links between self-supervised learning (with augmentations) and emergent communication: \n    \n    Avoiding hashing and encouraging visual semantics in referential emergent language games. Daniela Mihai and Jonathon Hare. 3rd NeurIPS Workshop on Emergent Communication. 2019. https://arxiv.org/abs/1911.05546\n    \nA quick search on arxiv suggests the same authors might also have followed this up (https://arxiv.org/abs/2101.10253) with a longer paper.\n\nWith this in mind, there are still some bits of the proposed model which are potentially interesting - particularly the game setup and the agent architecture. Overall, I think the proposed setup is an interesting variation of what has been done before, although I'd have liked to have seen stronger links made to prior work incorporating visual attention --- for example attending to different parts of an image to generate discrete messages is in essence what the Show, Attend & Tell model (https://arxiv.org/pdf/1502.03044.pdf) does (albeit in a fully supervised setting). I believe the approach to generating variable length message sequences is novel, although I do have questions (below).\n\n\nQuality\n-------\n\nTechnically the proposed model and accompanying quantitive evaluation results appear generally sound. As a reader I would have appreciated more to be said about the variance of the model from differing initializations (particularly in the experiments in section 4.6) --- it is clear that the model is very sensitive to learning rate, so one also wonders how much variance from random initializations there is in e.g. Fig 6b and 6c where the model performance under differing vocabulary and message lengths is _very_ close (is it so close that it's within the margin of error?). \n\nI do wonder if the authors could further justify their choices of metrics for measuring performance of the model (or perhaps clarify their intent in creating the model); more specifically, in the context of emergent communication it seems strange that one would consider a bag-of-words model or a truncated-message model as a good proxy for comparing positive signaling as they do not capture the interesting parts of language that one might wish to observe the emergence of (such as evidence of compositionally, etc). Perhaps the aim was to just explore models with discrete latent spaces (and perhaps an implicit 'importance' ordering on tokens)? If so, such a criticism is obviously mute, but in such a case would it then not make sense to focus more attention on exactly what makes the proposed model perform better than e.g. VQ-VAE or PatchVAE?\n\nRegarding PatchRank, it's not actually clear to me that the proposed approach does really control that message length much. Surely in expectation the length of every message is just ~K/2 as (approximately because of the soft-rank) half of the values in r(x) would be >K/2? Besides the potential for some small information leakage from the relaxation in the soft-rank algorithm I can't see that the model really has much ability to learn appropriate message lengths. Do you have any quantitive evidence (e.g. mean message length and its variance over many games) that the model really is actually learning variable length better (e.g. with higher variability, not necessarily shorter than the maximum) than previous models?\n\n\nClarity\n-------\n\nOverall I found the text to be easy to follow, however I think many of the points I've raised above hint to ways to improve the content and clarity.\n\n### Minor points\n- \\theta_{rank} in Eqn. 8 is inconsistent with the rest of the text (and Fig. 2) \n\n\nSignificance\n------------\n\nCommenting on potential significance is always challenging; in this case it's not immediately obvious to me that there is that much new given the points raised above, however this could change if my queries were addressed.  \n\n\nRationale for rating\n----------------------\nOverall my opinion is that there is little novel in the paper in its current form, although there are some potentially interesting aspects which I've tried to highlight in my review. My score reflects this. I've tried to give constructive feedback around some of the more original parts of the paper and sincerely hope these will help the authors make improvements.   \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are covered adequately. I am not convinced that the current Broader Impact section in the paper adds anything relevant to the discussion - my suggestion would be to rework this into something much more tangible about how people might benefit (or not) from potential future machinery that benefits from this work (for example, how an emergent communication system between agents might be used, or how variable-length discrete compressed representations could be utilised). \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "i4JBiWQfnwS", "reviewer": "Reviewer_i9eZ", "summary": "The authors present PatchGame, \"a referential game formulation where given an image, the speaker sends discrete symbolic signals in terms of mid-level patches, and the listener embeds these symbols to match them with another view of the same image in the presence of distractors.\"", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The novelty of the work is high, and as claimed it is the first work to develop an emergent communication via mid-level patches without using any pre-trained models or external labels. While I enjoyed reading the draft (the presentation is clear), I have a few questions and also comments:\n\n1) From reading the game setting, and its root in emergent communication formation, I have the feeling that the speaker and listener model is essentially trained to encode in the speaker view image into a minimum set of patch-based representations that the listener could re-identify the image similar to the input image from a set of images. Instead of reconstructing the same image back in an encoder-decoder architecture, the game here is trying to encode the input image into a sequence of patches (with length constraint) to help another agent to identify from a set.  Is it essentially a special case of an encoder-decoder system? If not, why?\n\n2) Have you considered and/or observed the speaker and listener learned to bypass the \"important\" image patch but learned to utilize artifacts (maybe a logo in the input image, and also present in the target image from the set of candidate images) to encode the message? According to the game setting, the optimized message would be the artifact, then the model won't be able to learn to rank \"important\" image patches right? In other words, since it is an unsupervised game setting, how to prevent the agents from \"cheating\" to succeed? \n\n3) It is a comment. Have you considered utilizing this game setting to formulate a novel evaluation mechanism for visual/representation learning, rather than the debatable downstream task classification accuracy-based ones? I feel this might be the more exciting usage of your proposed game. \n\nOther places need clarification:\n\nLine 332 means we our gain is minimal. -> delete \"we\"", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The broader impact section refers to \"A recent study [75] shows that 997 out of 1000 ImageNet categories are not ‘people’ categories.\"\nThis sentence reads out of context, could you clarify the purpose of stating this here? ", "review_text": "", "extra_scores": {"ethical_concerns": "None identified. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "BrmzAyTMwKd", "reviewer": "Reviewer_5nrF", "summary": "The current work proposes to study a signaling game between two agents that uses image patches instead of the entire image. Within this framework, experiments show that the agents select important patches from the image and the emergent language captures semantic information about the patches. Further, the resultant agents are used as self-supervised image classifiers and compared against existing patch-based/self-supervisory methods to show superior performance.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**  \n(S1) Leveraging mid-level semantic patches for referential games and using the resultant agent for downstream image classification task (without pretrained features) is a novel direction. \n\n(S2) The manuscript is well-written and is easy to read and understand. In particular, it states the hypothesis clearly and places it in the context of prior work. \n\n(S3) The authors do a good job with the experiments supporting the claims in the earlier part of the manuscript. The qualitative examples of heat maps and semantic grouping of emergent language throw more light on the proposed idea, and are useful.\n\n**Weaknesses**  \n(W1) The standard deviation for performances is missing for Table 1 and 2. Given that some of these numbers are close, it is not possible to deduce if the improvements are statistically significant. \n\n**Comments**  \n(C1) How generalizable are the features/images if images beyond ImageNet are used, for example MSCOCO or OpenImages?\n\n(C2) L239-240: Is there a way to quantify how much overlap exists between important patches and the dominant objects for images in the dataset? For example, using a subset of images and computing overlap with segmentation masks for either PASCAL or other datasets with segmentation annotations?\n\n(C3) Would a pair of agents participating in a referential game (but without patches) trained from scratch converge? If yes, what would their downstream performance on image classification be?\n\n**Typos**  \nL239: shows -> show\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors discuss the limitations and broader impact of their work in the manuscript. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "ANqO7DoSGWq", "reviewer": "Reviewer_2BvU", "summary": "This paper introduces a referential game in which a speaker “sees” an image and sends a discrete message of variable length to a listener. The message is designed to represent important patches in the input image. In addition to the speaker’s message, the listener is given a batch of images, one of which is an augmented view of the speaker’s input. The listener’s goal is to find this augmented view of the input image among all images in the batch. The speaker and listener are trained with self-supervision using InfoNCE loss. The learned communication protocol is evaluated on the underlying task as well as used as a pre-training step for the downstream task of classification.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\nThis work presents a novel referential game inspired by recent developments in self-supervised learning applied to computer vision in which two views of the same input image are used to learn a representation useful for downstream applications such as classification. To the best of my knowledge, the combination of a patch-based architecture of the speaker and a transformer architecture of the listener in addition to a discrete image representation is novel and well motivated. The authors discuss the computational complexity of the proposed model. The paper is written and organized clearly. It explores an interesting research direction at the intersection of self-supervised learning and referential games and favorable classification performance when compared to existing methods such as VQ-VAE-2 and PatchVAE.\n\n### Room for improvement\nWhile the authors do note that resource constraints have prevented them from reporting error bars in their experiments, I believe the work would benefit from including these to support the statistical significance of the results. For example, the plots in Fig 6 are difficult to interpret because there are no error bars present and all curves are very close to one another. Also, Fig 2a does not display the curves past epoch 20. \n\n### Questions\n1. Have you explored what effect the batch size has on classification performance?\n2. Just to clarify, is the x-axis of Fig 6b “Length of each patch” the number of symbols used per patch? \n3. What was the motivation behind setting $l $ (the number of symbols per patch) to 1? Is it the computational cost? \n4. What is meant by “meaningful representations” in line 107? Using autoencoders as a pre-training step has been shown useful for downstream tasks such as classification. \n\n### Nitpicks\n- Fig 2’s caption: “ConvNet to compute weights...”\n- Line 140: use either “encode” or “embed”, not both\n- Line 239: “We also show some of the failure...”\n- Line 247: “the performance of the Vision Transformers drops if...”\n- Line 302: “an effective batch size of 512”\n- Line 332: “means our gain is minimal” \n- Line 339: “from the Internet”\n\n### During discussion\n- After reading the authors' response to my comments and questions, I have decided to increase my score.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the limitations and societal impact (from the perspective of training data) are discussed in the paper. The authors might also consider mentioning any potential negative applications of the proposed model. ", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The author response addressed most of the concerns of the reviewers and all reviewers recommend to accept the paper after the author response and discussion.

I recommend accept with the expectation that the authors will revise the paper for the camera ready, addressing the reviewer's concerns according to the author response, including, but not limited to the following aspects:
1. Add the mean and standard error to Table 1, 2 and Figure 4, 6
2. Add results on Pascal VOC
3. Add an extra ablation study on augmentations and batch sizes
4. Add ablation study for Mihai et al. and discuss it in the related work
5. Move the discussion on sentence length from the appendix to the main paper
6. Fix typos and make other minor writing improvements
Decision: Accept (Poster)