PatchGame: Learning to Signal Mid-level Patches in Referential Games

1 Introduction . The ability to communicate using language is a signature characteristic of intelligence [ 60 ] . Language provides a structured platform for agents to not only collaborate with each other and accomplish certain goals , but also to represent and store information in a compressible manner . Most importantly , language allows us to build infinitely many new concepts by the composition of the known concepts . These qualities are shared by both the natural languages used in human-human communication and programming languages used in human-machine communication . The study of the evolution of language can hence give us insights into intelligent machines that can communicate [ 54 ] . Our goal in this paper is to develop and understand an emergent language , i.e. , a language that emerges when two neural network agents try to communicate with each other . Clark [ 14 ] argued that supervised approaches that consist of a single agent learning statistical relationships among symbols don ’ t capture the functional relationships between the symbols i.e. , the use of symbols leading to an action or an outcome . Krishna et al . [ 43 ] argued the same viewpoint in the context of images . We , therefore , resort to the recent works in emergent language [ 5 , 30 , 42 , 45 , 46 , 74 ] which show that a communication protocol can be developed or learned by two or more cooperative agents trying to solve a task . The choice of the task is quintessential since the language derives meaning from its use [ 78 ] . We choose a task where two agents , a speaker , and a listener , play a referential game , a type of signaling game first proposed by Lewis [ 48 ] . The speaker agent receives a target image and sends a message to the listener . The message consists of discrete symbols or words , capturing different parts of the image . The listener receives another view of the target image , and one or more distractor Code is available at https : //kampta.github.io/patch-game . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . images . The goal of the speaker and the listener agents is to maximize the agreement between the message and the target image . Fig . 1 illustrates the overview of the proposed referential game . In computer vision , a number of attempts have been made to represent images as visual words [ 38 ] , with a focus on low-level feature descriptors such as SIFT [ 49 ] , SURF [ 6 ] , etc .. Recent works in deep learning have attempted to describe the entire image with a fixed number of discrete symbols [ 58 , 59 , 63 ] , however , we postulate that large images contain a lot of redundant information and a good visual representation should focus on only the “ interesting ” parts of the image . To discover what constitutes the interesting part of the image , we take inspiration from the works on mid-level patches [ 18 , 37 , 70 ] , the patches in the image that are both representative and discriminative [ 28 , 70 ] . This means they can be discovered in a large number of images ( and hence representative ) , but simultaneously they should also be discriminative enough to set an image apart from the other images in the dataset . Hence , the speaker agent in our paper focuses on computing a symbolic representation in terms of these mid-level patches , as opposed to the entire image . To summarize , we propose PatchGame , a referential game formulation where given an image , the speaker sends discrete signal in terms of mid-level patches , and the listener embeds these symbols to match them with another view of the same image in the presence of distractors . Compared to previous works [ 22 , 30 , 45 ] , we make the following key changes : • Agents in the some of the prior works [ 22 , 30 , 45 ] have access to a pre-trained network , such as AlexNet [ 44 ] or VGG [ 69 ] , for extracting features from images . In this work , the agents rely on training on a large scale image dataset , and invariance introduced by various image augmentations , to learn the language in a self-supervised way [ 53 ] . • We propose a novel patch-based architecture for the speaker agent , which comprises of two modules : ( 1 ) PatchSymbol , a multi-layered perceptron ( MLP ) that operates at the patch-level and converts a given image patch into a sequence of discrete symbols , and ( 2 ) PatchRank , a ConvNet that looks at the complete image and ranks the importance of patches in a differentiable manner . • We introduce a novel transformer-based architecture for the listener agent , consisting of two modules : ( 1 ) a language module that projects the message received from the speaker to a latent space , and ( 2 ) a vision module that projects the image into the latent space . We use a contrastive loss in this latent space to train both the speaker and the listener agents simultaneously . • We propose new protocols to evaluate each of the speaker and listeners ’ modules . We assess the success of PatchGame via qualitative and quantitative evaluations of each of the proposed component , and by demonstrating some practical applications . First , we show that the speaker ’ s PatchRank model does indicate important patches in the image . We use the top patches indicated by this model to classify ImageNet [ 16 ] images using a pre-trained Vision Transformer [ 19 ] and show that we can retain over 60 % top-1 accuracy with just half of the image patches . Second , the listener ’ s vision model ( ResNet-18 ) can achieve upto 30.3 % Top-1 accuracy just by using k-NN ( k = 20 ) classification . This outperforms other state-of-the-art unsupervised approaches [ 28 , 63 ] that learn discrete representations of images by 9 % . Finally , we also analyze the symbols learned by our model and the impact of choosing several hyperparameters used in our experiments . 2 Related Work . Referential games . Prior to the advent of deep learning , significant research in the field of emergent communications has shown that a communication protocol can be developed or learned by agents by playing a language game [ 2–5 , 39 , 42 , 74 , 77 ] . However , the agents employed in these works were typically located in a synthetic world and made several assumptions about the world such as the availability of disentangled representations of objects with discrete properties . More recent works [ 15 , 23 , 36 , 45–47 , 73 , 75 ] have employed deep learning methods to develop a discrete language for communication between the agents . Lazaridou et al . [ 45 ] used neural network agents represented by a MLP to communicate concepts about real-world pictures . They used a fixed-sized message composed of a large vocabulary for their communication . Bouchacourt and Baroni [ 9 ] , Evtimova et al . [ 22 ] , Havrylov and Titov [ 30 ] relax this assumption and allow communication via variable-length sequences . Havrylov and Titov [ 30 ] allows the speaker agent to use an LSTM [ 33 ] to construct a variable-length message . Havrylov and Titov [ 30 ] , Lazaridou et al . [ 46 ] show that even when we allow agents to use variable-length sequences to represent a message , they tend to utilize the maximum possible sequence to achieve the best performance ( in terms of communication success ) . The idea of using a Gumbel-softmax distribution [ 34 , 51 ] with the straight-through trick [ 7 ] for learning a language in multi-agent environment was concurrently proposed by Mordatch and Abbeel [ 56 ] and Havrylov and Titov [ 30 ] . They show that we can achieve a more stable and faster training by using this technique as compared to reinforcement learning used in several other approaches . Evaluating emergent communication . Evaluating the emergent language turns out to be an equally challenging research problem . Existing approaches use the successful completion of the task or the correlation between learned language and semantic labels as evaluation metrics . Lowe et al . [ 50 ] and Keresztury and Bruni [ 40 ] show that simple task success might not be a good or sufficient metric for evaluating the success of a game . They discuss heuristics and advocate measuring both positive signaling and positive listening independently to evaluate agents ’ communication . Andreas [ 1 ] provides a way of evaluating compositional structure in learned representations . Discrete Autoencoders . In parallel to the works on emergent communication , there is a large body of research on learning discrete representations of images using some form of autoencoding or reconstruction [ 20 , 28 , 29 , 57 , 58 , 62 , 63 , 65 ] without labels . The focus of VQ-VAE [ 58 ] and VQ-VAE-2 [ 63 ] is to learn a discrete bottleneck using vector quantization . Once we can represent any image with these discrete symbols , a powerful generative model such as PixelCNN [ 58 , 66 ] , or transformer [ 61 , 76 ] is learned on top of these symbols to sample new images . PatchVAE [ 28 ] achieves the same using Gumbel-Softmax and imposes an additional structure in the bottleneck of VAEs . We argue that because of mismatch in the objectives of reconstruction and visual recognition tasks , each of these models trained using reconstruction-based losses do not capture meaningful representations in the symbols . Self-supervised learning in vision . Self-supervised learning ( SSL ) methods , such as [ 10–13 , 26 , 32 , 55 , 81 ] have shown impressive results in recent years on downstream tasks of classification and object detection . Even though the bottleneck in these methods is continuous ( and not discrete symbols ) , these methods have been shown to capture semantic and spatial information of the contents of the image . Unlike SSL methods , neural networks representing the two agents in our case , do not share any weights . Also , note that the continuous nature of representations learned by SSL techniques is fundamentally different from the symbolic representations used in language . And indeed , we show that a k-Nearest Neighbor classifier obtained from the continuous representations learned by SSL methods can perform better than the one obtained using Bag of Words ( or symbols ) . However , to the best of our knowledge , our work is one of the first attempts to make representations learned in a self-supervised way more communication- or language-oriented . Comparison with Mihai and Hare [ 52 , 53 ] . [ 52 , 53 ] extends Havrylov and Titov [ 30 ] by training a speaker and listener agents end-to-end without using pre-trained networks . However , the prior works [ 30 , 52 , 52 ] use a top-down approach to generate discrete representation ( or a sentence ) for an image , i.e. , they compute an overall continuous embedding of the image and then proceed by generating one symbol of the sentence at a time using an LSTM . The computational cost of LSTMs is prohibitive when length of a sentence is large , which is needed to describe complex images . The transformers , on the other hand , require constant time for variable length sentences at the cost of increased memory ( in the listener agent ) . However , generating the variable length sentences with the speaker agent using transformers is non-trivial . To solve this , we propose a bottom-up approach , i.e. , we first generate symbols for image patches and combine them to form a sentence . This approach allows for computationally efficient end-to-end training . Further , it allows the speaker to compose symbols corresponding to different parts of the image , instead of deducing it from a pooled 1D representation of the image .

Summary: This paper introduces a referential game in which a speaker “sees” an image and sends a discrete message of variable length to a listener. The message is designed to represent important patches in the input image. In addition to the speaker’s message, the listener is given a batch of images, one of which is an augmented view of the speaker’s input. The listener’s goal is to find this augmented view of the input image among all images in the batch. The speaker and listener are trained with self-supervision using InfoNCE loss. The learned communication protocol is evaluated on the underlying task as well as used as a pre-training step for the downstream task of classification.