Interpretable agent communication from scratch (with a generic visual processor emerging on the side)

1 Introduction . As deep networks become more effective at solving specialized tasks , there has been interest in letting them develop a language-like communication protocol so that they can flexibly interact to address joint tasks [ 1 ] . One line of work within this tradition has focused on what is arguably the most basic function of language , namely to point out , or refer to , objects through discrete symbols . Such ability would for example allow deep-net-controlled agents , such as self-driving cars , to inform each other about the presence and nature of potentially dangerous objects , besides being a basic requirement to support more advanced capabilities ( e.g. , denoting relations between objects ) . While discreteness is not a necessary prerequisite for agent communication [ 2 , 3 ] , practical and ethical problems might arise if communication is incomprehensible to humans . A discrete code analogous to language is certainly easier to decode for us , helping us to understand the agents ’ decisions , and ultimately contributing to the larger goal of explainable AI [ 4 ] . In this paper , we study emergent discrete referential communication between two deep network agents that are trained from scratch on the task . We observe that the referential discrimination task played by the networks is closely related to pretext contrastive objectives used in self-supervised visual representation learning [ 5–8 ] . We exploit this insight to develop a robust end-to-end variant of a communication game . Our experiments confirm that , in our setup : i ) the nets develop a set of discrete symbols allowing them to successfully discriminate objects in natural images , including novel ones that were not shown during training ; ii ) these symbols denote partially interpretable categories , so that their emergence can be seen as a first step towards fully unsupervised image annotation ; iii ) the visual representations induced as a by-product can be used as high-quality general-purpose features , 1Code : https : //github.com/facebookresearch/EGG/tree/master/egg/zoo/emcom_as_ssl . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . whose performance in various object classification tasks is not lagging much behind that of features induced by a popular self-supervised representation method specifically designed for this task . 2 Background . Deep net emergent communication There has recently been interest in letting deep nets communicate through learned protocols . This line of work has addressed various challenges , such as communication in a dynamic environment or how to interface the emergent protocol with natural language ( see [ 1 ] for a survey ) . Probably the most widely studied aspect of emergent communication is the ability of deep net agents to use the protocol to refer to objects in their environment [ e.g. , 9–16 ] . The typical setup is that of a referential , or discriminative , communication game . In the simplest scenario , which we adopt here , an agent , the Sender , sees one input ( the target ) and it sends a discrete symbol to another agent , the Receiver , that sees an array of items including the target , and has to point to the latter for communication to be deemed successful . Importantly , task success is the only training objective ; the communication protocol emerges purely as a by-product of game-playing , without any direct supervision on the symbol-transmission channel . In one of the earliest papers in this line of research , Lazaridou et al . [ 9 ] used images from ImageNet [ 17 ] as input to the discrimination game ; Havrylov and Titov [ 11 ] used MSCOCO [ 18 ] ; and Evtimova et al . [ 12 ] used animal images from Flickr . While they relied on natural images , all these studies were limited to small sets of carefully selected object categories . Moreover , in all these works , the agents processed images with convolutional networks pretrained on supervised object recognition . While this sped up learning , it also meant that all the proposed systems de facto relied on the large amount of human annotated data used for object recognition training . Lazaridou et al . [ 13 ] and Choi et al . [ 14 ] dispensed with pre-trained CNNs , but they used synthetically generated geometric shapes as inputs . Results on the interpretability of symbols in games with realistic inputs have generally been mixed . Indeed , Bouchacourt and Baroni [ 19 ] showed that , after training Lazaridou et al . [ 9 ] ’ s networks on real pictures , the networks could use the learned protocol to successfully communicate about blobs of Gaussian noise , suggesting that their code ( also ) denoted low-level image features , differently from the general semantic categories that words in human language refer to . In part for this reason , recent work tends to focus on controlled symbolic inputs , where it is easier to detect degenerate communication strategies [ e.g. , 10 , 15 , 16 ] . Learning discrete latent representations with variational autoencoders Like emergent communication , variational autoencoders and related techniques have been used to induce discrete variables without direct supervision [ 20–22 ] . There are however important differences stemming from the fact that this research line is interested in inducing latent representations to be used in tasks such as image generation , whereas the goal of emergent communication is to induce discrete symbols for inter-agent coordination . As a result , for example , discrete representations derived with variational autoencoders are typically of much higher dimensionality , making a direct comparison in terms of interpretability difficult . Still , it is remarkable that the goal of learning discrete representations in an unsupervised way independently arose in different fields , and future work should explore connections between these ideas . Self-supervised representation learning Self-supervised learning of general-purpose visual features has received much attention in recent years . The main idea is to train a network on a pretext task that does not require manual annotation . After convergence , the net is used to extract high-quality features from images , to be applied in various “ downstream ” tasks of interest . This is often done by training a simple classifier on top of the frozen trained architecture [ 23–26 ] . Early models used image-patch prediction as the proxy task [ 8 , 27 ] . Recent work has instead focused on an instance-level contrastive discrimination objective [ 5–7 , 28 ] . Two symmetric networks encode different views of the same input images obtained through a stochastic data augmentation pipeline . Optimization is done with variants of the InfoNCE loss [ 27 , 29 ] , that maximizes similarity among representations of the same image while minimizing similarity of different ones . Interestingly , the contrastive pretext task is very close to the one of identifying a target image among distractors , as in the standard emergent communication referent discrimination game . The influential SimCLR model proposed by Chen et al . [ 5 ] is particularly similar to our setup . It uses two twin networks with a shared convolutional module optimizing the ( dis ) similarity of sets of target/distractor images . The main conceptual differences are that there is no discrete bottleneck imposed on “ communication ” between the networks , and there is no asymmetry , so that both networks act simultaneously as Sender and Receiver ( both networks produce a continuous “ message ” that must be as discriminative for the other network as possible ) . We rely here on the connection with self-supervised learning in two ways . First , we import the idea of data augmentation from this literature into the communication game , showing how it helps in evolving a more semantically interpretable protocol . Second , we evaluate the discrimination game as a self-supervised feature extraction method . We find that the visual features induced by the CNNs embedded in our agents are virtually as good as those induced by SimCLR , while the emergent protocol is better for communication than the one obtained by adapting SimCLR to the discrete communication setup . 3 Setup . 3.1 The discrimination game . A Sender network receives as input a target picture , and it produces as output one of |V | symbols . A Receiver network receives in input this symbol , as well as a list of n pictures , one of them ( randomly placed in the ith position of the list ) being the same target presented to the Sender . Receiver produces a probability distribution of cardinality n , interpreted as its guess over the position of the target . The guess is correct iff Receiver concentrates the largest probability mass on the ith position , corresponding to the target slot . Agent architecture Agent architecture and game flow are schematically shown in Fig . 1 . Sender reads the target image through a convolutional module , followed by a one-layer network mapping the output of the CNN onto |V | dimensions and applying batch normalization [ 30 ] , to obtain vector v. Following common practice when optimizing through discrete bottlenecks , we then compute the Gumbel-Softmax continuous relaxation [ 31 , 32 ] , which was shown to also be effective in the emergent communication setup [ 11 ] . At train time , Sender produces an approximation to a one-hot symbol vector with each component given by mi = exp [ ( si+vi ) /τ ] ∑ j exp [ ( sj+vj ) /τ ] , where si is a random sample from Gumbel ( 0,1 ) and vi a dimension of v. The approximation is controlled by temperature parameter τ : as τ approaches 0 , the approximation approaches a one-hot vector , and as τ approaches +∞ , the relaxation becomes closer to uniform . Importantly , at test time the Sender ’ s output is generated by directly argmax-ing v , so that it is a discrete one-hot vector indexing one of |V | possible symbols . Receiver passes each input image through its visual module ( a CNN architecture ) , followed by a two-layer MLP with batch normalization and ReLU after the first layer [ 33 ] . It then computes temperature-weighted cosine scores for the linearly embedded symbol compared to each image representation . The resulting vector of cross-modal ( symbol-image ) similarities is transformed into a probability distribution over which image is the likely target by applying the softmax operation . For both Sender and Receiver , we use ResNet-50 [ 34 ] as visual module . As they are different agents , that could ( in future experiments ) have very different architectures and interact with further agents , the most natural assumption is that each of them does visual processing with its own CNN ( no weight sharing ) . We consider however also a setup in which the CNN module is shared ( closer to earlier emergent-communication work , where the agents relied on the same pre-trained CNN ) . Optimization Optimization is performed end-to-end and the error signal , backpropagated through Receiver and Sender , is computed using the cross-entropy cost function by comparing the Receiver ’ s output with a one-hot vector representing the position of the target in the image list . SimCLR as a comparison model Given the similarity between the referential communication game and contrastive self-supervised learning in SimCLR [ 5 ] , we use the latter as a comparison point for our approach . Fig . 2 schematically shows the SimCLR architecture . The crucial differences between SimCLR and our communication game are the following : i ) In SimCLR , the agents are parameterized by the same network , that is , the visual encoder and transformation modules in the two branches of Fig . 2 are instances of the same net . ii ) The setup is fully symmetric . Like our Receiver , both agents get a set of images in input , and , like our Sender , both agents can be seen as producing “ messages ” representing each input image . iii ) Instead of ( a probability distribution over ) symbols , the exchanged information takes the form of continuous vectors ( s in the figure ) . iv ) The loss is based on directly comparing embeddings of these continuous vectors ( z in the figure ) , maximizing the similarity between pairs representing the same images ( positive examples in contrastive-loss terminology ) and minimizing that of pairs representing different images ( negative examples ) . This differs from our loss , that maximizes the similarity of the Receiver embedding of the Sender-produced discrete symbol with its own representation of the target image , while minimizing the similarity of the symbol embedding with its representation of the distractors . It is important to stress the different roles that SimCLR as a comparison model will play in the experiments below . When playing the communication game ( Section 4.1 ) , our discretized SimCLR method must be simply seen as an interesting baseline , as the system was not designed for discrete communication in the first place ( it is indeed interesting that it performs as well as it does ) . When performing protocol analysis ( Section 4.2 ) , the discrete clusters explicitly built on SimCLR representations act as a challenging comparison point for the categories implicitly induced by our system through game playing . Finally , when evaluating the visual features learnt by the two systems ( Section 4.3 ) , the roles are inverted , with SimCLR being a standard method developed for these purposes , whereas in our setup the emergence of good visual representations is a by-product of communication-based model training . Data augmentation In the original discrimination game proposed in [ 9 ] , the agents are shown exactly the same target image.2 In self-supervised learning , on the other hand , it is common practice to “ augment ” images in different ways , e.g. , by applying different croppings or color perturbations 2Lazaridou and colleagues [ 9 ] also considered a variant of the game in which the agents see different pictures of the same category ( e.g. , the shared target is dog , but the agents get different dog pictures ) . This version of the game is however severely limited by the requirement of manual category annotation . Lazaridou et al . [ 13 ] also provide different images to Sender and Receiver , by feeding them different viewpoints of the same synthetically generated objects : again , a strategy that will not scale up to natural images . [ 5 , 35 , 36 ] . In standard contrastive learning frameworks , where all the weights are shared and there is no communication bottleneck , it is necessary to create these different views , or else the system would trivially succeed at the pretext contrastive task without any actual feature learning . We conjecture that data augmentation , while not strictly needed , might also be beneficial in the communication game setup : presenting different views of the target to Sender and Receiver should make it harder for them to adopt degenerate strategies based on low-level image information [ 19 ] . We follow the same data augmentation pipeline as [ 5 ] , stochastically applying crop-and-resize , color perturbation , and random Gaussian blurring to every image . Note that , for the experiments reported in the main paper , we do not apply data augmentation at test time . Results with augmentation also applied at test time are reported in Appendix A.4 . Implementation details All hidden and output layers are set to dimensionality 2048.3 Note that this implies |V | = 2048 , more than double the categories in the dataset we use to train the model ( see Section 3.2 below ) , to avoid implicit supervision on optimal symbol count.4 We fix Gumbel-Softmax temperature at 5.0 , and Receiver cosine temperature at 0.1 . The latter value is also used for the equivalent τ parameter in the NTXent-loss of our SimCLR implementation . We train with mixed precision [ 37 ] for 100 epochs , with a batch of size 16× 128 = 2048 , divided across 16 GPUs . Rather than sampling distractors from the entire dataset , we take them from the current device ’ s batch , thus playing the communication game with 127 distractor images in all reported experiments.5 We do not share distractors ( negative samples ) across devices . As in SimCLR , we use the LARS optimizer [ 38 ] with linear scaling [ 39 ] , resulting in an initial learning rate of 2.4 . We apply a cosine decay schedule without warmup nor restart [ 40 ] . Compute requirements are reported in Appendix A.1 . All models are implemented with the EGG toolkit [ 41 ] .

Summary: This paper studies the visual features emerging from a discrete referential game between networks. A sender network processes the input image into a discrete symbol; the receiver is tasked to identify the input image from discrators images using the encoded discrete symbol. The authors study the emerging features based on their referential accuracy, interpretability, and quality as unsupervised pre-trained features.