Interpretable agent communication from scratch (with a generic visual processor emerging on the side)

1 Introduction . As deep networks become more effective at solving specialized tasks , there has been interest in letting them develop a language-like communication protocol so that they can flexibly interact to address joint tasks [ 1 ] . One line of work within this tradition has focused on what is arguably the most basic function of language , namely to point out , or refer to , objects through discrete symbols . Such ability would for example allow deep-net-controlled agents , such as self-driving cars , to inform each other about the presence and nature of potentially dangerous objects , besides being a basic requirement to support more advanced capabilities ( e.g. , denoting relations between objects ) . While discreteness is not a necessary prerequisite for agent communication [ 2 , 3 ] , practical and ethical problems might arise if communication is incomprehensible to humans . A discrete code analogous to language is certainly easier to decode for us , helping us to understand the agents ’ decisions , and ultimately contributing to the larger goal of explainable AI [ 4 ] . In this paper , we study emergent discrete referential communication between two deep network agents that are trained from scratch on the task . We observe that the referential discrimination task played by the networks is closely related to pretext contrastive objectives used in self-supervised visual representation learning [ 5–8 ] . We exploit this insight to develop a robust end-to-end variant of a communication game . Our experiments confirm that , in our setup : i ) the nets develop a set of discrete symbols allowing them to successfully discriminate objects in natural images , including novel ones that were not shown during training ; ii ) these symbols denote partially interpretable categories , so that their emergence can be seen as a first step towards fully unsupervised image annotation ; iii ) the visual representations induced as a by-product can be used as high-quality general-purpose features , 1Code : https : //github.com/facebookresearch/EGG/tree/master/egg/zoo/emcom_as_ssl . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . whose performance in various object classification tasks is not lagging much behind that of features induced by a popular self-supervised representation method specifically designed for this task . 2 Background . Deep net emergent communication There has recently been interest in letting deep nets communicate through learned protocols . This line of work has addressed various challenges , such as communication in a dynamic environment or how to interface the emergent protocol with natural language ( see [ 1 ] for a survey ) . Probably the most widely studied aspect of emergent communication is the ability of deep net agents to use the protocol to refer to objects in their environment [ e.g. , 9–16 ] . The typical setup is that of a referential , or discriminative , communication game . In the simplest scenario , which we adopt here , an agent , the Sender , sees one input ( the target ) and it sends a discrete symbol to another agent , the Receiver , that sees an array of items including the target , and has to point to the latter for communication to be deemed successful . Importantly , task success is the only training objective ; the communication protocol emerges purely as a by-product of game-playing , without any direct supervision on the symbol-transmission channel . In one of the earliest papers in this line of research , Lazaridou et al . [ 9 ] used images from ImageNet [ 17 ] as input to the discrimination game ; Havrylov and Titov [ 11 ] used MSCOCO [ 18 ] ; and Evtimova et al . [ 12 ] used animal images from Flickr . While they relied on natural images , all these studies were limited to small sets of carefully selected object categories . Moreover , in all these works , the agents processed images with convolutional networks pretrained on supervised object recognition . While this sped up learning , it also meant that all the proposed systems de facto relied on the large amount of human annotated data used for object recognition training . Lazaridou et al . [ 13 ] and Choi et al . [ 14 ] dispensed with pre-trained CNNs , but they used synthetically generated geometric shapes as inputs . Results on the interpretability of symbols in games with realistic inputs have generally been mixed . Indeed , Bouchacourt and Baroni [ 19 ] showed that , after training Lazaridou et al . [ 9 ] ’ s networks on real pictures , the networks could use the learned protocol to successfully communicate about blobs of Gaussian noise , suggesting that their code ( also ) denoted low-level image features , differently from the general semantic categories that words in human language refer to . In part for this reason , recent work tends to focus on controlled symbolic inputs , where it is easier to detect degenerate communication strategies [ e.g. , 10 , 15 , 16 ] . Learning discrete latent representations with variational autoencoders Like emergent communication , variational autoencoders and related techniques have been used to induce discrete variables without direct supervision [ 20–22 ] . There are however important differences stemming from the fact that this research line is interested in inducing latent representations to be used in tasks such as image generation , whereas the goal of emergent communication is to induce discrete symbols for inter-agent coordination . As a result , for example , discrete representations derived with variational autoencoders are typically of much higher dimensionality , making a direct comparison in terms of interpretability difficult . Still , it is remarkable that the goal of learning discrete representations in an unsupervised way independently arose in different fields , and future work should explore connections between these ideas . Self-supervised representation learning Self-supervised learning of general-purpose visual features has received much attention in recent years . The main idea is to train a network on a pretext task that does not require manual annotation . After convergence , the net is used to extract high-quality features from images , to be applied in various “ downstream ” tasks of interest . This is often done by training a simple classifier on top of the frozen trained architecture [ 23–26 ] . Early models used image-patch prediction as the proxy task [ 8 , 27 ] . Recent work has instead focused on an instance-level contrastive discrimination objective [ 5–7 , 28 ] . Two symmetric networks encode different views of the same input images obtained through a stochastic data augmentation pipeline . Optimization is done with variants of the InfoNCE loss [ 27 , 29 ] , that maximizes similarity among representations of the same image while minimizing similarity of different ones . Interestingly , the contrastive pretext task is very close to the one of identifying a target image among distractors , as in the standard emergent communication referent discrimination game . The influential SimCLR model proposed by Chen et al . [ 5 ] is particularly similar to our setup . It uses two twin networks with a shared convolutional module optimizing the ( dis ) similarity of sets of target/distractor images . The main conceptual differences are that there is no discrete bottleneck imposed on “ communication ” between the networks , and there is no asymmetry , so that both networks act simultaneously as Sender and Receiver ( both networks produce a continuous “ message ” that must be as discriminative for the other network as possible ) . We rely here on the connection with self-supervised learning in two ways . First , we import the idea of data augmentation from this literature into the communication game , showing how it helps in evolving a more semantically interpretable protocol . Second , we evaluate the discrimination game as a self-supervised feature extraction method . We find that the visual features induced by the CNNs embedded in our agents are virtually as good as those induced by SimCLR , while the emergent protocol is better for communication than the one obtained by adapting SimCLR to the discrete communication setup . 3 Setup . 3.1 The discrimination game . A Sender network receives as input a target picture , and it produces as output one of |V | symbols . A Receiver network receives in input this symbol , as well as a list of n pictures , one of them ( randomly placed in the ith position of the list ) being the same target presented to the Sender . Receiver produces a probability distribution of cardinality n , interpreted as its guess over the position of the target . The guess is correct iff Receiver concentrates the largest probability mass on the ith position , corresponding to the target slot . Agent architecture Agent architecture and game flow are schematically shown in Fig . 1 . Sender reads the target image through a convolutional module , followed by a one-layer network mapping the output of the CNN onto |V | dimensions and applying batch normalization [ 30 ] , to obtain vector v. Following common practice when optimizing through discrete bottlenecks , we then compute the Gumbel-Softmax continuous relaxation [ 31 , 32 ] , which was shown to also be effective in the emergent communication setup [ 11 ] . At train time , Sender produces an approximation to a one-hot symbol vector with each component given by mi = exp [ ( si+vi ) /τ ] ∑ j exp [ ( sj+vj ) /τ ] , where si is a random sample from Gumbel ( 0,1 ) and vi a dimension of v. The approximation is controlled by temperature parameter τ : as τ approaches 0 , the approximation approaches a one-hot vector , and as τ approaches +∞ , the relaxation becomes closer to uniform . Importantly , at test time the Sender ’ s output is generated by directly argmax-ing v , so that it is a discrete one-hot vector indexing one of |V | possible symbols . Receiver passes each input image through its visual module ( a CNN architecture ) , followed by a two-layer MLP with batch normalization and ReLU after the first layer [ 33 ] . It then computes temperature-weighted cosine scores for the linearly embedded symbol compared to each image representation . The resulting vector of cross-modal ( symbol-image ) similarities is transformed into a probability distribution over which image is the likely target by applying the softmax operation . For both Sender and Receiver , we use ResNet-50 [ 34 ] as visual module . As they are different agents , that could ( in future experiments ) have very different architectures and interact with further agents , the most natural assumption is that each of them does visual processing with its own CNN ( no weight sharing ) . We consider however also a setup in which the CNN module is shared ( closer to earlier emergent-communication work , where the agents relied on the same pre-trained CNN ) . Optimization Optimization is performed end-to-end and the error signal , backpropagated through Receiver and Sender , is computed using the cross-entropy cost function by comparing the Receiver ’ s output with a one-hot vector representing the position of the target in the image list . SimCLR as a comparison model Given the similarity between the referential communication game and contrastive self-supervised learning in SimCLR [ 5 ] , we use the latter as a comparison point for our approach . Fig . 2 schematically shows the SimCLR architecture . The crucial differences between SimCLR and our communication game are the following : i ) In SimCLR , the agents are parameterized by the same network , that is , the visual encoder and transformation modules in the two branches of Fig . 2 are instances of the same net . ii ) The setup is fully symmetric . Like our Receiver , both agents get a set of images in input , and , like our Sender , both agents can be seen as producing “ messages ” representing each input image . iii ) Instead of ( a probability distribution over ) symbols , the exchanged information takes the form of continuous vectors ( s in the figure ) . iv ) The loss is based on directly comparing embeddings of these continuous vectors ( z in the figure ) , maximizing the similarity between pairs representing the same images ( positive examples in contrastive-loss terminology ) and minimizing that of pairs representing different images ( negative examples ) . This differs from our loss , that maximizes the similarity of the Receiver embedding of the Sender-produced discrete symbol with its own representation of the target image , while minimizing the similarity of the symbol embedding with its representation of the distractors . It is important to stress the different roles that SimCLR as a comparison model will play in the experiments below . When playing the communication game ( Section 4.1 ) , our discretized SimCLR method must be simply seen as an interesting baseline , as the system was not designed for discrete communication in the first place ( it is indeed interesting that it performs as well as it does ) . When performing protocol analysis ( Section 4.2 ) , the discrete clusters explicitly built on SimCLR representations act as a challenging comparison point for the categories implicitly induced by our system through game playing . Finally , when evaluating the visual features learnt by the two systems ( Section 4.3 ) , the roles are inverted , with SimCLR being a standard method developed for these purposes , whereas in our setup the emergence of good visual representations is a by-product of communication-based model training . Data augmentation In the original discrimination game proposed in [ 9 ] , the agents are shown exactly the same target image.2 In self-supervised learning , on the other hand , it is common practice to “ augment ” images in different ways , e.g. , by applying different croppings or color perturbations 2Lazaridou and colleagues [ 9 ] also considered a variant of the game in which the agents see different pictures of the same category ( e.g. , the shared target is dog , but the agents get different dog pictures ) . This version of the game is however severely limited by the requirement of manual category annotation . Lazaridou et al . [ 13 ] also provide different images to Sender and Receiver , by feeding them different viewpoints of the same synthetically generated objects : again , a strategy that will not scale up to natural images . [ 5 , 35 , 36 ] . In standard contrastive learning frameworks , where all the weights are shared and there is no communication bottleneck , it is necessary to create these different views , or else the system would trivially succeed at the pretext contrastive task without any actual feature learning . We conjecture that data augmentation , while not strictly needed , might also be beneficial in the communication game setup : presenting different views of the target to Sender and Receiver should make it harder for them to adopt degenerate strategies based on low-level image information [ 19 ] . We follow the same data augmentation pipeline as [ 5 ] , stochastically applying crop-and-resize , color perturbation , and random Gaussian blurring to every image . Note that , for the experiments reported in the main paper , we do not apply data augmentation at test time . Results with augmentation also applied at test time are reported in Appendix A.4 . Implementation details All hidden and output layers are set to dimensionality 2048.3 Note that this implies |V | = 2048 , more than double the categories in the dataset we use to train the model ( see Section 3.2 below ) , to avoid implicit supervision on optimal symbol count.4 We fix Gumbel-Softmax temperature at 5.0 , and Receiver cosine temperature at 0.1 . The latter value is also used for the equivalent τ parameter in the NTXent-loss of our SimCLR implementation . We train with mixed precision [ 37 ] for 100 epochs , with a batch of size 16× 128 = 2048 , divided across 16 GPUs . Rather than sampling distractors from the entire dataset , we take them from the current device ’ s batch , thus playing the communication game with 127 distractor images in all reported experiments.5 We do not share distractors ( negative samples ) across devices . As in SimCLR , we use the LARS optimizer [ 38 ] with linear scaling [ 39 ] , resulting in an initial learning rate of 2.4 . We apply a cosine decay schedule without warmup nor restart [ 40 ] . Compute requirements are reported in Appendix A.1 . All models are implemented with the EGG toolkit [ 41 ] .

Summary: The model aims to train a pair of two ConvNets in the setup of the so-called discrimination game -- one ConvNet (sender) can pass discrete information to the other ConvNet (receiver) to identify the same image presented to the Sender among a set of images. Generating such discrete (bottlenecked) information is learned in an end-to-end manner, which is claimed to provide a human-interpretable inter-ConvNet communication. Compared to SimCLR (Chen et al., ICML 2020), the proposed method shows better or matched performance in terms of discrimination game-playing accuracy and downstream image classification task.