Learning to Draw: Emergent Communication through Sketching

1 Introduction . Imagine you and a friend are playing a game where you have to get your friend to guess an object in the room by you sketching the object . No other communication is allowed beyond the sketched image . This is an example of a referential communication game . To play this game you need to have learned how to draw in a way that your friend can understand . This paper explores how artificial agents parameterised by neural networks can learn to play similar drawing games . More specifically , we reformulate the traditional referential game such that one agent draws a sketch of a given photo and the second agent guesses , based on the drawing , the corresponding photo from a set of images . Spurred by innovations in artificial neural networks , deep and reinforcement learning techniques , recent work in multi-agent emergent communication [ 4 , 17 , 19 , 29 , 37 ] pursues interactions in the form of gameplay between agents to induce human-like communication . Artificial communicating agents can collaborate to solve various tasks : image referential games with realistic visual input [ 19 , 28 , 29 ] , negotiation [ 2 ] , navigation of virtual environments [ 7 , 22 ] , reconstruction of missing input [ 4 , 17 ] and , more recently , drawing games [ 11 ] . The key to achieving the shared goal in many of these games is collaboration , and implicitly , communication . To date , studies on communication emergence in multi-agent games have focused on exploring a language-based communication channel , with messages represented by discrete tokens or token sequences [ 6 , 17 , 19 , 25 , 28 , 29 , 35 ] . However , these communication protocols can be difficult for a human to interpret [ 3 , 26 , 33 ] . In this work we propose a direct and potentially self-explainable means of transmitting knowledge : sketching . Evidence suggests pre- and early-humans were able to communicate by drawing long before developing the various stages of written language [ 20 , 38 ] . Drawings such as petrograms and petroglyphs 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . exist from the oldest palaeolithic times and may have been used to record past experiences , events , beliefs or simply the relation with other beings [ 13 , 21 ] . These pictorial characters which are merely impressions of real objects or beings stand at the basis of all writing [ 16 ] . This leads us to question if drawing is a more natural way of starting to study emergent communication and if it could lead to better written communication later on . This idea has recently gained interest in several domains . In the cognitive science literature , neural models of sketching have been developed to study the factors which enable contextual flexibility in visual communication [ 8 ] . Likewise , works such as Fernando et al . [ 11 ] attempt to automate the artistic process of drawing by training agents in a reinforcement learning framework , to play a variety of drawing games . However , the focus of this paper is to open the doorway to exploring different types of communication between artificial agents and humans . The novelty of our work is also evident in the model framework which can be easily extended well beyond aspects previously studied . Concretely , we propose a visual communication channel in the context of image-based referential games . We leverage recent advances in differentiable sketching that enables us to construct an agent that can learn to communicate intent through drawing . Through a range of experiments we show that : • Agents can successfully communicate about real-world images through a sketching game . However , training with a loss that tries to maximise gameplay alone does not lead to human decipherable sketches , irrespective of any visual system preconditioning ; • Introducing a perceptual loss improves human interpretability of the communication protocol , at little to no cost in the gameplay success ; • Changes to the game objective , such as playing an object-oriented game , can steer the emergent communication protocol towards a more pictographic or symbolic form of expression ; • Inducing a shape-bias into the agents ’ visual system leads to more explainable drawings ; • A drawing agent trained with a perceptual loss can successfully communicate and play the game with a human . 2 Communication between agents . Communication emerges when two or more participants are involved , share a goal , task or incentive which can be achieved only by transfer of information and so is beneficial for all parties involved . Studies on language origins [ 36 , 41 ] consider cooperation to be a key prerequisite to language evolution as it implies multiple agents having to self-organise and adapt to the same convention . Studies on the emergence of communication in cooperative multi-agent environments from recent years have focused on ( natural ) language learning [ 28 , 29 ] and its inherent properties such as compositionality and expressivity [ 17 , 18 , 37 ] . A number of works specifically relate to the overarching ideas of gameplay and learning in this paper . For example , Foerster et al . [ 12 ] proposed a framework for differentiable training of communicating agents which was later used by Jorge et al . [ 23 ] to solve image search tasks with two interacting agents communicating with atomic symbols . Lazaridou et al . [ 28 ] proposed an image-based referential game in which the agents again communicated using atomic symbols , and were trained using policy gradients . Havrylov and Titov [ 19 ] and Mordatch and Abbeel [ 35 ] both demonstrated that it was possible to use differentiable relaxations to train agents that communicated with sequences of symbols . In the former case , the agents played the referential game that we adopt for our experiments . One of the long-term goals of this research in language emergence is to develop interactive machines that can productively communicate with humans . As such we should ensure that whatever language artificial agents develop , it is one that human agents can understand . In our work , we take inspiration from the process and evolution of writing . Written language has undergone many transitions from early times to reach the forms we now know : from pictures and drawings to word-syllabic , syllabic and , finally , alphabetic systems . In the beginning , our early ancestors did not know how to communicate in writing . Instead , they began drawing and painting pictures of their life , representing people and things they knew about [ 16 ] . Studies on the communication systems developed in primitive societies compare ancient drawings to the very early sketches drawn by children and talk about their tendency of concretely identifying certain things or events in their surrounding world [ 16 , 24 ] . Psychological and behavioural studies have shown that children try to communicate to the world through the images they create even when they can not associate them with words [ 9 ] . 3 A model for learning to communicate by drawing . We present a model consisting of two agents , the sender and the receiver in which the sender learns to draw by playing a game with the receiver . The overall architecture of the agents in the context of the game they are learning to play is shown in Figure 1 . Full code for the model and all experiments can be found at https : //github.com/Ddaniela13/LearningToDraw . 3.1 The Game Environment . Our experimental setup builds upon the image referential game previously explored in studies of emergent communication [ 19 , 28 , 29 ] that derives from Lewis ’ s signalling game [ 30 ] . We implemented several variants of Havrylov and Titov [ 19 ] ’ s image guessing game . The overall setting of these games is formulated as follows : 1 . Two target photographs , Ps and Pr , and set of K distractor photographs , { P ( k ) d } Kk=1 , are selected . 2 . There are two agents : a sender and a receiver . 3 . After being presented the Ps target image , the sender has to formulate a message conveying information about that image . 4 . Given the message and the set of photographs , { P ( k ) d } Kk=1 ∪ { Pr } , consisting of all the distractors and the target Pr , the receiver has to identify the target correctly . The specifics of how the photographs are selected ( step 1 above ) depend on the game variant as described below . Success in these games is measured by the binary ability of the receiver to correctly guess the correct image or not ; as such , the measure of communication rate is used to assess averaged performance over many games using independent images to those used during training . Unlike Havrylov and Titov [ 19 ] ’ s game in which the sender helps the receiver identify the correct image by sending a message constructed as a sequence of tokens drawn from a predefined vocabulary , we propose using a directly interpretable means of communication : sketching the target photograph . Original game variant . In Havrylov and Titov [ 19 ] ’ s variant of the game there is a pool of photos from which the distractors and target Ps are drawn randomly without replacement . The target Pr is set to be equal to Ps . In our original variant experiments the number of distractors , K , is set to 99 . Object-oriented game variants . In addition to the original setup , we explored two slightly different and potentially harder game configurations which were intended to induce the agents to draw sketches that would be more representative to the object class they belong to rather than to the specific instance of the class . These setups use labelled datasets where each image belongs to a class based on its contents . In the first of these variants ( we refer to this as OO-game same ) , the target Pr is set to be equal to Ps , and the distractors and target are sampled such that their class labels are disjoint ( that is every photo provided to the receiver has a different class ) . The second setup ( OO-game different ) is similar to the first , but the target Pr is chosen to be a different photograph with the same class label as target Ps . The intention behind these games is to explore a universally interpretable depiction of the different object classes , which does not focus on individual details but rather conveys the concept . To some extent , this task is an example of multiple instance classification within a weakly supervised setting [ 1 ] , which has been previously explored in the emergent communication literature [ 28 ] . 3.2 Agents ’ Architectures . Both agents act on visual inputs . The agents are parameterised by deep neural networks and are trained using standard gradient techniques ( Section 3.3 ) . The agent ’ s early visual system . We choose to model the early visual systems of both agents with the head part of the VGG16 CNN architecture [ 40 ] through to the ReLU activation at the end of the last convolutional layer ( commonly referred to as the ReLU5_3 layer ) before the final max-pooling and fully connected layers . In all experiments , we utilise pretrained weights and freeze this part of the model during training . We justify this choice on the basis that it provides the agents with an initial grounding in understanding the statistics of the visual world , and ensures that the visual system can not collapse and remains universal . The weights are the standard torchvision ImageNet weights , except in the cases where we explore the effect of shape bias ( see Section 4.5 ) . As these pretrained weights were learned with images that were normalised according to the ImageNet statistics , all inputs to the VGG16 backbone ( including sketches ) are normalised accordingly . The output feature maps of this convolutional backbone are flattened and are linearly projected to a fixed dimensional vector encoding ( 64-dimensions unless otherwise specified ) . Because the datasets used in gameplay have different resolutions , the number of weights in the learned projection varies . Sender Agent . The goal of the sender is to produce a sketch from the input photograph . For experiments in Section 4 , we restrict the production of sketches to be a drawing composed of 20 black , constant width , straight lines on a white canvas of the same size as the input images . Experiments with fewer lines can be found in Appendix A . It is of course possible to have a much more flexible definition of a sketch and incorporate many different modelling assumptions . We choose to leave such exploration for future work and focus on the key question of whether we can actually achieve successful ( and potentially interpretable ) communication with our simplified but not unrealistic setup . Given an input image , the agent ’ s early visual system produces a vector encoding which is then processed by a three-layer multilayer perceptron ( MLP ) that learns to decode the primitive parameters used to draw the sketch . This MLP has ReLU activations on the first two layers and tanh activation on the final layer . Unless otherwise specified , the first two layers have 64 and 256 neurons respectively . The output layer produces four values for each line that will be drawn ; the values are the start and end coordinates of each line stroke in an image canvas with the origin at the centre and edges at ±1 . To produce a sketch image from the line parameters output by the MLP , we utilise the differentiable rasterisation approach introduced by Mihai and Hare [ 34 ] . At a high level , this approach works by computing the distance transform on a pixel grid for each primitive being rendered . A relaxed approximation of a rasterisation function is applied to the distance transform to compute a raster image of the specific primitive . Finally , a differentiable composition function is applied to compose the individual rasters into a single image . More specifically , the squared Euclidean Distance Transform is computed , D2seg ( s , e ) over all pixels in the image , for each line segment starting at coordinate s and ending at e. These squared distance transforms are simply images in which the value of each pixel is replaced with the closest squared distance to the line ( computed when the pixels are mapped to the same coordinate system as the line — so the top left of the image is ( −1 , −1 ) and bottom-right is ( 1 , 1 ) ) . Using the subscript i to refer to the i-th line in the sketch , each D2seg ( si , ei ) is rasterised as Ri = exp Ç − D2seg ( si , ei ) ) σ2 å , ( 1 ) where σ2 is a hyperparameter that controls how far gradients flow in the image , as well as the visible thickness of the line ( σ2 = 5× 10−4 for all experiments in this paper ) . We adopt the soft-or composition function [ 34 ] to compose the individual line rasters into a single image , but incorporate an inversion so that a sketch image , S , with a white canvas and black lines is produced , S = n∏ i=1 ( 1−Ri ) , ( 2 ) where n is the number of lines . Finally , because the backbone CNNs work with three-band colour images , we replicate the greyscale sketch image three times across the channel dimension . Receiver Agent . The receiver agent is given a set of photographs and a sketch image , and is responsible for predicting which photograph matches the sketch under the rules of the specific game being played . The receiver ’ s visual system is coupled with a two-layer MLP with a ReLU nonlinearity on the first layer ( the latter layer has no activation function ) . Unless otherwise specified , all experiments use 64 neurons in the first layer and 64 in the final layer . The sketch image and each photograph are passed through the visual system and MLP independently to produce a feature vector representation of the respective input . A score vector x is produced for the photographs by computing the scalar product of the sketch feature with the feature of each respective photograph . This score vector is un-normalised but could be viewed as a probability distribution by passing it through a softmax . The photograph with the highest score is the one predicted .

Summary: I’m impressed by the authors’ inclusion of a human evaluation. While they paint a somewhat unclear story about human interpretability (hard to tell how good 38% is), including the results in commendable and these kinds of studies are sorely missing from the literature.  I’ve increased my score accordingly. I’m borderline on increasing my score even further: I think I would have liked to see some of the other experiments I suggested in the initial review to really nail down the interpretability idea more completely. But still, this is a very solid paper now—kudos to the authors!  ---  This paper is a study of emergent communication in an image-based referential game where the communication channel between two agents is not discrete sequences of tokens, but instead *sketches* of images; in other words, the agents learn to communicate via drawing. The speaker uses a recently-proposed differentiable drawing procedure, where line segments generated by an agent are combined and converted to a grayscale image via a relaxed rasterization process. The listener then receives the rasterized drawing and needs to identify the speaker's image from a set of distractor images. The entire procedure is end-to-end differentiable.  Optimizing solely communication success unsurprisingly results in non-interpretable sketching. To encourage the communication to be more interpretable, authors propose adding an auxiliary perceptual loss, where the speaker's drawings are optimized not just for listener success, but also trained to be perceptually similar to the target image, by minimizing distance between their representations according to the speaker's trained vision module. This results in sketches that often capture the rough details of the target image, with little-to-no degradation in communication success.  The authors try several additional experiments, e.g. varying the width of the communication channel, using different pretrained backbones that are more biased towards shape, etc. Many of these results do seem to have some effect, but the comparisons between models are limited to a qualitative comparison between 5 example images, so the conclusions we can gain from these experiments are a bit limited.