O n the S pontaneous E mergence of D iscrete and C ompositional S ignals (2020)

Abstract: We propose a general framework to study language emergence through signaling games with neural agents. Using a continuous latent space, we are able to (i) train using backpropagation, (ii) show that discrete messages nonetheless naturally emerge. We explore whether categorical perception effects follow and show that the messages are not compositional.

## introduction in a signaling game, artificial agents learn to communicate to achieve a common goal: a sender sees some piece of information and produces a message, which is then sent to a receiver that must take some action @xcite . if the action is coherent with the sender's initial piece of information, the choice of the message and its interpretation is reinforced. for instance, in a referential game, sender and receiver see a set of objects, and the sender knows which of these the receiver must pick; the sender then sends a message to the receiver, who must interpret it to pick up the right object @xcite @xcite . this setting has been used to study the factors influencing the emergence of various fundamental properties of natural language, such as compositionality @xcite @xcite @xcite . in this paper, we add focus on two other so-called 'design features' of natural language @xcite : discreteness (i.e. words form clusters in acoustic space), and displacement (i.e. efficient communication can occur about objects and facts beyond the immediate context of the conversation). from an implementation point of view, we follow the recent literature which has shown that a sig-naling game is essentially an autoencoder setting, with the encoder playing the role of the sender, and the decoder the role of the receiver (see fig. 1 ). in this literature, however, the discreteness of the communication protocol is assumed, since the networks then traditionally use a (normally sequential and) discrete latent space @xcite @xcite . our main contribution is a generalization of the current implementation of signaling games as autoencoders. our implementation covers a broader variety of signaling games, and it crucially incorporates the possibility of displacement and makes no a priori assumption of discreteness. our main result is that under appropriate conditions, discreteness emerges spontaneously: if the latent space is thought about as a continuous acoustic space, then trained messages form coherent clusters, just like regular words do. we also show that the messages are not compositional. in addition to contributing to our understanding of the emergence of communication protocols with features like natural language, our results have technical significance: by using a continuous communication protocol, with discreteness spontaneously emerging, we can train end-to-end using standard backpropagation, instead of reinforcement learning algorithms like reinforce and its refinements @xcite @xcite , which are difficult to use in practice. ## related work a related line of work attempts to avoid the difficulties of reinforcement learning-used when there are stochastic nodes in a computation graphby reparameterization and/or non-stochastic estimators @xcite . in the emergent communication case, where the stochastic nodes are discrete (e.g. sampling a message from a sender distribution), the gumbel-softmax estimator has become increasingly popular @xcite . that work enables standard backpropagation to be used for training by optimizing approximations to the true reinforcement learning signal. by contrast, we do not approximate the discrete rl learning signal, but rather ask under what conditions discreteness will emerge. several earlier papers explore similar topics in the emergence of discrete symbols. @xcite show that the division of the acoustic space is an emergent property of language use under noise. it assumes that speakers have a fixed language and asks which such ones are stable. in our setting, the language itself is changing as the result of reinforcement from communication and transmission itself is not noisy. de @xcite simulates the emergence of vowel systems in artificial agents modeled after phonetic production and perception in humans, resulting in a self-discretizing acoustic space and a vowel system that resembles human ones. this makes the agents much closer to what we know about humans, but also limits its scope. results about emergent communication can tell us both about the emergence of human language, but also about communication protocols in general, that may be used by very different agents, e.g. autonomous ones, or animals @xcite . ## function games we here introduce a general communication game setting, which we call function games. our games contain three basic components: (i) a set of contexts c, (ii) a set of actions a, (iii) a family of functions f , from contexts to actions. one play of a function game game runs as follows: abstractly, the function f represents some piece of knowledge available primarily for sender, and which determines what action is appropriate in any given context. two concrete interpretations will help illustrate the variety of communication protocols and goals that this framework encompasses. ## experiment because we are interested in the simultaneous emergence both of discrete and of compositional signals, we use a function game called the extremity game designed to incentivize and test rich compositionality (steinert-threlkeld, 2018, 2020). in this game, one may think of the n dimensions of the objects as gradable properties, e.g. size and darkness, so that a 2d object is determined by a given size and shade of gray. for the functions, we set @xmath0, arg max i : 0 ≤ @xmath1. an emerging language may contain compositional messages like 'most + big', 'least + dark'. ## conclusion we propose a general signaling game framework in which fewer a priori assumptions are imposed on the conversational situations. we use both production and perception analyses, and find that under appropriate conditions, which are met by most studies involving neural signaling games, messages become discrete without the analyst having to force this property into the language (and having to deal with non-differentiability issues). we find no evidence of compositional structure using vector analogies and a generalization thereof but do find sharp boundaries between the discrete message clusters. future work will explore other measures and alternative game settings for the emergence of compositionality, as well as more subtle psychological effects (categeorical perception) of continuous biological systems exhibiting discrete structure, like the auditory system.