“ L az I mpa”: Lazy and Impatient neural agents learn to communicate efficiently (2020)

Abstract: Previous work has shown that artificial neural agents naturally develop surprisingly non-efficient codes. This is illustrated by the fact that in a referential game involving a speaker and a listener neural networks optimizing accurate transmission over a discrete channel, the emergent messages fail to achieve an optimal length. Furthermore, frequent messages tend to be longer than infrequent ones, a pattern contrary to the Zipf Law of Abbreviation (ZLA) observed in all natural languages. Here, we show that near-optimal and ZLA-compatible messages can emerge, but only if both the speaker and the listener are modified. We hence introduce a new communication system, “LazImpa”, where the speaker is made increasingly lazy, i.e., avoids long messages, and the listener impatient, i.e., seeks to guess the intended content as soon as possible.

## introduction recent emergent-communication studies, renewed by the astonishing success of neural networks, are often motivated by a desire to develop neural network agents eventually able to verbally interact with humans @xcite . to facilitate such interaction, neural networks' emergent language should possess many natural-language-like properties. however, it has been shown that, even if these emergent languages lead to successful communication, they often do not bear core properties of natural language @xcite @xcite . in this work, we focus on one basic property of natural language that resides on the tendency to use messages that are close to the informational optimum. this is illustrated in the zipf's law of abbreviation (zla), an empirical law that states that in natural language, the more frequent a word is, the shorter it tends to be @xcite @xcite . crucially, zla is considered to be an efficient property of our language @xcite . besides the obvious fact that an efficient code would be easier to process for us, it is also argued to be a core property of natural language, likely to be correlated with other fundamental aspects of human communication, such as regularity and compositionality @xcite . encouraging it might hence lead to emergent languages that are also more likely to develop these other desirable properties. despite the importance of such property, @xcite showed that standard neural network agents, when trained to play a simple signaling game @xcite , develop an inefficient code, which even displays an anti-zla pattern. that is, counterintuitively, more frequent inputs are coded with longer messages than less frequent ones. this inefficiency was related to neural networks' "innate preference" for long messages. in this work, we aim at understanding which constraints need to be introduced on neural network agents in order to overcome their innate preferences and communicate efficiently, showing a proper zla pattern. to this end, we use a reconstruction game where we have two neural network agents: speaker and listener. for each input, the speaker outputs a sequence of symbols (which constitutes the message) sent to the listener. the latter needs then to predict the speaker's input based on the given message. also, similarly to the previous work, inputs are drawn from a power-law distribution. we first describe the experimental and optimization framework (see section 2). in particular, we introduce a new communication system called 'lazimpa', comprising two different constraints (a) laziness on the speaker side and (b) impatience on the listener side. the former constraint is inspired by the least-effort principle which is attested to be a ubiquitous pressure in human communication @xcite @xcite . however, if such a constraint is applied too early, the system does not learn an efficient system. we show that incrementally penalizing long messages in the cost function enables an early exploration of the message space (a kind of 'babbling phase') and prevents converging to an inefficient local minimum. the other constraint, on the listener side, relies on the prediction mechanism, argued to be important in language comprehension (e.g., @xcite , and is achieved by allowing the listener to reconstruct the intended input as soon as possible. we also provide a two-level analytical method: first, metrics quantifying the efficiency of a code; second, a new protocol to measure its informativeness (see section 3). applying these metrics, we demonstrate that, contrary to the standard speaker/listener agents, our new communication system 'lazimpa' leads to the emergence of an efficient code. the latter follows a zla-like distribution, close to natural languages (see sections 4.1 and 4.2). besides the plausibility of the introduced constraints, our new communication system is, first, taskand architecture-agnostic (requires only communicating with sequences of symbols), and second allows stable optimization of the speaker/listener. we also show how both listener and speaker constraints are fundamental to the emergence of a zla-like distribution, as efficient as natural language (see section 4.3). ## experimental framework we explore the properties of emergent communication in the context of referential games where neural network agents, speaker and listener, have to cooperatively communicate in order to win the game. speaker network receives an input @xmath0. the symbols of the message belong to a vocabulary @xmath1, s 2 , ..., s voc_size-1 , eos} of size voc_size where eos is the 'end of sentence' token indicating the end of speaker's message. listener network receives and consumes the message m. based on this message, it outputs î. the two agents are successful if listener manages to guess the right input (i.e., î = i). we make two main assumptions. first inputs are drawn from i following a power-law distribution, where i is composed of 1000 one-hot vectors. consequently, the probability of sampling the k th most frequent input is: in our analysis, we only consider the successful runs, i.e., the runs with a uniform accuracy strictly higher than 97% over all possible 1000 inputs. an emergent language consists then of the input-message mapping. that is, for each input @xmath2, we note its output m. by m, we define the set of messages m used by our agents after succeeding in the game. ## analytical method as zla is defined informally, we first introduce reference distributions for comparison. then, we propose some simple metrics to evaluate the overall efficiency of our emergent codes. eventually, we provide a simple protocol to analyze the distribution of information within the messages. ## results in this section, we study the code of our new communicative system, lazimpa, and compare it to the standard agents baseline and the different reference distributions. we show that lazimpa leads to near-optimal and zla-compatible languages. eventually, we demonstrate how both impatience and laziness are required to get human-level efficiency. all the quantitative results of the considered codes are gathered in ## conclusion we demonstrated that a standard communication system, where standard speaker and listener lstms are trained to solve a simple reconstruction game, leads to long messages, close to the maximal threshold. surprisingly, if these messages are long, lstm agents rely only on a small number of informative message symbols, located at the end. we then introduce lazimpa, a constrained system that consists of lazy speaker and impatient listener. on the one hand, lazy speaker is obtained by introducing a cost on messages length once the communication is successful. we found that early exploration of potentially long messages is crucial for successful convergence (similar to the exploration in rl settings). on the other hand, impatient listener aims to succeed at the game as soon as possible, by predicting speaker's input at each message's symbol. we show that both constraints are necessary for the emergence of a zla-like protocol, as efficient as natural languages. specifically, lazy speaker alone would fail to shorten the messages. we connect this to the importance of the impatience mechanism to locate useful information at the beginning of the messages. if the function of this mechanism is subject to a standing debate (e.g., @xcite , many prior works had pointed to its necessity to human language understanding (e.g., @xcite . we augment this line of works and suggest that impatience could be at play in the emergence of zla-obeying languages. however, if impatience leads to zla, it is not sufficient for human-level efficiency. in other words, efficiency needs constraints both on speaker and listener sides. our work highlights the importance of introducing the right pressures in the communication system. indeed, to construct automated agents that would eventually interact with humans, we need to introduce taskagnostic constraints, allowing the emergence of more human-like communication. moreover, while being general, lazimpa provides a more stable optimization compared to the unconstrained system. finally, this study opens several lines of research. one would be to investigate further the gap from optimality. indeed, while lazimpa emergent languages show human-level efficiency, they do not reach optimal coding. specifically, emergent languages still have non-informative symbols at the end of the messages. if these additional non-useful symbols drift the protocol from optimality, we encounter similar trend in human @xcite and animal communication @xcite . we leave the understanding of the role of these non-informative symbols and how we can reach optimal coding for future works. a second line of research would be to apply this system to other games or nlp problems and study how it affects other properties of the language such as regularity or compositionality.