The Emergence of High-Level Semantics in a Signaling Game (2024)

Abstract: The symbol grounding problem—how to connect a symbolic system to the outer world—is a longstanding question in AI that has recently gained prominence with the progress made in NLP in general and surrounding large language models in particular. In this article, we study the emergence of semantic categories in the communication protocol developed by neural agents involved in a well-established type of signaling game. In its basic form, the game requires one agent to retrieve an image based on a message produced by a second agent. We first show that the agents are able to, and do, learn to communicate high-level semantic concepts rather than low-level features of the images even from very indirect training signal to that end. Second, we demonstrate that the introduction of an adversarial agent in the game fosters the emergence of semantics by producing an appropriate training signal when no other method is available.

## introduction how would it be possible to acquire and represent the meaning of words, not simply their function in language but also their connection to the outer world? a cogent account of this question, known as the problem of symbol grounding, is that of @xcite . in the case where all we ever have access to is pure linguistic data, harnad likens the question of attributing meaning representations to a neverending chain of dictionary look-ups. harnad's approach to circumvent this problem is to require agents to deal with iconic and categorical representations, in addition to manipulating symbols. iconic representations are nonsymbolic representations of perceptual inputs; categorical representations are nonsymbolic representations of categories or concepts. together, they form the basis of the interface between the agent's symbolic system and the outer world, and it is this interface that gives meaning to, or grounds, the symbols manipulated by the agent. since harnad's article, researchers in ai and nlp have often stressed supplementary requirements beyond perceptual data for the development of meaningful and grounded representations, mentioning embodiment (e.g., @xcite , intent (e.g., @xcite or interactions with other agents as well as the environment @xcite . in effect, there is a growing consensus that meaningful representations can only emerge in goal-driven interactive situations. the study of emergent communication is the study of how interacting agents (human or otherwise) can successfully establish effective communication protocols @xcite , and under which conditions this is possible. recently, much interest has been devoted to emergent communication between neural agents involved in signaling games (e.g., @xcite , in which the agents have to cooperate through information exchange in order to retrieve some target. such setups have the advantage that they can provide a very tight control on experimental conditions. in the present paper, we focus on a two-agents single-round signaling game, in which the two agents, playing the role of a sender and a receiver, are to cooperate by exchanging sequences of arbitrary symbols so that the receiver successfully retrieves an image based on one that was shown to the sender. we propose to study what conditions are necessary to the emergence of semantic categories in neural agents in this setting through two sets of experiments. it has been shown that under certain circumstances, neural agents trained in similar setups develop "trivial" strategies, describing low-level features of their input @xcite . accordingly, we hypothesize that in the absence of any form of pressure towards generalization capabilities, the agents will not tend towards conveying high-level information, but will rather settle on exchanging about low-level image-specific information. we test this assumption in our first set of experiments by contrasting emergent communication protocols in three different environments: in the first, agents have a direct training signal towards learning to communicate categorical information; in the second, an indirect signal is given, but categorical information is not necessary to solve the task at hand; in the third, agents have no explicit information about categories. to our surprise, we observe that, given enough (training) time, the agents in the second type of environments reliably pick up the indirect signal about the existence of categories and spontaneously shift from communicating low-level features to high-level information (even though they are equally useful to solve the training task). in the third type of environments, semantic categories might be recovered but to a much lesser extent. this leads us to our second set of experiments, where we study whether a category-level training signal can be synthesized by introducing an adversarial agent. this adversary aims to exploit the message sent by the sender to fool the receiver, and thereby implicitly guides the sender away from communicating information that is too easily falsifiable. we observe that introducing such an adversarial agent in the game can significantly bolster the emergence of high-level semantics in the agents' communication. ## related works grounding, viz., how to relate the symbols of a symbolic system (e.g., a language) to other aspects of the world, has been a fecund domain of research over the past decades. in particular, @xcite provides an insightful thought experiment, inspired by searle's controversial chinese room argument, and aimed at showing the necessity of grounding: "suppose you had to learn chinese as a first language and the only source of information you had was a chinese/chinese dictionary! this is more like the actual task faced by a purely symbolic model of the mind" (pp.339-40). he also outlines a cogent program towards practical implementations of grounded hybrid systems, involving trained nonsymbolic input and categorical representations interfacing a symbolic system with the outer world. more recent discussions on this concept have been written by @xcite , who emphasize the role of speakers' intent, or @xcite , who stresses the importance of embodied usages of symbols. note, however, that it has been shown that some structures of the outer world can be found in the topology of the embedding space of ungrounded language models (e.g., @xcite with color terms). there is now sustained interest in establishing if and how symbol grounding can occur within modern large language models, and to what extent their productions match our expectations for situated, intentional and semantically coherent communication @xcite @xcite . many works focus on harnessing the boons that come with systems handling multiple channels of inputs, be it to create generalist agents (e.g., @xcite , to enrich their inputs (e.g., @xcite , or to facilitate human-robot interactions (e.g., @xcite . however, practitioners of nlp rarely study the multi-agent aspects of grounding @xcite , despite them being outlined as a crucial component; @xcite goes as far as stating that standard supervised learning alone, possibly involving multiple modalities but without proper agent-agent or agent-environment interaction, cannot solve the symbol grounding problem. at the same time, there is also a growing interest in using multimodal neural networks as models of how perceptual information is used in humans (esp. @xcite ; this line of work could therefore benefit from developments on multi-agents nlp system. in that respect, previous works that include simulations of how language and communication can emerge @xcite is especially useful in that they provide data and define a framework to test hypotheses related to symbol grounding. these works generally involve multiple agents negotiating the use of symbols in order to solve a task through the interaction with nonlinguistic data. while prior work has studied multi-turn communication (a.o., @xcite , populations and generations of agents (e.g., @xcite @xcite or nonsymbolic communication channels (e.g., @xcite , we focus on a straightforward signaling game @xcite involving multiple agents communicating through a symbolic channel @xcite @xcite . more precisely, our starting point is the setup of @xcite , where we introduced a computer-generated image dataset, studied the impact of many design choices of the learning process (pertaining to the loss function and regularization, the selection of training instances, and pretraining methods) on a two-agent signaling game, and defined several metrics used to study the properties of the emergent languages. the work of @xcite is close to ours in that they study how the choice of training instances in a signaling game can improve the systematicity of the emergent languages. however, they mainly do so by explicitly strengthening the training signal pertaining to semantic classes (through the use of sets of images instantiating these classes), while we try to achieve similar effects without relying on a priori known semantic classes. one novelty of the present work is the introduction of an adversary agent in the signaling game. relevant precedents in the literature include noncooperative language games, such as the competitive setup of @xcite . to our knowledge, the present work is the first to introduce a gan-like agent @xcite in an emergent communication setting. ## signaling game definition we start by presenting the basics of the signaling game that we study in this section. we document departures from this base setup where relevant. ## influence of target and distractor choice the goal of the present work is to establish what training signal is necessary for categorical information to appear in the communication protocols developed by the agents. this requires modifying they do take advantage of to some extent. in contrast, we are here interested in the agents communicating about categorylevel information as much as possible, and thus partition the categories differently: we take as generalization categories (cube, small, blue, down, left)-chosen arbitrarily-and all 10 categories differing from it on exactly three features. the environment in which the agents interact so that its categorical structure is more or less obvious. ## fostering the emergence of high-level semantics as we just saw, encoding category-level information systematically seems to require the agents to have access (directly or indirectly) to categorylevel information. we now turn to whether we can dispense from including this explicit information while retaining category-level information in the messages. spike (2017, §.5) suggests that noisy inputs can foster more robust and effective communication channels: adding noise to input images would prevent agents from communicating about very lowlevel information (e.g., specific pixel brightness), since this information may not match with what the receiver would perceive. such a procedure is therefore a natural candidate to explore. however, preliminary experiments involving the addition of normal noise to the images showed this technique to only make the training process less reliable, without any observable benefit. instead, we focus on a more involved approach: incorporating an agent playing an adversarial role to discourage the sender and receiver to exchange image-specific information. ## conclusions do agents learning to identify images through symbolic communication develop a language able to describe category-level features of these images? interestingly, indirect signal environments provide evidence that models are able to develop high-level semantics even when the only relevant training signal is extremely tenuous. the results of models in no signal environments suggest, however, that one cannot expect the sender to encode category-level information systematically without an appropriate training signal. our last experiment shows that even without relying on the availability of semantic categoriesas is often the case with natural images-, fostering the emergence of high-level semantics is possible via the introduction of an adversarial agent. in the future, we would be interested in studying whether this technique is effective on other datasets that the one used here, and in whether improvements of the (delicate) training procedure of the adversary may lead to a stronger impact on the emergent languages.