Emergence of Hierarchical Reference Systems in Multi-agent Communication (2022)

Abstract: In natural language, referencing objects at different levels of specificity is a fundamental pragmatic mechanism for efficient communication in context. We develop a novel communication game, the hierarchical reference game, to study the emergence of such reference systems in artificial agents. We consider a simplified world, in which concepts are abstractions over a set of primitive attributes (e.g., color, style, shape). Depending on how many attributes are combined, concepts are more general (“circle”) or more specific (“red dotted circle”). Based on the context, the agents have to communicate at different levels of this hierarchy. Our results show that the agents learn to play the game successfully and can even generalize to novel concepts. To achieve abstraction, they use implicit (omitting irrelevant information) and explicit (indicating that attributes are irrelevant) strategies. In addition, the compositional structure underlying the concept hierarchy is reflected in the emergent protocols, indicating that the need to develop hierarchical reference systems supports the emergence of compositionality.

## introduction humans excel at using language to convey information efficiently in context. a speaker does not have to communicate every detail. rather, a listener can infer the intended meaning of an utterance by assuming that sufficient information was provided. this idea was first explicitly formulated by @xcite in his conversational maxims, in particular the maxim of quantity: "1. make your contribution as informative as is required (for the current purposes of the exchange). 2. do not make your contribution more informative than is required." an illustration of this mechanism can be given in the form of a simple referential context. in a scene with a red circle and a green triangle, "circle" is enough information to identify the referent, whereas more complex scenes may require the speaker to name both object attributes-shape and color-to allow for an unambiguous interpretation. the maxim of quantity requires a hierarchical reference system, that allows the selection of the most appropriate level of specificity for a given context. in this paper, we follow the proposal by @xcite and define concepts as compositional abstractions over a set of primitive attributes (e.g., color, style, shape), see figure 1a . the concepts are maximally specific at the leaf nodes, where all attribute values are determined. moving from the subordinate levels up to the superordinate levels, the number of concept-defining attribute values decreases. thus, each parent concept is an abstraction (i.e. a subset) over its children and over the original set of attribute values. given this definition of a concept hierarchy, we use a language emergence paradigm with artificial agents to study whether a corresponding reference system can emerge given a structured perception of the world. in most language emergence simulations, a sender and a receiver agent are trained on a reference game (e.g., @xcite @xcite , based on the signaling game originally developed by @xcite . the sender sees a target object and sends a message to the receiver. using that message, the receiver tries to identify the target among a set of distractor objects. crucially, in the current form, reference games completely ignore that different contexts may require referential expressions at different levels of abstraction. having no access to the distractors, the sender cannot choose relevant object attributes in a contextdependent way. moreover, random sampling of the distractors typically encourages the sender to communicate all object attributes. therefore, the standard reference game cannot account for the emergence of hierarchical concepts in communication. we develop a hierarchical reference game to address this shortcoming. instead of an object, the sender receives a concept as input. the concept is defined by an attribute vector (object) and a relevance vector (context). the relevance vector indicates for each attribute whether it is relevant in the current context or not. based on the sender's message, the receiver must identify an object that instantiates the target concept among a set of distractors. the input concepts have a compositional and hierarchical structure. while the game is designed to encourage communication at different levels of abstraction, it does not regulate how this abstraction is realized; in particular, there is no explicit pressure on the emergent language to reflect the compositional input structure. first, we evaluate if the agents can successfully play the game, i.e. communicate specific contextually relevant object attributes. second, to measure whether the agents' strategies are systematic, we test whether they can generalize to novel concepts, and also whether they consistently use the same expressions for the same concepts at all levels of abstraction. third, we investigate the emerging protocols to study the mechanisms by which systematic abstraction is achieved, see figure 1b . in natural language, there is holistic abstraction as in "dalmatian" ⊆ "dog" ⊆ "animal"; but also compositional abstraction as in "filled red circle" ⊆ "red circle" ⊆ "circle". abstraction can further be implicit, by omitting irrelevant attributes, and explicit, by indicating that certain attributes are irrelevant (as in saying "a circle of any color"). we evaluate which, if any, of these abstraction strategies are used by the agents. our work makes several contributions. we develop the hierarchical reference game and show that it can be used to model the emergence of ref-erential expressions at different levels of abstraction. we also provide novel metrics to examine how the agents achieve abstraction. working with different data sets, i.e. different concept hierarchies, allows us to disentangle data set specific and general effects. not least, our results suggest that communication about concept hierarchies supports the emergence of compositionality. ## experiments our implementation is based on pytorch, and uses the egg toolkit @xcite . 3 our code and results are available at https://github.com/xeniaohmer/ hierarchical_reference_game . ## results in this section, quantitative and aggregated results will be presented. random examples of concepts and messages, together with a qualitative analysis can be found in appendix d. appendix d.1 shows example mappings between abstract concepts and messages and appendix d.2 highlights different abstraction strategies. ## conclusion in this work, we developed a hierarchical reference game to study the emergence of hierarchical reference systems. in the game, concepts are defined as abstractions over a set of attributes. to refer to these concepts, our agents developed abstract terms and used these terms systematically, in the sense that they could generalize to novel objects and novel abstractions. it seems that, aside from more obvious strategies such as leaving out irrelevant information, the agents developed abstraction operators to explicitly indicate the irrelevance of certain attributes. even more surprisingly, for some data sets, they used the same few symbols to indicate irrelevance across attributes, rather than a dedicated symbol per attribute. while the game design encourages the emergence of abstract concepts, the use of specific abstraction operators emerged without any explicit pressure. in addition, our results suggest that compositional language may emerge as part of a hierarchical reference strategy. in the classical reference game, the sender typically tries to communicate the union of all object attributes. without additional pressures, the emerging languages are not compositional @xcite @xcite . in the hierarchical reference game, in contrast, the sender must pick out specific attributes for communication, which potentially stimulates disentanglement. this interpretation is in line with the finding that the emergence of compositionality is supported by an increasing number of relevant events that can be referred to @xcite . in the hierarchical reference game, cross-situational reuse is increased, as reference to attribute values occurs not only across objects but also across levels of abstraction. we envision two main directions for future work. first, we would like to implement a hierarchical reference game with raw visual inputs instead of symbolic input vectors. @xcite have developed a neural network (scan) that not only learns disentangled visual primitives in an unsupervised manner but also abstractions over such primitives from very few symbol-image pairs that apply to a particular concept. combining our language emergence game with such a network would allow us to study the simultaneous emergence of abstract visual and linguistic concepts, as well as interactions between these two processes. second, instead of hard-coding the relevance vector, the relevance of certain attributes should arise from the agents' intentions. ideally, the agents would play a more complex game and determine themselves which properties of the environment are relevant for their objectives in the current context. besides, sender and receiver could use pragmatic reasoning (as for example in @xcite @xcite to encode and decode which attributes should be emphasized to communicate certain concepts.