Disentangling Categorization in Multi-agent Emergent Communication (2022)

Abstract: The emergence of language between artificial agents is a recent focus of computational linguistics, as it offers a synthetic substrate for reasoning about human language evolution. From the perspective of cognitive science, sophisticated categorization in humans is thought to enable reasoning about novel observations, and thus compose old information to describe new phenomena. Unfortunately, the literature to date has not managed to isolate the effect of categorization power in artificial agents on their inter-communication ability, particularly on novel, unseen objects. In this work, we propose the use of disentangled representations from representation learning to quantify the categorization power of agents, enabling a differential analysis between combinations of heterogeneous systems, e.g., pairs of agents which learn to communicate despite mismatched concept realization. Through this approach, we observe that agent heterogeneity can cut signaling accuracy by up to 40%, despite encouraging compositionality in the artificial language. We conclude that the reasoning process of agents plays a key role in their communication, with unexpected benefits arising from their mixing, such as better language compositionality.

## introduction a recent interest in the design of multi-agent systems is the unsupervised emergence of complex behaviors @xcite . rather than completely supervising agents to perform a mapping betweem inputs and outputs, it is observed that with enough capacity and bandwidth, agents eventually learn to produce emergent properties for their own benefit, e.g., the creation of artificial language to solve a task more efficiently @xcite @xcite . with carefully designed constraints, the agents may eventually produce a proto-language that mimics the complex properties of human lanthrough disentangled representations (denoted z ′ ), generated by either (a) prototypical parts network (protop-net) @xcite , where each dimension is the input's activation of a prototypical part previously seen by the agent, or (b) concept whitening (cw) @xcite , where each dimension corresponds to a previously-observed data category. due to the subjectivity of experiential knowledge, a natural consequence of the categorical reasoning theory is that two human minds may be heterogeneous, i.e., they do not share the same internal categorization of a situation or object. natural language serves to mediate these cognitive differences, thus enabling the communication of ideas @xcite . under our synthetic analogue of communicating multi-agent systems, how well can heterogeneous agents mediate their differences in categorization ability through the use of emergent language? in this paper, we offer a differential analysis of agents leveraging heterogeneous techniques for abstract categorization, taking full advantage of disentangled representations as a computational analogue for categorical reasoning. contrary to the scenario of different internal representations @xcite , our agents can possess either different or identical means of categorical abstraction, which influences the course of their language evolution, e.g., improving compositionality or signaling success of the final protolanguage. our analysis offers a substrate to answer four main research questions about heterogeneous multi-agent communication, revealing subsequent findings as a result: q1. what is gained or lost with agent heterogeneity? a1: we find that heterogeneity is at odds with signaling performance. despite this, heterogeneous systems exhibit better potential survival through higher compositionality. q2. does categorization power and heterogeneity affect signaling ability on novel objects? a2: simpler disentangled agents with poor performance in supervised tasks can outperform all other agents on novel objects by up to 9% on signaling accuracy. in some cases, mismatched agents manage to improve compositionality score. q3. can disentangled representations directly inform agent utterances? a3: through our proposed latent self-attention (lsa) module, incorporating the disentangled representation into agent utterances can encourage inputmessage alignment and improve signaling accuracy by up to 27%. q4. does increased categorization ability imply better communication? a4: agents with complex concept realization are potentially poor receivers, despite a successful upstream classification task. signaling can be performed up to 33% better with "simpler", i.e., smaller k, agents. to encourage reproducibility, we share the code for our experiments online. foot_0 2 background ## methodology in order to answer the research questions in section 1, we implement agents which can play a lewis signaling game (described in section 2.1) using disentangled neural networks, e.g., prototypical parts network (protopnet) by @xcite and concept whitening (cw) by @xcite . protopnet is trained through unsupervised disentanglement, where given a pre-defined number of concepts per class (totaling k over all classes), it extracts prototypical image patches representing each concept. concept whitening instead aligns orthogonal directions in the encoded input's latent space with k pre-defined (supervised) concepts through training. in practice, cw can only align a subset of the pre-existing latent space dimensions, leaving the rest of the dimensions to handle residual information. to avoid ambiguity, we denote a sample from this aligned subspace as z ′ ∈ r k . since pro-topnet learns a new latent space, it can be said to align the entire latent space to extracted concepts, thus z ′ = z. in either case, we use the latent vector z as the initial input to the sender agent's decoder, g s . the whole-system perspective of this approach is shown in figure 2 , where the bottom case (ours) corresponds to the use of disentangled networks, compared to a traditional approach above (original). we can manipulate which agents are paired (e.g., protopnet talking to cw) in order to simulate scenarios where two agents have different categorical reasoning. likewise, we can model categorization power by increasing or decreasing the number of realized concepts k in disentangled agents. this enables the study of incongruous agents to answer q1, q2, and q4. ## experiments preliminaries. to address q1-q4, we investigate mixed/same-encoder variants of multi-agent systems. pairs of agents play the lewis signaling game over five random seeds. each agent in a two-player game is instantiated with distinct image encoder checkpoints, thus requiring ten image encoder checkpoints total for each variant (with each checkpoint using a different seed). in all reported results of the main text, we compare against finetuned resnet-50 encoders (denoted convnet) as baselines @xcite , and subsequently use them as the base models for training protopnet and cw variants. we offer supplemental results for vgg-16 @xcite and densenet-161 @xcite variants in appendix a.4.1. with resnet-50 as the base architecture, each disentangled variant has a similar parameter count on the order of 24m (reported in appendix categorization power on novel objects (q2): to better understand the description of novel objects, in figure 4 we repeat the previous q1 experiment by swapping cub10 for mini-imagenet, whose test set consists of object classes not seen during training. we observe a similar trend that aligns with a1, except cw-cw outperforms other methods, at best scoring 85.585 ± 5.476 signaling accuracy compared to convnet-convnet (76.586 ± 6.400) and protopnet-protopnet (67.542 ± 3.001) despite realizing the least amount of concepts (@xmath0. in some instances, e.g., convnet-protopnet, the mismatched system can match the convnet-convnet performance within 3% accuracy. in measures for mini-imagenet. despite only reaching a signaling accuracy of 33.328 ± 19.324, the highest top.sim. and dis.sim. scores (0.450) are realized by mismatched protopnet-cw, thus compositionality was able to emerge despite the absence of generalization ability, as originally reported by @xcite . cw-cw and protopnet-protopnet scored similarly, obtaining 85.585 ± 5.476 and 67.542 ± 3.001 best epoch signaling accuracy, respectively. the best mixed-encoder system in figure 4 was convnet-protopnet (74.929 ± 3.079 accuracy), with a similarly high topographic similarity. we thus conclude: apart from incorporating new information into the sender's message decoding process, our proposed lsa module may enable better diversity in the agent utterances. recall from section 3.1 that the disentangled representation is linearly recombined and multiplied with message logits at each time-step, thus we posit that the lsa module encourages agents to use vocabulary that is otherwise ignored, which may explain the higher topographic similarity scores observed in tables 3 and 4 . we study this idea for cw in figure 5 (and protopnet in appendix a.5.1) by taking the top-performing multi-agent system for convnet (top row), cw (middle row), and cw+lsa (bottom row). each inset heat map is the activated vocabulary usage (x-axis) across input data classes (y-axis) in the cub10 test set at a given time step for each figure column (i.e., columns @xmath1, with the cumulative sum over time displayed in the rightmost column. we quantify the vocabulary usage by calculating normalized area (a) of shaded heatmap pixels, shown below each cumulative matrix. convnet and cw variants (top and middle row, respectively) evidence a tendency to re-use vocabulary at each time-step and do not explore different combinations of vocabulary across time steps. lsa variants (bottom row) instead have a higher distribution of vocabulary across time, which can be seen qualitatively by comparing early time-steps to sender arch. top. sim. dis. sim. recv. acc. cw z ′ utterance 0.325 ±0.031 0.503 ±0.006 75.670 ±1.820 cw 0.278 ±0.079 0.339 ±0.044 52.523 ±17.600 cw+lsa 0.266 ±0.087 0.439 ±0.076 79.859 ±2.417 convnet 0.391 ±0.055 -86.753 ±0.651 ## conclusion our results suggest that agents in a cooperative game setting are capable of communication despite differences in concept realization. despite this, we find that heterogeneous agents can be at odds with each other, often under-performing compared to their same-encoder variants. in this way, the reasoning process of an agent can largely determine its interaction with other agents. on novel objects, there is a notable advantage to using disentangled networks, and in some cases mismatched agents, evidenced by higher signaling success and language compositionality. our proposed latent self-attention module illustrates the advantage of incorporating an agent's categorization power into utterances, evidenced by higher disentangled similarity in experiments. the interplay between disentanglement and interpretability of the emerged language will be investigated in future work. as a result of the grid-search, we selected the best performing hyperparameter combination which balanced training time and communication success rate performance (e.g., receiver signaling accuracy). this combination is shown in