N e LLC om- X : A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication (2024)

Abstract: Recent advances in computational linguistics include simulating the emergence of human-like languages with interacting neural network agents, starting from sets of random symbols. The recently introduced NeLLCom framework (Lian et al., 2023) allows agents to first learn an artificial language and then use it to communicate, with the aim of studying the emergence of specific linguistics properties. We extend this framework (NeLLCom-X) by introducing more realistic role-alternating agents and group communication in order to investigate the interplay between language learnability, communication pressures, and group size effects. We validate NeLLCom-X by replicating key findings from prior research simulating the emergence of a word-order/case-marking trade-off. Next, we investigate how interaction affects linguistic convergence and emergence of the trade-off. The novel framework facilitates future simulations of diverse linguistic aspects, emphasizing the importance of interaction and group dynamics in language evolution.

## introduction human language can be viewed as a complex adaptive dynamical system @xcite @xcite , in which individual behaviours of language users drive linguistic emergence and change at the population level. languages are shaped by the brains of individuals who are learning them @xcite and novel conventions and meanings are negotiated during interaction and language use @xcite @xcite . the effect of these mechanisms on linguistic patterns has been studied extensively, and it is recognized that language systems do not spring from the mind of a single individual, but are the result of constant rein-* shared senior authorship. recently, this interactive and dynamic property of human language was recognized as a key factor to improve ai @xcite , leading to a large interest in simulating the emergence of human-like languages with neural network agents @xcite @xcite . typically, a pair of agents is simulated where a speaking agent tries to help a listener recover an intended meaning by generating a message the listener can interpret. early frameworks have been progressively expanded to display important aspects of human language and communication, like generational transmission @xcite @xcite , group interaction @xcite @xcite @xcite and other aspects @xcite . within this body of work, most studies start from sets of random symbols, with a strong focus on tracking the emergence of human-like language properties such as compositionality @xcite @xcite or principles of lexical organization like zipf's law of abbreviation @xcite . however, neural agent emergent communication frameworks could also be a valuable tool to simulate the evolution of more specific aspects of language. studies with human participants have addressed many other aspects such as specific syntactic patterns like word order or morphology @xcite @xcite , a tendency to reduce dependency lengths @xcite , colexification patterns and the role of iconicity or metaphor in the emergence of new meanings @xcite @xcite @xcite , and combinatorial organisation of basic building blocks @xcite @xcite . what most of these studies have in common is that participants are asked to learn and/or interact with pre-defined artificial languages specifically designed by the experimenters to study the linguistic property of interest. however, the existing neural-agent communication frameworks (often based on egg @xcite ), do not enable training agents on predefined languages. a different body of work has studied the learnability by neural networks of various types of artificial languages @xcite @xcite @xcite . this paradigm has led to important insights, revealing inductive biases of neural models, but is limited to studying learnability in a passive supervised learning setting, unlike the dynamic and interactive setting in which human language has evolved. a framework combining agent communication with the ability to learn pre-defined artificial languages was recently introduced by @xcite . in nellcom (neural agent language learning and communication), agents are first trained on an initial language through supervised learning, followed by a communication phase in which a speaking and listening agent continue learning together through reinforcement learning by optimizing a shared communicative reward. in this paper, we extend nellcom with group interaction with the aim of studying the interplay between learnability of specific pre-defined languages, communication pressures, and group size effects under the same framework. to this end, we first extend the vanilla nellcom agent to act as both listener and speaker (i.e. role alteration, cf. figure 1 ), which was identified as an important gap in the emergent communication literature by @xcite . then, we design a procedure to let such 'full-fledged' agents interact in pairs with either similar or different initial language exposure, or in groups of various sizes. with the extended framework, nellcom-x, we replicate the key findings of @xcite and additionally show that (i) pairs of agents trained on different initial languages quickly adapt their utterances towards a mutually understandable language, (ii) languages used by agents in larger groups become more optimized and less redundant, and (iii) a word-order/case-marking trade-off emerges not only in individual speakers, but also at the group level. we release nellcom-x to promote simulations of other language aspects where interaction and group dynamics are expected to play a key role. foot_0 2 related work role-alternating agents initially, most work on emergent communication modeled agents to fulfill separate, complimentary roles (i.e. one agent always speaks, the other always listens). human language users are, of course, able to take both roles. when listing a set of "design features" of human language, hockett (1960) refered to interchangeability as the ability of language speakers to reproduce any linguistic message they can understand. in experiments with humans communicating via artificial languages, participants also usually take turns being the speaker and listener @xcite @xcite @xcite . therefore, @xcite named rolealternation as a missing key ingredient to close the gap between outcomes of simulations and findings from human language evolution data. exceptions to this trend include the rolealternating architectures of @xcite , and @xcite . recently, @xcite propose a method to couple a speaker and listener among a group of speaking and listening agents. by what they call "partitioning", the listener-part is only trained to adapt to its associated speaker, while the listener parameters are frozen during communication with other speakers. hence, the speaking and listening parts of an agent are tied softly, i.e. no "physical" link via shared modules. while being workable, this partitioning seems less realistic in terms of cognitive plausibility and communication, as human listeners continually refine their understanding during all kinds of interactions (speaking as well as listening). what all these studies have in common is their focus on protocols emerging from scratch, i.e. starting from random symbols, which does not allow for simulations with pre-defined languages. closer to our goal, @xcite train agents on artificial languages and observe them drift in a simple iterated learning setup that does not model communication success. they use sequence-to-sequence networks that can function both as speaker and listener by representing both utterances and meanings as sequences and merging meaning and word embeddings into a single weight matrix, tied between input and output. we combine elements of the above techniques to design agents that can learn artificial languages and use them to interact in a realistic manner. group communication natural languages typically have more than two speakers, and language structure is shaped by properties of the population. according to the linguistic niche hypothesis, for example, languages used by larger communities tend to be simpler than those used in smaller, more isolated groups @xcite . similarly, experiments with human participants have shown that interactions in larger groups can result in more systematic languages @xcite . various emergent communication simulations have been designed to investigate group effects, revealing the emergence of natural language phenomena. tieleman et al. (2019) , for example, found that representations emerging in groups are less idiosyncratic and more symbolic. they model a population of community-autoencoders and since the identities of the encoder and decoder are not revealed within a pair, the emerging representations develop in such way that all decoders can use them to successfully reconstruct the input, resulting in a more simple language as also found in humans. @xcite found that larger agent groups develop more compositional languages. harding @xcite investigated various language contact scenarios with populations of agents that have first developed distinct languages within their own groups, and could observe the emergence of simpler 'creole' languages, resembling findings from human language contact. @xcite vary the connectivities between agents in groups, and find the spontaneous emergence of linguistic dialects in large groups with over a hundred agents having only local interactions. again, none of these frameworks support training agents on pre-defined languages, limiting the extent to which they can be applied to specific human-like linguistic features. in this work, we showcase how nellcom-x agents can interact in groups using artificial languages that were specifically designed to study the emergence of word-order/case-marking patterns. ## nellcom-x we summarize the original nellcom framework @xcite ) and then explain how we extend it with role alternation and group communication. ## experimental setup as our use case, we adopt the same artificial languages as @xcite . these simple verbfinal languages vary in their use of word order and/or case marking to denote subject and object, and were originally proposed by @xcite to study the existence of an effortinformativeness trade-off in human learners. ## replicating the trade-off with full-fledged agents before moving to interactive communication, we validate the new nellcom-x framework through a replication of @xcite 's main findings. the simple speaker-listener communication setup of nellcom could be seen as a speaker-internal monitoring mechanism predicting the utterance understandability @xcite . here, we compare nellcom results to those of nellcom-x full-fledged agents only engaging in self-play. we use sl to train two sets of agents on the exact same languages as @xcite , respectively: 100s+67m for fixed-order and 50s+67m for flexibleorder. then, every agent performs 60 self_turn iterations causing its production preferences to drift. after sl, our agents have successfully learnt both languages but no regularization happens, as expected. by contrast, the results of self-play averaged over each 50-agent set indicate that both languages progressively lose markers. crucially, the fixed-order language does so faster than the flexible one, where markers are often necessary for agent/patient disambiguation. in sum, self-play in nellcom-x results in very similar trends as the simple nellcom setup, confirming the emergence of a human-like order/marking trade-off @xcite . detailed replication results are provided in appendix b. here, we report communication success during self-play and production preferences at the end of self-play for the flexible language (figure 2 , top row). self-understanding increases through rl leading to a much more informative language, while production preferences reveal that this spans from an overall decrease in order entropy with marking proportion remaining almost the same on average (solid circle). while some agents approach the optimal points of fixedorder/no-marking (bottom-left corner) or flexibleorder/full-marking (top-right), the large variability in production preferences suggests many agents settle on less optimized, redundant languages, as also found by @xcite . ## interactive communication results this section presents our main results: in section 6.1 we focus on pairwise interaction and show how nellcom-x can be used to simulate communication between speakers of different languages, which was not possible in the original framework; in section 6.2 we move to group communication and study the effect of group dynamics on communication success and production preferences. training details for this section are given in appendix c. ## discussion and conclusion we introduced nellcom-x, a framework for simulating neural agent language learning and communication in groups, starting from pre-defined languages. agents in this framework display the cognitively plausible property of interchangeability @xcite , by which anything they can understand, they can say and vice versa, while also having the ability to align to other individuals. we replicated an earlier finding by @xcite and showed that a word-order/case-marking tradeoff still appears with the adjusted full-fledged agent architecture. subsequently, we simulated interactions between agents trained on different languages. we found that pairs quickly adapt their utterances towards a mutually understandable language and that the neutral language drifts in different directions depending on the preferences of the other agent. moreover, agents converge on a shared language faster, and reach higher accuracy in cases where one of the two agents has a stronger word order preference. we then assessed the effect of performing self-play during interactive communication and found it necessary to ensure our fullfledged agents continue to understand themselves, while also realistically adapting to other individuals. lastly, we studied group dynamics and found that nellcom-x agents manage to establish a successful communication system even in larger groups (up to size 20). moreover, we generally see a larger entropy reduction in the languages developed by larger groups as compared to the languages used by pairs of agents. this finding aligns with previous work on group-level emergent communication, where it was shown that groups developed less idiosyncratic languages than pairs @xcite as well as with human experiments which demonstrated more systematic languages to emerge in larger groups @xcite . in our simulations, pairs and smaller groups sometimes settle on less optimized and partly still redundant solutions, while large groups end up with more efficient communication systems. in the future, nellcom-x can be used to study the influence of learning and group dynamics on many other language universals. we plan to keep refining the framework to allow studying different connectivities between the agents, multilingual populations and generational transmission of emerged languages to new agents.