Emergent Communication under Varying Sizes and Connectivities

1 Introduction . Communication plays a vital role for us humans to interact with one another [ Austin , 1975 , Allwood , 1976 , Linell , 2009 ] . We acquire new information about the environment and other people around us through communication , which further facilitates collaboration or competition and allows us to coordinate our future decisions and actions better . Over the past decades , numerous research outcomes have succeeded in mimicking human behavior and making artificial agents equipped with interactive communication abilities [ Batali , 1998 , Cangelosi and Parisi , 2002 , Christiansen and Kirby , 2003 , Steels , 2003 , Wagner et al. , 2003 ] . The communication in the artificial system is emergent in a way that there are no pre-specified usage rules , semantics , or syntax , all of which are formed by the artificial agents upon their necessities [ Lazaridou et al. , 2017 ] . Not only does empowering agents with their own communicative abilities set an important milestone in interactive AI systems , it also brings us steps closer to the world where machines cooperate with humans [ Mikolov et al. , 2016 , Crandall et al. , 2018 , Lazaridou and Baroni , 2020 ] . In human societies , hundreds of millions of people use the shared , agreed communication protocols to interact . As the group becomes more complex with increased size and connectivity , communication becomes more and more vital for us to coordinate with one another . Fortunately , we humans excel at using languages for large-group coordination and problem-solving [ Tomasello , 2010 , Lupyan and Bergen , 2016 ] . However , in the AI system counterpart , the question of how emergent communication will function under different group settings is understudied . As the group becomes more complex , will emergent communication become more effective for the system in promoting group coordination , as in human society , or will it lose its functionality ? Will the emergent communication still converge 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . to a shared , agreed language ? In the worst case , efficient interaction among agents through emergent communication can be hampered because different agents will start to use different languages to describe the same observation ( the left-most panel of Figure 1 ) . This research provides an analytical study on emergent communication in artificial groups of different sizes and connectivities . Our goal is to investigate the effect of emergent communication and analyze how agents ’ general agreement on the communication protocol changes under varying group settings . To study this question , we construct settings within which communication among agents is indispensable for achieving their shared goals . Communication is their only means to interact and compensate for their limited access to information from an environment . We divide agents into two groups and allow an agent to communicate with another agent in a different group through a bi-partite communication graph . To quantitatively evaluate the functional aspect of communication , we look at ( a ) the language similarity or message agreement of agents within the same group when describing the same referents and ( b ) the rate at which an arbitrary pair of agents in the bi-partite communication group succeed in communication and achieve the shared goal . To this end , our contributions and findings are as follows : ( 1 ) We find that as the size of the communication graph scales up to hundreds , the language used in the group becomes less similar amongst agents . However , the communication success rate is maintained . ( 2 ) We observe that when agents only get to communicate with nearby agents , a particular form of communication peculiar to agents ’ locality emerges . At the same time , distant pairs of agents that have not interacted during training can still communicate when a certain threshold for the maximal communication distance is reached . ( 3 ) We present and analyze the optimized communication graph in terms of overall communication success rate when ( a ) the node outdegrees are fixed or when ( b ) we penalize the formation of communication links between a distant pair of agents . Compared to the fully-connected all-to-all graph , the link density decreases by 81.25 % , but the communication success rate drops just by 5.89 % . With random , unoptimized communication links , the success rate drops by over 40 % with the same link density . Throughout the optimization process , we observe the emergence of hub nodes taking up a large share of the total communication channels and connecting the majority of other agent nodes , as we see in the real-world scale-free social networks [ Barabási and Albert , 1999 ] . 2 Related work . There have been successive attempts to let agents develop their own communication protocols through emergent communication [ Batali , 1998 , Cangelosi and Parisi , 2002 , Christiansen and Kirby , 2003 , Steels , 2003 , Wagner et al. , 2003 ] , followed by more recent approaches that are based on deep neural networks [ Foerster et al. , 2016 , Sukhbaatar et al. , 2016 , Hausknecht , 2016 , Havrylov and Titov , 2017 ] . 1 The idea is applied and extended to a variety of collaborative [ Lowe et al. , 2020 , Hu and Foerster , 2020 , Jaques et al. , 2019 , Kajić et al. , 2020 , Mordatch and Abbeel , 2018 ] and competitive [ Singh et al. , 2019 , Jaques et al. , 2019 ] environments . Recently , advances have been made for the emergent communication to be operationalized at scale ; to prevent agents from being `` lazy '' and let them generate meaningful signals [ Sunehag et al. , 2018 ] ; to effectively schedule 1The work by Lazaridou and Baroni [ 2020 ] surveys the nascent literature . the communication in a limited bandwidth [ Kim et al. , 2019 ] ; to foster conditions that encourage compositional and easier-to-teach languages [ Li and Bowling , 2019 ] ; to understand other agents ’ intention , belief , and point of view , i.e. , theory of mind [ Hu and Foerster , 2020 ] ; to win a combat against another group [ Singh et al. , 2019 ] . Orthogonally , there has been attempts to make the interactive communication more dynamic and more human-like by increasing the complexity of the environment [ Jaques et al. , 2019 , Das et al. , 2019 ] , making the communication span multiple turns [ Evtimova et al. , 2018 , Jorge et al. , 2016 ] . These lines of research focus on increasing the efficacy functionality of the emergent communication by suggesting more advanced perceptual , navigational , and generative AI capabilities . On the other hand , our work falls into the category of an analytical study of understanding the emergent communication from various perspectives . Recent studies find properties of emergent language that sets itself aside from [ Chaabouni et al. , 2019 , Kottur et al. , 2017 ] or are shared with [ Resnick et al. , 2020 , Baroni , 2020 ] natural human languages and seek for different possibilities to benefit existing NLP models [ Lee et al. , 2018 , Lazaridou et al. , 2020 ] . Bouchacourt and Baroni [ 2019 ] and Cao et al . [ 2018 ] studies various conditions under which the emerged language can be consonant and agreed as apposed to giving rise to idiolects or agent identity playing a crucial factor in interactive systems . The work of Graesser et al . [ 2019 ] is especially similar to ours in that it also studies the linguistic phenomena of emergent communication at a group ( community ) level . Specifically , the authors formulate one , two , or five artificial communities each of which comprising agents up to size 30 . By changing the inter and intra-community connection densities , the authors reproduce linguistic behaviors that can be observed in human communities . One of the findings is the emergence of the `` linguistic continuum '' , which to a great degree resembles our finding of `` dialects '' in section 5.4 . Our work is distinctive in a way that we do not assign agents into hard clusters of communities . Therefore , we observe finer granularities of linguistic continuums ( dialects ) at agent-levels rather than at community-levels . Finally , Dubova and Moskvichev [ 2020 ] , Dubova et al . [ 2020 ] , Tieleman et al . [ 2019 ] also studies emergent communications in group settings . 3 Game design . We design a communication game based on partial information . The purpose of the game is similar to that of the referential game [ Lazaridou et al. , 2017 ] , which is a variant of the signaling game [ Lewis , 1969 ] ; we test whether an artificial agent can perceive a given observation , speak through the emergent protocol about the observation according to the instruction , listen and make the right decision . Throughout the game , the message from the speaker agent should convey crucial information about the observation for the game to succeed . Compared to the referential game , the difference is : 1 ) agents only get to have partial , mutually exclusive information of the observations , so the communication should solely be about describing what information each agent has from their partial , limited observation , 2 ) The instruction information is symmetric for both listener and speaker : both agents have access to the instruction . In each game trial , two agents are presented with multiple pixel-based objects : a target , distractors , and instruction . The goal is to choose the correct target object for the description in the instruction . Crucially , communication is indispensable for the agents to correctly find the target since each of the two agents has its own limitation in the perception : One agent can only recognize different shapes , and the other can only differentiate colors . Therefore , the information of the object retrieved by the agents from their observations is mutually exclusive , but at the same time , is guaranteed to recover the complete state information when aggregated . Under this setting , the agents communicate based on their limited observations to compensate for their deficiencies and correctly choose the target object described in the instruction . In section 5.3 , we derive similar findings using the more conventional referential game as well . Agent and action Each of the two non-embodied agents is chosen from two distinctive groups . We denote agents that can only perceive colors ( i.e. , the color-only agents ) by c , and the corresponding group by Gc , such that c 2 Gc . Similarly , the shape-only agents and the corresponding group are denoted by s and Gs . The number of agents of both groups is set to be the same , i.e. , |Gc| = |G s | = N . We consider a situated environment , where the action space A is divided into the disjoint communication action space Am and decision-making action space Ae , such that Am [ Ae = A and A m \A e = ; . Of color-only and shape-only agents , one is randomly selected and act as a speaker ; it takes its observations , i.e. , objects and instruction , as inputs and sends a message m 2 Am to the other agent as an output , which can be either discrete or continuous . Next , the other agent is randomly selected and act as a listener . This agent takes the message along with the observations and makes a decision d 2 Ae . The decision is about choosing the position of the correct target object . Thus , the size of the decision making action space |Ae| is equal to the number of objects in the observation . Finally , the communication between agents ( c , s ) is successful if the decision d finds the target object . In the referential games , both speaker and listener agents get to observe the full information about the objects , i.e. , both shapes and colors , but only the speaker agent has an access to the instruction and the orders of the objects are shuffled for the listener agent . State and observation A state s is a 3-tuple s = ( ot , Od , w ) with the target object ot , the distractor objects Od , and the instruction w. Each object is of different colors and shapes 2 , and the instruction tells of choosing an object with certain color and shape . Therefore , for n and m different colors and shapes , there can be n ⇥m varying instructions . We represent the instruction using a fixed-sized vector by concatenating n and m-dimensional one-hot vectors . Observations comprise objects and instructions . In section 5.3 , we also conduct experiments using real-world images [ Krizhevsky , 2009 ] instead of the artificial shape and color observations . Communication graph A bipartite communication graph G is formed between Gc and Gs as multiple pairs ( c , s ) communicate , i.e. , form a set of directed communication links ( c , s ) , ( s , c ) 2 E , and establish communication protocols . Message agreement and communication success rate We introduce message agreement and communication success rates used to evaluate language convergence . First , let a and a0 be two agents randomly sampled from the same color-only or shape-only group , i.e. , a , a0 2 Gc or a , a0 2 Gs , and let ma and ma 0 be the messages generated by the agents from the same observation o 2 O . The message agreement of a certain group , which is either color-only or shape-only , in a communication graph is measured by the expected similarity between two messages ma and ma 0 : Ea , a0⇠G0 , o⇠O [ f ( ma , ma 0 ) |o ] , where f ( · , · ) is a similarity measure and G0 2 { Gc , Gs } . Next , the communication success rate of a communication graph is measured as : Ec⇠Gc , s⇠Gs , o⇠O [ r ( c , s , o ) ] with r being an identity function that returns 1 if agents successfully find the target object .

Summary: The paper investigates the results of Emergent Communication learning in different settings (number of agents, communication connectivity, task difficulty). The authors use a reference game variant where the communicating parties are limited in their perceptual abilities and hence need to communicate in order to solve the task (select a specific object among a number of alternatives). The authors obtain a number of interesting observations (for example, observing that larger groups of agents result in more diverse communication protocols, which (in certain settings) still allow to achieve high communication success rate).  ## Strengths  - The authors tackle a highly relevant problem - The experimental setup aims to investigate the impact of a number of factors that may indeed be crucial in determining the properties of emergent communication - The paper has a large number of illustrations and is generally well-written, which promotes understanding.  ## Weaknesses - The decision to use early stopping, while motivated by valid concerns, does not actually address these concerns. A universally controlled amount of training in terms of total number of interaction per agent would be much easier to interpret. Ideally, more than one schedule should be tested: 1-training until convergence, 2-training for a set number of interactions for any given agent. - Literature review overlooks a number of relevant works and thus inadvertently presents the experimental setup as more novel than it is.