What Preserves the Emergence of Language?

1 INTRODUCTION . Unveiling the principles behind the emergence and evolution of language is attractive and appealing to all . It is believed that this research field is of great significance for promoting the development of enabling agents to evolve an efficient communication protocol ( Nowak & Krakauer , 1999 ; Kottur et al. , 2017 ; Chaabouni et al. , 2019 ) or acquire existing one ( Li & Bowling , 2019 ) , especially when interacting with humans . Previously , many studies have investigated some intriguing properties of language and their effects on the emergence of language ( Andreas & Klein , 2017 ; Lazaridou et al. , 2018 ; Mordatch & Abbeel , 2018 ) . The motivation behind these is that human language is considered as a remarkable degree of structure and complexity ( Givon , 2013 ) and each character is the result of evolution , thus they believe that understanding the language itself is an indispensable step to take . Unlike existing work , we , from a different perspective , focus on a fundamental question that what made the emergence of language possible during evolution . One of the dominant theories in the community of emergent communication is : cooperation boosts language to emerge ( Nowak & Krakauer , 1999 ; Cao et al. , 2018 ) . Hence , there has been a surge of work investigating this field in cooperative multi-agent ( mostly two agents ) referential games ( Lazaridou & Peysakhovich , 2017 ; Kottur et al. , 2017 ; Das et al. , 2017 ; Evtimova et al. , 2018 ; Lazaridou et al. , 2018 ) , a variant of the Lewis signaling game ( David , 1969 ) . However , they seem to miss some basic elements in the human language . On one hand , human language emerges from the community , not just two persons , after all , language is learnable and can spread from one place to other ( Dagan et al. , 2020 ) . Studying a language in two-player games is like looking at the world through a keyhole . On the other hand , many works make an agreement that prior to the emergence of language some pre-adaptations occurred in the hominid lineage , and one of the candidates is the ability to use symbols ( Deacon , 2003 ; Davidson , 2003 ; Christiansen & Kirby , 2003 ) . It seems understanding the emergence of symbolic signals is the key to approach the truth of the origin of language ( Deacon , 1998 ) . However , chimpanzees have demonstrated a degree of language capacity by using arbitrary symbols as well as the ability for the cross-modal association , abstract thought , and displacement of thought in time ( Meddin , 1979 ) . So why don ’ t they have a language like us ? One of the theory is selfishness has kept animal communication at a minimum ( Ulbaek , 1998 ) . In more detail , if an individual can obtain a higher benefit by deceiving the other party in the cooperation , why not deceive ? Once deception emerges , mistrust among individuals will haunt . For those who are cheated , once bitten and twice shy , cooperation will no longer be a good option . As a result , motivation for communication , as well as demands of the emergence of language will perish . But human beings are so special since we have overcome this kind of obstacle and evolved language . Then , what preserves the emergence of language ? We aim to answer this question in a brand new framework of agent community , reinforcement learning ( RL ) , and natural selection . We believe this process should occur in the pre-language period since lying is possible as long as agents can communicate . Therefore , our investigating communication protocol uses symbols to transmit meaning based on a social convention or implicit agreement . In this paper , we introduce several agents to form one community and allow natural evolution and elimination among them . Both liars ( agents tell lies ) and truth tellers ( agents always tell truth ) exist in the community . Each tournament , every agent is supposed to play a non-cooperative game ( Nash Jr , 1950 ; Nash , 1951 ; Schelling , 1958 ; Binmore et al. , 1986 ; Von Neumann & Morgenstern , 2007 ) with others . In our multi-round bargaining game , agents are required to reach an agreement about how many items to give out so that the total quantity can satisfy the market ’ s demand and they can keep their loss to the minimum in the meantime . We believe this is a perfect fit for the nature of human beings and more common in the hominid lineage compared with the cooperation game . Importantly , during the process of natural selection , the fraction of liars and truth tellers may change from time to time and this allows us to observe what factor imposes influence on the motivation of communication , which is the prerequisite of the emergence of language . It is worthy of note that pre-language communication was subject to the constraints of Darwinian evolution . While linguistic change , which began in the post-language communicative era of hominid evolution , is by and large tied to society and culture ( Givón & Malle , 2002 ; Li & Hombert , 2002 ) . Thus , we disregard the factors related to linguistic change since we are investigating motivation for communication from which language evolved . Moreover , apart from the normal setting mentioned above , we add up two more rules to further dig out . Firstly , we introduce a credit mechanism for truth tellers . In other words , we make sure truth tellers know the existence of liars which is one step in the evolution process . Specifically , every liar has credit in the mind of truth teller , and the credit varies with the profit of truth teller . Cooperation would be impossible between two agents as soon as the credit drops to negative . Secondly , an additional penalty will be brought in as a price of lying , and we consider it as social pressure for resisting lying behaviors . All in all , we want to make a thorough investigation about how the individual or social resistance to lying affects communication . Empirically , we show that in normal settings , two truth tellers can make a fair agreement , and liars can achieve a huge advantage over truth teller by telling lies . As for two liars , there is always a better liar that gains relative more than the other . In the credit setting , liars can learn a sophisticated lying strategy that deceives the credit mechanism and makes more profits meanwhile . In both settings , as time goes on , truth tellers seem not to show enough competition against liars and thus die out . In the society setting , we find out liars are afraid of lying if punishment is sufficiently large . This again proves the theory ( Ulbaek , 1998 ) : in the human lineage , social cooperation based on obligatory reciprocal altruism ( Trivers , 1971 ) as well as a system which punishes people morally and physically for cheating has evolved . In such an environment language is finally possible . 2 EXPERIMENTAL FRAMEWORK . 2.1 GAME SETTINGS . We explore emergent language in the context of multi-round non-cooperative bargaining game ( Nash Jr , 1950 ; Nash , 1951 ) as illustrated in Figure 1 . The core is binding cooperative strategy with the highest profit is impossible whereas selfish strategy sometimes can . In this case , the behavior of telling lies can be meaningful since it undermines cooperation and grabs more benefits from others . In the game , two agents i , j bargain over how to satisfy the demand of market . Agents are presented with N different items . They possess a fixed number of quantity for each item ( { qin } Nn=1 , { qjn } Nn=1 ) , and have their own hidden utilities for N items ( { uin } Nn=1 , { ujn } Nn=1 ) . Agents move sequentially . Suppose at bargaining round t , it is turn for agent i to be proposer and it makes proposal { pit , n } Nn=1 about how many to give out to the market , which means the other agent j should contribute the rest { dn − pit , n } Nn=1 to the market , where dn is the market demand for item n. Then agent j would choose to accept the proposal or not . The game will be terminated when they make an agreement about the proposal or the number of bargaining rounds reaches the upper limit Tmax , and agents i and j will receive rewards ∑N n=1 ( q i n − pit , n ) × uin and ∑N n=1 ( q j n − dn + pit , n ) × ujn respectively or get zero when no agreement has been made in the end . In order to make the reward comparable , it will be further normalized by the maximum reward achievable for each agent . Both agents want to keep as more items as possible rather than giving out . Therefore , agents are supposed to seek a tradeoff between keeping more items and satisfying the market demand . In this part , we illustrate how lying mechanism works , where lying means the proposal will not be followed by actions . Suppose agents i , j satisfy the market demand at round t , and agent j chooses to tell lies about the proposal and then gives nothing to the market . This leads to that the market will have demand gap which is { pjt , n } Nn=1 , and the market would force each agent to hand over half of the gap for remedy . It is noted that liars are allowed to tell lies about any items . In our settings , we conform to the principle of high risk and high return . To illustrate , when d1 = 10 agent j tells lies and keeps offering pjt,1 = 9 , agent i is definitely delightful and more likely to take d1 − p j t,1 = 1 . Although agent j can easily put others on the hook , it can not gain a large advantage since the final result is not much better than evenly contributing . On the contrary , if agent j takes pjt,1 = 1 , it allows agent j keeping more items and obtaining more profits . However , this proposal is less appealing to agent i . The key point is we want to link attractive lies to relatively lower profits . As agents take turns to play , first-mover has absolute advantages since the other one is compelled to accept unfair proposal or get nothing . We once tried to convert our bargaining game from moving alternately to moving simultaneously to solve this problem . In detail , two agents make proposals at the same time and their total quantity should surpass the demand . However , it turns out that agents only learn to evenly divide the market demand . In this setting , no agent can guarantee an agreement , therefore the proposal made by an agent could be more conservative , meaning it is hard for the agent to infer others ’ utilities during bargaining . Ultimately , we adopt the strategy ( Cao et al. , 2018 ) by randomly sampling Tmax between 4 and 10 to mitigate the first-mover effect .

Summary: This paper attempts to address a question in the emergent communication literature: what preserves / maintains the stability of emerged communication protocols.  The authors manipulate the prevalence of lying behavior in a community of agents playing a variant of a Nash bargaining game.  The main take-away is that explicit punishment, from the environment and from truth-tellers not wanting to communicate with liars, can prevent the spread of exploitative lying behavior in the community.