What Preserves the Emergence of Language?

1 INTRODUCTION . Unveiling the principles behind the emergence and evolution of language is attractive and appealing to all . It is believed that this research field is of great significance for promoting the development of enabling agents to evolve an efficient communication protocol ( Nowak & Krakauer , 1999 ; Kottur et al. , 2017 ; Chaabouni et al. , 2019 ) or acquire existing one ( Li & Bowling , 2019 ) , especially when interacting with humans . Previously , many studies have investigated some intriguing properties of language and their effects on the emergence of language ( Andreas & Klein , 2017 ; Lazaridou et al. , 2018 ; Mordatch & Abbeel , 2018 ) . The motivation behind these is that human language is considered as a remarkable degree of structure and complexity ( Givon , 2013 ) and each character is the result of evolution , thus they believe that understanding the language itself is an indispensable step to take . Unlike existing work , we , from a different perspective , focus on a fundamental question that what made the emergence of language possible during evolution . One of the dominant theories in the community of emergent communication is : cooperation boosts language to emerge ( Nowak & Krakauer , 1999 ; Cao et al. , 2018 ) . Hence , there has been a surge of work investigating this field in cooperative multi-agent ( mostly two agents ) referential games ( Lazaridou & Peysakhovich , 2017 ; Kottur et al. , 2017 ; Das et al. , 2017 ; Evtimova et al. , 2018 ; Lazaridou et al. , 2018 ) , a variant of the Lewis signaling game ( David , 1969 ) . However , they seem to miss some basic elements in the human language . On one hand , human language emerges from the community , not just two persons , after all , language is learnable and can spread from one place to other ( Dagan et al. , 2020 ) . Studying a language in two-player games is like looking at the world through a keyhole . On the other hand , many works make an agreement that prior to the emergence of language some pre-adaptations occurred in the hominid lineage , and one of the candidates is the ability to use symbols ( Deacon , 2003 ; Davidson , 2003 ; Christiansen & Kirby , 2003 ) . It seems understanding the emergence of symbolic signals is the key to approach the truth of the origin of language ( Deacon , 1998 ) . However , chimpanzees have demonstrated a degree of language capacity by using arbitrary symbols as well as the ability for the cross-modal association , abstract thought , and displacement of thought in time ( Meddin , 1979 ) . So why don ’ t they have a language like us ? One of the theory is selfishness has kept animal communication at a minimum ( Ulbaek , 1998 ) . In more detail , if an individual can obtain a higher benefit by deceiving the other party in the cooperation , why not deceive ? Once deception emerges , mistrust among individuals will haunt . For those who are cheated , once bitten and twice shy , cooperation will no longer be a good option . As a result , motivation for communication , as well as demands of the emergence of language will perish . But human beings are so special since we have overcome this kind of obstacle and evolved language . Then , what preserves the emergence of language ? We aim to answer this question in a brand new framework of agent community , reinforcement learning ( RL ) , and natural selection . We believe this process should occur in the pre-language period since lying is possible as long as agents can communicate . Therefore , our investigating communication protocol uses symbols to transmit meaning based on a social convention or implicit agreement . In this paper , we introduce several agents to form one community and allow natural evolution and elimination among them . Both liars ( agents tell lies ) and truth tellers ( agents always tell truth ) exist in the community . Each tournament , every agent is supposed to play a non-cooperative game ( Nash Jr , 1950 ; Nash , 1951 ; Schelling , 1958 ; Binmore et al. , 1986 ; Von Neumann & Morgenstern , 2007 ) with others . In our multi-round bargaining game , agents are required to reach an agreement about how many items to give out so that the total quantity can satisfy the market ’ s demand and they can keep their loss to the minimum in the meantime . We believe this is a perfect fit for the nature of human beings and more common in the hominid lineage compared with the cooperation game . Importantly , during the process of natural selection , the fraction of liars and truth tellers may change from time to time and this allows us to observe what factor imposes influence on the motivation of communication , which is the prerequisite of the emergence of language . It is worthy of note that pre-language communication was subject to the constraints of Darwinian evolution . While linguistic change , which began in the post-language communicative era of hominid evolution , is by and large tied to society and culture ( Givón & Malle , 2002 ; Li & Hombert , 2002 ) . Thus , we disregard the factors related to linguistic change since we are investigating motivation for communication from which language evolved . Moreover , apart from the normal setting mentioned above , we add up two more rules to further dig out . Firstly , we introduce a credit mechanism for truth tellers . In other words , we make sure truth tellers know the existence of liars which is one step in the evolution process . Specifically , every liar has credit in the mind of truth teller , and the credit varies with the profit of truth teller . Cooperation would be impossible between two agents as soon as the credit drops to negative . Secondly , an additional penalty will be brought in as a price of lying , and we consider it as social pressure for resisting lying behaviors . All in all , we want to make a thorough investigation about how the individual or social resistance to lying affects communication . Empirically , we show that in normal settings , two truth tellers can make a fair agreement , and liars can achieve a huge advantage over truth teller by telling lies . As for two liars , there is always a better liar that gains relative more than the other . In the credit setting , liars can learn a sophisticated lying strategy that deceives the credit mechanism and makes more profits meanwhile . In both settings , as time goes on , truth tellers seem not to show enough competition against liars and thus die out . In the society setting , we find out liars are afraid of lying if punishment is sufficiently large . This again proves the theory ( Ulbaek , 1998 ) : in the human lineage , social cooperation based on obligatory reciprocal altruism ( Trivers , 1971 ) as well as a system which punishes people morally and physically for cheating has evolved . In such an environment language is finally possible . 2 EXPERIMENTAL FRAMEWORK . 2.1 GAME SETTINGS . We explore emergent language in the context of multi-round non-cooperative bargaining game ( Nash Jr , 1950 ; Nash , 1951 ) as illustrated in Figure 1 . The core is binding cooperative strategy with the highest profit is impossible whereas selfish strategy sometimes can . In this case , the behavior of telling lies can be meaningful since it undermines cooperation and grabs more benefits from others . In the game , two agents i , j bargain over how to satisfy the demand of market . Agents are presented with N different items . They possess a fixed number of quantity for each item ( { qin } Nn=1 , { qjn } Nn=1 ) , and have their own hidden utilities for N items ( { uin } Nn=1 , { ujn } Nn=1 ) . Agents move sequentially . Suppose at bargaining round t , it is turn for agent i to be proposer and it makes proposal { pit , n } Nn=1 about how many to give out to the market , which means the other agent j should contribute the rest { dn − pit , n } Nn=1 to the market , where dn is the market demand for item n. Then agent j would choose to accept the proposal or not . The game will be terminated when they make an agreement about the proposal or the number of bargaining rounds reaches the upper limit Tmax , and agents i and j will receive rewards ∑N n=1 ( q i n − pit , n ) × uin and ∑N n=1 ( q j n − dn + pit , n ) × ujn respectively or get zero when no agreement has been made in the end . In order to make the reward comparable , it will be further normalized by the maximum reward achievable for each agent . Both agents want to keep as more items as possible rather than giving out . Therefore , agents are supposed to seek a tradeoff between keeping more items and satisfying the market demand . In this part , we illustrate how lying mechanism works , where lying means the proposal will not be followed by actions . Suppose agents i , j satisfy the market demand at round t , and agent j chooses to tell lies about the proposal and then gives nothing to the market . This leads to that the market will have demand gap which is { pjt , n } Nn=1 , and the market would force each agent to hand over half of the gap for remedy . It is noted that liars are allowed to tell lies about any items . In our settings , we conform to the principle of high risk and high return . To illustrate , when d1 = 10 agent j tells lies and keeps offering pjt,1 = 9 , agent i is definitely delightful and more likely to take d1 − p j t,1 = 1 . Although agent j can easily put others on the hook , it can not gain a large advantage since the final result is not much better than evenly contributing . On the contrary , if agent j takes pjt,1 = 1 , it allows agent j keeping more items and obtaining more profits . However , this proposal is less appealing to agent i . The key point is we want to link attractive lies to relatively lower profits . As agents take turns to play , first-mover has absolute advantages since the other one is compelled to accept unfair proposal or get nothing . We once tried to convert our bargaining game from moving alternately to moving simultaneously to solve this problem . In detail , two agents make proposals at the same time and their total quantity should surpass the demand . However , it turns out that agents only learn to evenly divide the market demand . In this setting , no agent can guarantee an agreement , therefore the proposal made by an agent could be more conservative , meaning it is hard for the agent to infer others ’ utilities during bargaining . Ultimately , we adopt the strategy ( Cao et al. , 2018 ) by randomly sampling Tmax between 4 and 10 to mitigate the first-mover effect .

Summary: This paper investigates conditions under which communities of cooperative agents are stable. Communities in multi-round bargaining games with evolutionary dynamics are evaluated in three main setups. The first imposes no restrictions on the agents' behavior and is shown to be easily invaded by deceitful agents. The second enables agents to refuse to bargain with deceitful agents. Nevertheless, such communities are shown to be invadable. Finally, in the third setup, a global punishment system is shown to be able to drive out deceitful invaders. The main take-home message is that, when lying is an option, agents(' communities) need to be prepared for it.