Introduction
This letter arises from two intriguing questions about human language. The first question is: To what extent language, and also language evolution, can be viewed as a graph-theoretical problem? Language is an amazing example of a system of interrelated units at different organization scales. Several recent works have stressed indeed the fact that human languages can be viewed language as a (complex) network of interacting parts BIBREF0, BIBREF1, BIBREF2, BIBREF3. Within the graph-based approach to human language, one may think word-meaning mappings (that is, vocabularies) as bipartite graphs, formed by two sets: words and meanings BIBREF2.
The second question is: What is the nature of the language evolution process that affects the shape of graph-based language representations? To answer this question, we assume that human communication is constrained (at least) by two forces BIBREF2: one that pushes towards communicative success and another one that faces the trade-off between speaker and hearer efforts. The first force involves simpler decentralized models of linguistic interactions within populations of artificial agents, endowed with minimal human cognitive features, negotiating pieces of a common language: the so-called language games BIBREF4, BIBREF5, BIBREF6, BIBREF7. In the simplest language game, the naming game BIBREF8, BIBREF9, at discrete time step a pair of players (typically one speaker and one hearer) interacts towards agreement on word-meaning associations.
Next, we also consider the communication cost to establish word-meaning mappings. G. Zipf referred to the lexical trade-off between two competing pressures, ambiguity and memory, as the least effort principle BIBREF10, BIBREF11: speakers prefer to minimize memory costs; whereas, hearers prefer to minimize disambiguation costs. As remarked by several works, an interesting proposal has stated that human-like vocabularies appear as a phase transition at a critical stage for both competing pressures BIBREF12, BIBREF13, BIBREF14, BIBREF15, BIBREF16. The appearance of a drastic stage of competing pressures can be understood moreover as an explanation of the empirical Zipf's law, which establishes a dichotomy between low-memory words (like the word “the") and low-ambiguity words (like the word “cat"). Within a statistical point of view, text corpora evidence strong scaling properties in word-frequencies BIBREF17, BIBREF18, BIBREF19, BIBREF20, BIBREF21, BIBREF22, BIBREF23, BIBREF24.
The main aim is to address a decentralized approach (based on a previous proposal of two authors of this letter BIBREF25) to the emergence of Zipfian properties in a human-like language, while players communicate with each other using bipartite word-meaning mappings. To structurally characterize changes in the system, our methodology is mainly based on a phase transition description, arising from both classical statistical mechanics tools and graph-mining techniques. We run numerical simulations over simple population topologies. We apply graph-mining techniques, particularly a clustering notion for bipartite graphs BIBREF26.
The model ::: Key concepts on (bipartite) graphs
A bipartite graph is a triple $B=(\top ,\bot ,E)$, where $\top $ and $\bot $ are two mutually disjoint set of nodes, and $E \subseteq \top \times \bot $ is the set of edges of the graph. Here, $\top $ represents the set of word nodes, whereas $\bot $ represents the set of meaning nodes. We remark that edges only exist between word nodes and meaning nodes. A classical useful tool in graph theory is the matrix representation of graphs. Here, we only consider the adjacency matrix. Let us denote by $A = (a)_{wm}$ the adjacency matrix for the (bipartite) graph $B$. From the bipartite sets $\top $ and $\bot $, representing respectively word and meaning nodes, we define the rows of $A$ as word nodes, and the columns as meaning nodes, where $(a)_{wm}=1$ if the word $w$ is joined with the meaning $m$, and 0 otherwise.
The neighbors of order 1 of $u\in \top $ are the nodes at distance 1: $N(u)=\lbrace v\in \bot : uv \in E\rbrace $ (if $u \in \bot $ the definition is analogous). Let us denote by $N(N(u))$ the set of nodes at distance 2 from $u$. The degree $d(u)$ of the node $u$ is simply defined by $d(u)=|N(u)|$. We denote by $d^{\max }_W = \max _{w \in W} d(w)$ the maximum degree for word nodes ($\top $). Analogously, $d^{\max }_M = \max _{m \in M} d(m)$ the maximum degree for meaning nodes ($\bot $).
The notion of clustering coefficient (in classical graphs) captures the fact that when there is an edge between two nodes they probably have common neighbors. More generally, such notion captures correlations between neighborhoods. Based on this point of view, BIBREF26 proposed a clustering coefficient notion for bipartite graphs:
where $cc(u,v)$ is a notion of clustering defined for pairs of nodes (in the same set $\top $ or $\bot $):
Interestingly, $cc(u,v)$ captures the overlap between the neighborhoods of $u$ and $v$: if $u$ and $v$ do not share neighbors $cc(u,v)=0$; if they have the same neighborhood $cc(u,v)=1$.
To give an overall overview of bipartite clustering for the graph $B$, the average bipartite clustering reads
The model ::: Basic elements of the language game
The language game is played by a finite population of participants $P=${1,...,p}, sharing both a set of words $W=\lbrace 1,...,n\rbrace $ and a set of meanings $M=\lbrace 1,...,m\rbrace $. Each player $k\in P$ is endowed with a graph-based word-meaning mapping $B^k=(\top ^k,\bot ^k,E^k)$. In our case, $B^k$ is a bipartite graph with two disjoint sets: $\top ^k \subseteq W$ (word nodes) and $\bot ^k \subseteq M$ (meaning nodes). Each player $k \in P$ only knows its own graph $B^k$.
Two technical terms are introduced. First, we say that a player $k \in P$ knows the word $w \in W$ if $w \in \top ^k$. Clearly, this definition is equivalent to the existence of the edge $wm \in E^k$, for some $m \in \bot ^k$. Second, the ambiguity of the word $w$, denoted $a(w)$, is defined as its node degree $d(w)$.
The model ::: Language game rules
The dynamics of the language game is based on pairwise speaker-hearer interactions at discrete time steps. At $t \geqslant 0$, a pair of players is selected uniformly at random: one plays the role of speaker $s$ and the other plays the role of hearer $h$, where $s,h \in P$. Each speaker-hearer communicative interaction is defined by two successive steps. The speaker-centered STEP 1 involves the selection of a meaning and a word to transmit them. At STEP 2, the hearer receives the word-meaning association and both speaker and hearer behave according to either repair or alignment strategies.
STEP 1. To start the communicative interaction, the speaker $s$ selects the topic of the conversation: one meaning $m^* \in M$. To transmit the meaning $m^*$, the speaker needs to choose some word, denoted $w^*$. There are two possibilities for the selection of $w^*$:
if the edge $wm^* \notin E^s$ for any $w \in \top ^s$, the speaker chooses (uniformly at random) the word $w^*$ from the set $W$ and adds the edge $w^*m^*$ to the graph $B^s$;
otherwise, if $w^*m^* \in E^s$ for some $w^* \in \top ^s$, the speaker calculates $w^*$ based on its interests, that is, based on its own conflict between ambiguity and memory.
To calculate $w^*$ for the second case ($w^*m^* \in E^s$), the speaker behaves according to the ambiguity parameter $\wp \in [0,1]$. Let $random \in [0,1]$ be a random number. Then, two actions are possible:
if $random \geqslant \wp $, the speaker calculates $w^*$ as the least ambiguous word
otherwise, the speaker calculates $w^*$ as the most ambiguous word
The speaker transmits the word $w^*$ to the hearer.
STEP 2. The hearer behaves as in the naming game. On the one hand, mutual speaker-hearer agreement (if the hearer knows the word $w^*$) involves alignment strategies BIBREF9. On the other hand, a speaker-hearer disagreement (if the hearer does not know the word $w^*$) involves a repair strategy in order to increase the chance of future agreements (that is, for $t^{\prime }>t$). More precisely,
if the hearer knows the word $w^*$, both speaker and hearer remove all edges formed by $wm^*$, where $w$ respectively belongs to $\top ^s \setminus \lbrace w^*\rbrace $ and $\top ^h \setminus \lbrace w^*\rbrace $.
otherwise, the hearer adds the edge $w^*m^*$ to its graph $B^h$.
Methods
The population of agents is located on the vertices of a complete graph of size $|P|=100$, typically called the mean field approximation. For the description of other simple graph topologies, see the caption of Fig. FIGREF15. The population shares both a set of $n=|W|=128$ words and a set of $m=|M|=128$ meanings. Starting from an initial condition in which each player $k \in P$ is associated to a bipartite graph $B^k$ where $B^k_{ij} = 1$ or $B^k_{ij} = 0$ with probability 0.5 (put differently, for each possible edge $ij$, $i \in W$ and $j \in M$, exists with probability 0.5), the dynamics performs a speaker-hearer interaction at each discrete time step $t \geqslant 0$. The bipartite word-meaning mappings $B^s$ and $B^h$ are then reevaluated according to communicative success. All results consider averages over 10 initial conditions and $3\times 10^5$ time steps. We denote by $t_f$ the final time step. The ambiguity parameter $\wp $ is varied from 0 to 1 with an increment of 1%.
Results ::: Three structural phases in language formation
Two key quantities have been analyzed for different values of $\wp $: the average population clustering $cc$,
which captures the average correlation between word neighborhoods; and the (effective) lexicon size at time step $t$, $V(t)$, defined as BIBREF12, BIBREF25
where $V(t)=1$ if $|\top ^k|=n$, while $V(t)=0$ if $|\top ^k|=0$.
Three clear domains can be noticed in the behavior of $\langle cc \rangle $ versus $\wp $, at $t_f$, as shown in Fig. FIGREF15 (blue squares). Phase I: $\langle cc \rangle $ increases smoothly for $\wp < 0.4$, indicating that for this domain there is a small correlation between word neighborhoods. Full vocabularies are attained also for $\wp < 0.4$; Phase II: a drastic transition appears at the critical domain $\wp ^* \in (0.4,0.6)$, in which $\langle cc \rangle $ shifts abruptly towards 1. An abrupt change in $V(t_f)$ versus $\wp $ is also found (Fig. FIGREF16) for $\wp ^*$; Phase III: single-word languages dominate for $\wp > 0.6$. The maximum value of $\langle cc \rangle $ indicate that word neighborhoods are completely correlated.
Results ::: Bipartite graphs to visualize the phase transition
We now shift our focus from graph-based measures towards a holistic level in which we illustrate the described phase transition using bipartite graph representations of language formation. We stress the fact that our framework based on a language game with players endowed with bipartite word-meaning mappings is able to visualize the structural changes of the three phases (I, II and III). Fig. FIGREF18 display, from top to bottom, the bipartite word-meaning mappings for ambiguity parameters $\wp $ in $\lbrace 0.1, 0.52,1\rbrace $. As expected, there are radical structural changes between bipartite graphs associated to such ambiguity parameters. Full vocabularies are attained for $\wp =0.1$ (Phase I), located at the hearer-centered phase. Zipfian vocabularies seem to appear for $\wp =0.52$ (Phase II), where speaker and hearer costs have a similar value. Finally, a single-word vocabulary (that is, one word, several meanings) is exhibited for $\wp =1$ (Phase III).
Results ::: Critical values of energy
The appearance of the three-phased language behavior described here is closely related to previous results of two authors of this letter BIBREF25. Indeed, in the cited paper the energy-like functional $e_{KL}$ (a kullback-leibler-based measure) is minimized around the parameter $\wp \approx 0.5$. Remarkably, here it is showed numerically that around the critical parameter $\wp \approx 0.52$ a drastic transition for both the effective vocabulary and the bipartite average clustering tends to appear (see Fig. FIGREF18).
A first strategy to profound on the problem established between the phase transitions described here and energy-based approaches, is to measure the information-theoretic energy $\Omega _\wp (tf)$ (as defined in BIBREF12) as a function of the parameter $\wp $. $\Omega _\wp (tf)$ is a combination of the respective efforts of speakers and hearers: $\Omega _\wp (tf)=\wp H(R|S)+(1-\wp )H(S)$. Figure FIGREF20 showed that $\Omega _\wp (t_f)$ is minimized around $\wp \approx 0.5$. This suggests a new way to understand language evolution and formation, by reconciling models focused on self-organization and information-theoretic accounts.
Discussion
In this letter, we have described a decentralized model of the emergence of Zipfian features in a human-like language, where agents play language games communicating with bipartite word-meaning mappings. The model evidences a phase transition that corresponds to the formation of a human-like vocabulary satisfying Zipfian word-meaning properties. Our central graph-mining tool has been a notion of clustering for bipartite graphs. This function allowed us to suggest that the drastic transition is, in some sense, a qualitative transition in word's correlations.
To further understand the nature of the described transition, we remark a recent proposal BIBREF28, reinterpreting an old question about language learning with a novel approach: if language learning by a child involves setting many parameters, to what extent all these need to be innate? According to the Principles and Parameters theory BIBREF29, children are biologically endowed with a general “grammar" and then the simple exposition to a particular language (for example, Quechua) fixes its syntax by equalizing parameters. This debate was illuminated by proposing a statistical mechanics approach in which the distribution of grammar weights (where language is modeled by weighted context-free grammars) evidences a drastic transition. Language learning is, for this proposal, a transition from a random model of grammar parameter-weights to the one in which deep structure (that is, syntax) is encountered.
Here, the language learning problem is situated in a decentralized process, with agents negotiating a common word-meaning mapping exhibiting Zipfian scaling properties. Interestingly, our approach can shed light on the debate opened by BIBREF28. Indeed, our model questioned, first, the fact that language learning is traditionally viewed as an individual process, without any consideration of population structure (in general, language games question this fact). Secondly, we argue that our view pointed out the minimal necessity of cognitive principles for cultural language formation: the least effort principle. We hypothesize that players only need the most basic cognitive features for language learning (and formation) and the rest is an emergent property from the local speaker-hearer interactions. It is interesting to remark that several works have stressed the fact that language formation can be viewed as a phase transition within an information-theoretic approach BIBREF12, BIBREF13, BIBREF14, BIBREF15, BIBREF16.
Future work could explore an intriguing hypothesis: Zipfian properties have strong consequences for syntax and symbolic reference. BIBREF30 has proposed indeed that Zipf's law is a necessary precondition for full syntax, and for going beyond simple word-meaning mappings. They hypothesized moreover that the appearance of syntax have been as abrupt as the transition to Zipf's law. This is a goal for future work: to propose a decentralized model in which agents (constrained by specific cognitive features) develop a Zipfian language that acts as a precondition for the abrupt transition to simple forms of syntax (based, for example, on BIBREF31). Another related research line arises from models assuming the interplay between maximization of the information transfer and minimization of the entropy of signals (see, for example, BIBREF32). As previously remarked, these models evidence a lack of population structure. Current work asks how a community of individuals playing the language game proposed here can minimize the energy functional $\Omega (\lambda )$ of word-meaning mappings. We may hypothesize that reaching global consensus at the critical phase (that, is for $\wp \approx 0.5$) is closely related to the global minima of $\Omega (\lambda )$ (as a first approach to this problem, see Section SECREF19). This idea opens fascinating novel ways to study human language, reconciling models seeing communication as a global minima of information entropic energies and models focused on populations self-organizing themselves towards a shared consensus.
F.U. thanks CONICYT Chile for financial support under the Grant 3180227.

Question: What are three possible phases for language formation?
Answer: Phase I: $\langle cc \rangle $ increases smoothly for $\wp < 0.4$, indicating that for this domain there is a small correlation between word neighborhoods. Full vocabularies are attained also for $\wp < 0.4$, Phase II: a drastic transition appears at the critical domain $\wp ^* \in (0.4,0.6)$, in which $\langle cc \rangle $ shifts abruptly towards 1. An abrupt change in $V(t_f)$ versus $\wp $ is also found (Fig. FIGREF16) for $\wp ^*$, Phase III: single-word languages dominate for $\wp > 0.6$. The maximum value of $\langle cc \rangle $ indicate that word neighborhoods are completely correlated