Seeing through the mess: evolutionary dynamics of lexical polysemy (2023)

Abstract: Evidently, words can have multiple senses. For example, the word mess refers to a place to have food or to a confusing situation. How exactly multiple senses emerge is less clear. In this work, we propose and analyze a mathematical model of the evolution of lexical meaning to investigate mechanisms leading to polysemy. This model features factors that have been discussed to impact the semantic processing and transmission of words: word frequency, non-conformism, and semantic discriminability. We formally derive conditions under which a sense of a word tends to diversify itself into multiple senses that coexist stably. The model predicts that diversification is promoted by low frequency, a strong bias for non-conformist usage, and high semantic discriminability. We statistically validate these predictions with historical language data covering semantic developments of a set of English words. Multiple alternative measures are used to operationalize each variable involved, and we confirm the predicted tendencies for twelve combinations of measures.

## introduction in natural language, lexical polysemy, i.e., the presence of multiple senses for a single word form, is the rule rather than the exception. the word mess, for instance, can denote, among other things, a room in which food is served, semi-liquid food, a confusing situation, or a physical state of disorder. from a communicative point of view, the fact that one form refers to multiple senses is, at first sight, sub-optimal given that ambiguity acts against successful communication. yet, populations of speakers sustain a multitude of polysemous words in their communicative systems, viz., languages. where does this semantic diversity come from? evidently, multiple senses of a word do not just simply appear. rather, word meaning evolves over time and diversifies itself as additional senses are established in the population of speakers @xcite @xcite @xcite @xcite . figure 1a displays the evolutionary tree of the semantics of the word mess. the goal of this paper is to identify conditions under which semantic diversity is enforced with the help of analytic models of the population dynamics of words. more specifically, we define a model of the spread of words through a population of speakers in which word transmission is governed by social and cognitive factors that have been suggested to be relevant to semantic change: word frequency, non-conformism, and semantic discriminability. an analytical assessment of our model shows that these factors indeed affect the tendency to establish additional senses in speaker populations. the predictions of the model are subsequently tested against empirical language data. foot_0 ## approach we consider three factors that have been suggested to be relevant to lexical transmission, semantic processing, and change (frequency, non-conformism, discriminability; see background section 3 below). the question is if and how these factors impact the diversification of a word's meaning, and, more importantly, what the potential mechanisms leading to diversification might be. our approach to tackle these questions involves two parts, namely a theoretical and an empirical part. on the theoretical level, we model the evolution of a word's meaning over time and analyze under which conditions a word's meaning is split into two separate and stably coexisting senses. this model captures the population dynamics of the semantic variants of a word, that is, we investigate how word variants spread through a population of speakers. interactions among speakers and how they transmit word variants are governed by effects of frequency, non-conformism, and discriminability that we assume beforehand. more concretely, we assume (i) that the successful transmission of a word variant from one individual to another is more likely if it is used frequently, (ii) that rare linguistic behavior can have an advantage during interactions as it stands out, and (iii) that the cognitive ability to discriminate between senses impacts what is actually perceived as the same or different linguistic behavior in interactions. based on the model, we deduce the evolutionary dynamics of a word's meaning and we derive which effects of frequency, non-conformism, and discriminability on the tendency to diversify meaning the model would predict. mathematically, our model consists of two stacked dynamical systems defined by ordinary differential equations: one that defines the longterm evolutionary dynamics of meaning, and one that defines the short-term dynamics of lexical usage in a speaker population. in combination, the model lets us show that diversification of meaning is encouraged for words subject to low frequency, high non-conformism biases, and high semantic discriminability (cf. note that although the model would, provided all necessary historical information, theoretically allow us to obtain semantic trees for single words and single semantic dimensions (like 'sentiment', 'edibility', 'objectness') similar to the one in figure 1a , this is not what we do. we also do not need to focus on the linguistic behavior of individual speakers. rather, the strength of our model is that we can use it to predict more generally under which conditions semantic diversification can occur on the population level. thus, our model reveals potential mechanisms governing the establishment of multiple coexisting senses. on the empirical level, we measure the impact of frequency, non-conformism, and discriminability on the tendency to diversify meaning. we base our analysis on historical language data covering about 3000 english words @xcite . based on this data set, we show that the predictions of the theoretical model are also supported on empirical grounds. in what follows, we motivate our choice of factors in semantic evolution, present an outline of the evolutionary model, its underlying population dynamics, and its predictions of how long-term evolution unfolds. after that, we describe the setup and outcome of our empirical account. the discussion section links both parts of this study. ## background we adopt the established definition of polysemy as a one-to-many form-meaning mapping that comes about via changes on the meaning side. the key criterion here is historical relatedness @xcite . this contrasts with the phenomenon of homophony (e.g., steak vs. stake), which results from changes on the form side (see @xcite , for a more recent discussion). multiple factors have been suggested to impact semantic change and the evolution of polysemy. in this research, we consider three of them that have been discussed in the linguistic, cognitive, and computational literature. the most prominent factor that was suggested to impact lexical semantics is the word frequency. in frequency φ non-conformism bias β discriminability δ definition number of occurrences of a word variant tendency at which a word variant is used in non-conformist vs. conformist settings ability to distinguish between two semantic uses of a word empirical measurement normalized token frequency of a word (contemporary vs. historical) fraction of frequency in nonconformist genres vs. frequency in conformist genres lexical concreteness ratings and average correlation between sentence embeddings of all sense descriptions of a word model assumptions about interactions high exposure increases chance of successful adoption in interactions imposes advantage in the propagation of the word variant when rare (common strategies lose their distinctive status) modulates effects of (non-) conformist behavior in interactions among individuals using different variants of a word prediction t ∼ 1/φ ✓ t ∼ β ✓ t ∼ δ ✓ the negative relationship between semantic change and frequency was suggested to be an effect of entrenchment @xcite @xcite . frequent words are typically learned early, and this leads to cognitive entrenchment, i.e., routinization of their processing and usage @xcite . by consequence, it is less likely for high-frequency words to change their semantic profile than this might be the case for low-frequency words that could be, perhaps erroneously, used in novel contexts. the second factor that was suggested to increase diversity in the domain of cultural evolution is that of non-conformist behavior, i.e., to behave differently than the majority does. nonconformist behavior entails what is referred to as negative frequency dependence @xcite @xcite . that is, behavioral strategies are particularly successful if they are rarely employed. non-conformist behavior is present in the linguistic domain as well. for one, several phenomena of linguistic innovations have been discussed under the notion of linguistic extravagance. speakers have been argued to choose unorthodox linguistic strategies to stand out, get recognized, and in turn to promote the transmission of their message @xcite @xcite @xcite . often, extravagant linguistic behavior extends to the semantic domain. so, originally negative words like awesome have gained a pos-itive connotation (this particularly holds true for slang with its function to signal out-group membership) @xcite . the initial salience of these words, however, becomes weaker the more abundantly they are used. non-conformism is counteracted by conformism, i.e., the disproportionally high tendency to behave just like the majority does @xcite . conformism functions as stabilizing mechanism in the transmission of communicative systems. intuitively, conformism biases should prevent semantic ambiguity since the introduction of novel senses is discouraged. the third factor, discriminability, refers to an individual's ability to discriminate between two contextual uses of a word @xcite . intuitively, if discriminability is high then individuals are less likely to view two different uses of a word as belonging to the same sense (for instance, if individuals can discriminate well, they would suggest the word mess to have two different senses in the utterances i see through the mess and i have a burger in the mess). there is much less research on the effect of semantic discriminability on language change than for the other two factors discussed above, but modeling and simulation results suggest that high discriminability entails more complex signaling inventories @xcite . from a cognitive point of view, interdependencies among word utterances are weaker if they are (perceived as being) different from each other, i.e., in the case of high discriminability (den @xcite . this also extends to interdependencies imposed by (non-)conformism pressures in interactions among individuals @xcite . ## theoretical analysis our model has the following components: we assume that words are transmitted through populations of speakers. at any point in time, each word has a set of one or more senses that we take to constitute a word's meaning. we assume that each word sense can be represented as real value on a semantic dimension. more specifically, we assume that x takes a value on a continuous scale so that @xmath0. for instance, x could represent the word's sentiment ranging from 0 (negative) to 1 (positive). then, a sense of a word like mess might have had a rather positive sentiment of about @xmath1.8 in the middle of the 19th century. speakers might introduce a new variant of an already existing word sense. this happens, for instance, when someone starts using a word sense in a slightly different context (e.g., 'potentially disgusting soup' rather than just 'soup'). when this innovative behavior is adopted by the speaker population, the semantic value x of a word sense changes within x. note that this only means that a certain semantic property of a sense changes. the introduction of an innovative variant of an already existing variant does not automatically entail that both of them coexist stably in the speaker population. stable coexistence would mean that the semantic value representing one sense splits into two variants defined by two values x 1 and x 2 within x that now represent two separate senses (i.e., a more positive and a more negative sense of mess) that are both sustained by the speaker population. such a scenario is schematically visualized in figure 1b . the crucial question is under which conditions are splits like this possible? to shed light on this question, we use the canonical equation of adaptive dynamics which models the evolution of continuous properties in populations of individuals @xcite @xcite . theoretically, the equation was shown to be linked to other well-studied models of evolution such as replicator dynamics (from evolutionary game theory) or the price equation (page and nowak, 2002; meszena et al., 2002). 2 in our context, the canonical equation of adaptive dynamics models change in a semantic property. more specifically, it defines the rate of change of x in the semantic subspace x as a function of (a) the number of users of the word variant characterized by x, and (b) the fitness of the word variant relative to another variant: here, @xmath2. furthermore, û (x) denotes the number of users u of the word sense characterized by the value x on the semantic dimension at population dynamic equilibrium. 3 intuitively, this means that a higher number of users facilitate larger amounts of change (because each user could introduce an innovation). finally, f (x, y) denotes the fitness, i.e., the growth rate of a word variant with a slightly different value of the semantic property @xmath3. this quantity is referred to as invasion fitness because it determines whether or not a new variant can successfully invade to be used by the speaker population (for a visualization of f , see figure 4 in appendix a.1.2). the term ∂f (x, y)/∂y| @xmath4. equation ( 1 ) can exhibit different types of longterm behavior @xcite . two relevant scenarios are (a) convergence to a steady-state, i.e., the existence of an attracting evolutionarily stable strategy in x and (b) diversification, i.e., the existence of an evolutionary branching point in x @xcite . the latter case implies the stable coexistence of two different variants (formally, two stable equilibria of equation ( 1 )). in the following, we investigate conditions for the existence of a branching point, as this relates to polysemy. whether or not such points exist in x for equation (1) depends on the quantities û (x) and f (x, y). both of them are determined by the underlying population dynamics of word variants. the population dynamics of a single word variant characterized fectious diseases @xcite , in economics to examine technological change leading to efficiency gains @xcite , or in cultural studies to model the evolution of religious behavior @xcite . by value x are defined by an ordinary differential equation (ode) tracing the change in the number of users u (x) of that variant. the change in the number of users is influenced by the factors outlined in section 3. the dynamics are based on the following assumptions: the ode modeling the change in the number of users u given assumptions (i-iii) reads here, ∆ = xy denotes the difference between two semantic property values x and y. a detailed description of this ode can be found in appendix a.1.1. this leads us to the following proposition 1. consider a semantic dimension x. assume that innovations in x occur in relatively small steps and that new innovations occur only after a previous innovation has either spread successfully or disappeared. given the population dynamics of single word variants characterized by a single value in x as defined in (2), the canonical equation of adaptive dynamics in (1) shows an evolutionary branching point if (i) frequency φ is sufficiently low, (ii) non-conformism bias β is sufficiently high, (iii) and discriminability δ is sufficiently high. a proof is shown in appendix a.1. in short, the model predicts the tendency to diversify meaning to be (i) negatively associated with frequency, (ii) positively associated with non-conformism biases and (iii) positively associated with discriminability. ## empirical analysis we test if predictions (i-iii) resulting from our theoretical analysis are supported empirically. to do so, we investigate a sample of words and check if (i) frequency, (ii) non-conformism biases, and (iii) discriminability are associated with the tendency to diversify meaning in this sample. that is, our model would predict words with high nonconformism biases, high discriminability between senses, and low frequency to become more polysemous over time. we base our analysis on a set of 3358 words compiled by @xcite containing information about the semantic development of words. in this data set, each word is accompanied by (i) a list of the descriptions of all of its senses foot_3 taken from the oxford english dictionary (oed), and (ii) for each decade in the 19th and 20th century, the estimated probabilities for all senses of that word, originally derived from the corpus of historical american english (coha) @xcite through sentencebert embeddings. the development of the distribution of senses of the word mess is shown in figure 2a . after only considering words with subjective concreteness ratings in @xcite al. (2014) (see 5 3 below), we ended up with a set of 3165 words. we restrict our analysis to all decades between 1850 and 2000 since earlier subcorpora of coha are relatively small and display a different distribution of genres. ## discussion and conclusion in this paper, we have analyzed the evolution of lexical polysemy in two ways: first, by means of a mathematical model that captures the population dynamics of word variants; second, through an empirical analysis covering the factors implemented in the population-dynamic model. the approaches yield converging outcomes. as far as frequency is concerned, our results are in line with @xcite observation that frequency interacts with semantic change, namely in such a way that frequency in fact impedes change and fosters semantic stability. in a similar vein, @xcite have shown that frequent concepts resist change more easily. note that our way of conceptualizing semantic change subtly differs from that in @xcite who measured the impact of frequency on any type of change in word meaning. in contrast, the focus of our analysis is on a word's tendency to become more polysemous over time. note, though, that our model not only predicts that low frequency entails, under certain circumstances, semantic diversification but also that high frequency leads to the existence of an attracting evolutionarily stable strategy in the semantic space, hence eventually demoting semantic change for high-frequency words altogether. that is, our model provides a mechanistic explanation for the empirical observation made in the literature: high-frequency words are more likely to adopt stable states in the semantic trait space that optimize ease of adoption. 12 this explanation is an interesting alternative to the predominant account, which traces the positive relationship between frequency and semantic stability back to the mechanism of cognitive entrenchment @xcite . here, the argument is that frequent words are strongly entrenched, i.e., subject to routinized perception, processing, and production so that they can resist change more easily @xcite . 13 importantly, our account is agnostic with respect to mechanisms of entrenchment. in our model, the observed evolutionary behavior emerges from the assumption that frequent words are acquired more easily. as a final note on the effect of frequency, it is worth mentioning that @xcite result was criticized as being a methodological artifact that arises from effects of frequency on cosine-similarities between word embeddings @xcite . while we do acknowledge this criticism it is important to emphasize that, from a methodological point of view, neither the theoretical nor the empirical account in our approach can be invalidated by these effects. hence, our research provides independent evidence for @xcite findings. in addition to the effect of frequency, our model predicts a positive relationship between nonconformism biases and the tendency of a word to establish more senses over time. the prediction is supported by our empirical account in ten out of twelve configurations. in the model, the mechanism that promotes the stable coexistence of two semantic senses is that of negative frequency dependence, i.e., an advantage of lexical variants if and as long as they are rare. the prediction that this mechanism promotes polysemy is intuitive, and not very surprising for that matter. if rare variants have an advantage in their proliferation then this facilitates the establishment of innovative uses. this resembles the argument that is brought forth in the discussion about the role that extravagant usage plays in lexical change @xcite @xcite . we have also shown empirically and theoretically (in line with @xcite ) that non-conformism leads to diversifica-tion of meaning only if discriminability is high. if discriminability is low then semantic innovations that are close to an established sense will be effectively perceived as coinciding with that sense. consequently, they will suffer from the same disadvantage imposed by negative frequency dependence that all common types are subject to, even though these innovations are initially rare. we consider it as a main strength of our approach that we do not only support (hypothesized) relationships between factors in semantic change and polysemy on empirical grounds, but that we can also point at the mechanisms driving semantic change based on our theoretical analysis. this is so, in particular, due to the two-level architecture of the mathematical model, which combines short-term population dynamics driven by linguistic interactions that are subject to assumed biases with evolutionary long-term dynamics of lexical semantics. in that sense, our work goes beyond correlational accounts of polysemy.