Quantifying Cognitive Factors in Lexical Decline (2021)

Abstract: We adopt an evolutionary view on language change in which cognitive factors (in addition to social ones) affect the fitness of words and their success in the linguistic ecosystem. Specifically, we propose a variety of psycholinguistic factors—semantic, distributional, and phonological—that we hypothesize are predictive of lexical decline, in which words greatly decrease in frequency over time. Using historical data across three languages (English, French, and German), we find that most of our proposed factors show a significant difference in the expected direction between each curated set of declining words and their matched stable words. Moreover, logistic regression analyses show that semantic and distributional factors are significant in predicting declining words. Further diachronic analysis reveals that declining words tend to decrease in the diversity of their lexical contexts over time, gradually narrowing their ‘ecological niches’.

## introduction many researchers, from schleicher (1863) up to the present @xcite @xcite @xcite , have drawn analogies between biological evolution and the evolution of languages-their structure, their semantics, and their lexicons. lexically speaking, as schleicher first pointed out, diachrony can be viewed as a struggle for survival by individual words whose propagation into future generations is contingent on their continued fitness for one or more niches in the ecology of the speech community-as determined by a host of factors. here we study the question of lexical decline-a gradual decrease in frequency and ultimate obsolescence of words. what explains that the word 'amusements' has declined in the last 200 years, but 'foundations' has not, as shown in figure 1 (along with other similar pairs)? social factors clearly play a role, as changes in culture and technology may lead words to fall in and out of use. but cognitive and linguistic factors also influence lexical survival (e.g., @xcite . words that are semantically similar to many other words may come to be used less because of intense competition in the cognitive process of lexical access @xcite . words that can occupy many niches, distributionally speaking, should have better chances of being learned and used, and therefore perpetuated, than words that are confined to a narrow range of contexts or senses @xcite . on the other hand, words that are phonologically very different from other words may suffer because they are more difficult to access, sitting as they do at the formal fringes of the mental lexicon @xcite . in other words, we suggest that semantic, distributional, and phonological factors all play a role in the natural selection of words. while attention to predicting which words will emerge, live, and die goes back to schleicher, there is relatively little computational work on this subject (examples include @xcite @xcite . in particular, little attention has been paid to the factors that contribute to lexical decline (but see @xcite for related work on lexical replacement). this is unfortunate because understanding this phenomenon answers an important scientific question about language change-how lexicons become as they are. we ground these phenomena in an evolutionary model of linguistic diachrony in which fitness is influenced by independently motivated cognitive processes like lexical access. our study spans 20 decades and three languages. we find that there are consistent factors-semantic, distributional, and phonological-that predict whether a word is likely to substantially decline in frequency. we propose that our observations are consistent with a model where there is a feedback loop between cognition and usage driving the diachronic development of lexicons. 1 2 related work there is a vast body of research on lexical change of various kinds; in this section we focus on work involving the birth and death of words, as it is most closely related to our study here. lexical neology-introduction of new wordsis one of the most evident types of lexical change. various computational studies have suggested a range of factors underlying the phenomenon of neology, including semantic, distributional, and phonological influences. @xcite show that lexical neology can be partly explained by the factor of supply-new words tend to emerge in areas of semantic space where they are needed most, i.e., areas exhibiting relative sparsity. drawing on theories of patterns of word growth @xcite @xcite , additional studies suggest that (among other factors) greater linguistic distribution across individuals and topics plays a significant positive role in the fate of novel lexical items in online forums @xcite . considering phonological factors, @xcite show that new words emerge under the joint constraints 1 all data and code is available at https://github .com/ellarabi/linguistic_decline . compared to research on neology, the work on lexical obsolescence and loss is relatively sparse. while the study of neology often draws on occurrence of new word forms in contemporary corpora, in contrast, the study of lexical loss inherently relies on the availability (and the quality) of large diachronic textual resources. tichỳ (2018) proposes a methodology for identifying declining words in such a corpus, and performs qualitative analysis of a sample of such words, focusing on spelling standardization and changes in wordformation strategies. using the google-books dataset @xcite study the 'death rate' of words primarily stemming from misspellings and print errors typical to historical corpora, focusing on the rate and not the causes of linguistic decline. other work touches on lexical decline less directly, but explores potential predictive factors (as we do) in related processes-factors that may also play a role in decline. @xcite consider the factors that influence meaning shift-rise and decline of meanings within a word (rather than of words themselves)-and find that both word frequency and number of meanings play a role. turney and mohammad (2019) track the evolution of 4k english synsets, attempting to predict a synset 'leader'-the member of the synset with highest frequency. they find the current 'leadership' of a word to be the most predictive factor of its future status as a 'leader', again illustrating the driving force of frequency in lexical status. however, while a word may become a synset leader at the expense of other words, this work does not perform a systematic study of factors predictive of lexical decline. finally, @xcite conducted a study of lexical replacement-a closelyrelated but narrower phenomenon than lexical decline-exploring similar semantic factors to those we investigate here using a markedly different methodology. their study is focused on a small set of core vocabulary in indo-european languages (' @xcite , from @xcite ). our research here addresses a much broader phenomenon of general lexical decline, and proposes a wider range of factors influencing that process. ## overview of our approach motivated by the perspective of the lexicon as an evolving ecosystem, in which words are subject to various cognitive pressures that can influence their 'survival', we aim here to identify factors that may be indicative of words that are likely to decline. specifically, we propose factors that, when calculated at a given time in history, t (in our study, 1800-1810), are hypothesized to be predictive of lexical decline during a subsequent stretch of time, up to t+n (here 2000-2010). first we note one potential factor whose influence on decline is not explored here: that of a word's frequency. having seen that (relatively higher) frequency of a word is the single best predictor of future (relatively higher) frequency (turney and mohammad, 2019), a natural hypothesis is that lower frequency may conversely be predictive of future decline. however, since relatively low frequency may indicate a word already 'on its way out', we instead control for frequency: given words of similar frequency, we explore what other properties are most predictive of which will subsequently decline and which survive. as noted in §1, we consider that semantic, distributional, and phonological factors all may play a role in lexical decline, due to their influence on the ease or difficulty of learning and accessing of words, which may impact a word's continued role in the lexical ecosystem. here we provide the motivation for the factors we consider; §4.3 provides detail on how they are computed. while this discussion may suggest causal relationships (e.g., words decline because their lexical access is more difficult), our subsequent analyses focus on correlations of the factors with decline, and are thus agnostic with respect to causality. we consider several semantic factors, drawing on inspiration from the acquisition and processing literatures. first, we consider the role of the semantic space a word occurs in. while some work has found that lexical access is facilitated by having dense semantic neighborhoods (having many closely related words) @xcite , other work has noted their inhibitory effect on semantic processing @xcite , in line with findings of inhibitory competition in phonological neighborhoods @xcite . such inhibitory effects may underlie the observation that words in semantically dense (i.e., more competitive) environments are more likely to be driven out, to the benefit of others that can potentially be used to express roughly the same meaning (e.g., @xcite . thus, similarly to @xcite , we estimate the density of a word's immediate semantic neighbourhood, where we predict words with a higher semantic density to be more likely to decline. next, we consider properties of the semantics of the word itself. psycholinguistic studies have found that more concrete words-roughly, those referring to a perceptible entity-are learned and retrieved more easily (e.g., @xcite . moreover, concrete words may form a more stable subset of the lexicon @xcite ; cf. a similar finding in @xcite using imageability ratings, a notion that is highly correlated with concreteness). because words conveying a more concrete meaning appear more likely to survive, we consider the level of concreteness of a word as a second semantic factor, where lower concreteness predicts a higher chance of decline. additionally, having a higher degree of polysemy has been shown to have a facilitatory effect on a word's lexical access, due to multiple related senses contributing to aggregate activation of the word @xcite . access to a word across many senses may similarly lead to greater survivability @xcite , and we thus predict that words with a higher number of meanings will be less likely to fall out of use in a language. in addition to the influence of semantic properties, others have proposed a central role for distributional factors in lexical learning and processing @xcite . in particular, words that occur in more varied contexts are easier both to learn @xcite and to access @xcite . in addition, words with broader topical dissemination tend to become more robustly entrenched into the lexicon @xcite ; conversely, we expect that words that occur in a narrower range of contexts will be more apt to fall out of use. this too follows our lexical evolution perspective: just as species that can occupy many niches in a natural ecology are more likely to survive, generation to generation, lexemes that occupy many niches in the linguistic ecology are less likely to face extinction (or decline). we adopt the distributional factor of contextual diversity to model this fact. like semantic and distributional effects, phonological effects are also known to interact, in a complex way, with lexical processing, and we hypothesize that such factors may also be predictive of lexical decline. for example, psycholinguistic studies have found that phonotactically typical words are recognized more quickly than atypical words @xcite . we correspondingly predict that phonological typicality will be associated with lower rates of lexical decline. 2 as with semantic neighborhoods, psycholinguistic experiments have also found mixed effects of phonological neighborhoods on lexical processing: both competition among similar phonological forms (as noted above, @xcite , as well as potential facilitation from having a higher number of phonologicallyclose neighbors @xcite @xcite . given a preponderance of evidence of facilitatory effects on lexical processing, we predict that phonological density will be inversely correlated with lexical decline. 3 finally, we predict that words with greater phonological complexity-for our purposes, longer in terms of the number of syllables-will be more likely to decline. this hypothesis follows from the speculation that words are processed as sequences of syllables rather than sequences of phonemes, and that longer words are more effortful to process. specifically, we hypothesize that words with higher number of syllables (per phoneme) will be more likely to decline. in order to assess the factors both individually and as a collection, we perform two kinds of analyses. we identify a set of words that decline in usage over a 200-year period, and pair those with a set of words that are stable in frequency over the same period. we first consider whether the values (in the initial decade) of each of these proposed factors differs in the expected direction between the declining and stable words. next, we see which factors may be most explanatory of decline when the set of 7 factors are used collectively in a logistic regression analysis. in §4 we describe how we select our declining and stable words, and estimate the above factors, and in §5 we present the results of these two analyses. we follow this in §6 with further diachronic analysis of the pattern of contextual usage in how words decline. ## materials and methods our goal is to explore whether the factors identified above can indeed distinguish words at a time t that will decline over a subsequent period of time t + n, from words that remain relatively stable over that same time period. to this end, we develop measures to identify a set of words that have declined over a historical period, and a set of stable words for comparison. however, we cannot form our experimental word sets by simply selecting words randomly from each of these lists. because confounding lexical properties (such as frequency) may interact with our identified factors of interest, we must adopt a more controlled approach, standard in cognitive research, of matching our declining and stable words on a set of potential confounds. in §4.1, we first motivate our approach to forming our experimental itemspairs of declining and stable words matched on covariate properties. we then detail how those word pairs are selected ( §4.2), and finally explain how we estimate our identified factors of interest over these experimental items ( §4.3). ## diachronic analysis of lexical loss we next explore whether there are diachronic patterns in the contextual dissemination of words, over the 21 decades of our data, that differ between declining and stable words. a specific question is whether a word falling out of use in a language uniformly reduces its frequency across the entire diversity of its contextual environments, or if it instead gradually 'abandons' particular contextual niches, thereby narrowing its linguistic dissemination. we hypothesize that declining and stable words differ in the diachronic trend of their cdiv values; specifically, that declining words gradually fade out from certain contextual usages, thereby reducing the number of linguistic environments they populate @xcite . to corroborate this, we perform diachronic analysis of contextual diversity. we approach this question by using linear regression to fit a temporal trend line over each word w's cdiv values, across the 21 decades -that is, regressing cdiv w t on @xmath0..21]. we expect this trend line to show a decreasing tendency for declining words, indicative of contextual shrinkage, and a stable or increasing tendency for stable words, indicative of stability or growth of contexts. in particular, the regression line coefficients of the declining set should be significantly lower than that of stable words. however, we must adopt a multiple regression approach that incorporates variables (other than time) that could also contribute to variation in a word's cdiv values. specifically, we identified two properties that may bias the cdiv of a word when comparing across decades: 16 (1) the number of unique books used for data extraction in that decade, and (2) the frequency of the word in that decade. first, a greater number of unique per-decade books is likely to increase contextual diversity, since a higher number of distinct literature sources raises the chance of a wider range of contextual domains. second, lower frequency of a word is likely to negatively affect its contextual diversity-the lower the frequency, the less opportunity there is for a word to occur in different contexts. we address these potential confounds by using the per-decade values of each of these properties as additional independent variables, along with time t, in a multiple regression. this yields the following regression model: figure 4 presents two boxplots of the distributions of the two sets of β 3 coefficients-for the 300 declining words (left) and 300 stable words (right) in our english dataset. 18 the means of the two distributions significantly differ from each other, as well as from 0, when applying a wilcoxon test (@xmath1.001 for all tests). we thus find support for the claim that declining and stable words have different diachronic patterns of contextual diversity. the negative mean and median of the coefficients for the declining words (@xmath2.0006; @xmath3.0005) further support our specific hypothesis of diachronic contextual loss for these words. in contrast, the coefficients of stable words 18 similar results were found for french and german. ## conclusions we have proposed factors of various typessemantic, distributional, and phonological-and shown that the semantic and distributional features are robust predictors of whether a word will remain stable in frequency or fall into decline. in particular, we have focused on factors that can influence the cognitive processing of words, affecting how likely they are to be used and learned. given that broad external influences, such as language contact, as well as social and technological developments, are known to have a massive effect on the content of vocabulary, this study constitutes an important demonstration of the potential influence of internal cognitive mechanisms on the 'survivability' of words. our findings suggest that factors affecting a word's trajectory are more likely to be semantic or distributional than phonological, perhaps because speakers or writers, when they are looking for a word, are guided primarily by syntactic and semantic criteria. the behavior of most of the factors we proposed matches the expectations for declining vs. stable words that were motivated by the psycholinguistic literature. our findings are consistent with an evolutionary view where psycholinguistic factors influence the 'reproductive fitness'-the fitness for self-perpetuation-of words. this, in turn, supports a broader evolutionary research agenda in historical linguistics. by nserc grant rgpin-2017-06506 to suzanne stevenson. this material is based in part on research sponsored by the air force research laboratory under agreement number fa8750-19-2-0200. the u.s. government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. the views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the air force research laboratory or the u.s. government. the other languages. although the accuracy for english is not high, it is well above random, and it must be remembered that we are only testing our cognitive features, and not including the myriad social and cultural influences on lexical change. the results here further support our findings in section 5.2, indicating again that the features we have proposed have useful predictive power in identifying the declining word of a pair that shares similar frequency, length, and pos.