Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slang (2022)

Abstract: Languages are continuously undergoing changes, and the mechanisms that underlie these changes are still a matter of debate. In this work, we approach language evolution through the lens of causality in order to model not only how various distributional factors associate with language change, but how they causally affect it. In particular, we study slang, which is an informal language that is typically restricted to a specific group or social setting. We analyze the semantic change and frequency shift of slang words and compare them to those of standard, nonslang words. With causal discovery and causal inference techniques, we measure the effect that word type (slang/nonslang) has on both semantic change and frequency shift, as well as its relationship to frequency, polysemy and part of speech. Our analysis provides some new insights in the study of language change, e.g., we show that slang words undergo less semantic change but tend to have larger frequency shifts over time.

## introduction language is a continuously evolving system, constantly resculptured by its speakers. the forces that drive this evolution are many, ranging from phonetic convenience to sociocultural changes @xcite . in particular, the meanings of words and the frequencies in which they are used are not static, but rather evolve over time. several previous works, in both historical and computational linguistics, have described diachronic mechanisms, often suggesting causal relationships. for example, semantic change, i.e. change in the meaning of a word, has both been suggested to cause @xcite and be caused by @xcite polysemy, while also part figure 1 : we observe very different change dynamics for the slang word "duckface" and the nonslang word "inclusive." "inclusive" has acquired a new meaning, reflected in a high semantic change score of 0.77 as measured by our model. "duckface" undergoes little semantic change, scored 0.39 by our model, while its usage frequency varies greatly. in this work, we focus on the linguistic evolution of slang, defined as colloquial and informal language commonly associated with particular groups @xcite , and use a causal framework to compare the change dynamics of slang words to those of standard language. more specifically, we compare the semantic change as well as the changes in frequency, i.e., frequency shift, over time between slang words and standard, nonslang words. we learn a causal graphical model @xcite to assess how these variables interact with other factors they have been previously found to correlate with, such as frequency, polysemy and part of speech @xcite . having discovered a graph, we proceed to use do-calculus @xcite to evaluate the causal effects of a word's type (slang/nonslang) on semantic change and frequency shift. semantic change is measured using the average pairwise distance (apd) @xcite between time-separated contextualized representations, which were obtained from a twitter corpus via a bi-directional language model @xcite . our method builds on recent semantic change literature @xcite , with novel additions of dimensionality reduction and a combined distance function. by deploying a causal analysis, we establish that there is not just an association, but a direct effect of a word's type on its semantic change and frequency shift. we find that a word being slang causes it to undergo slower semantic change and more rapid decreases in frequency. to illustrate, consider the slang word "duckface" and the nonslang word "inclusive" as shown in figure 1 . duckface is a face pose commonly made for photos @xcite in the early 2010s, and while it has largely decreased in frequency since, its meaning has not changed. in contrast, the nonslang word "inclusive" has developed a new usage in recent years @xcite and was given a high semantic change score by our model. our analysis also sheds light on a couple of previous findings in the diachronic linguistics literature. we find support for the s-curve theory @xcite , showing a causal effect from a word's polysemy to its frequency. this relationship is evident in the increase in frequency that the word "inclusive" displays in figure 1 after it develops a new meaning @xcite . however, similar to @xcite , we do not find causal links to semantic change from frequency, polysemy, or pos, which have been suggested in previous works @xcite . in summary, our main contributions are threefold: (i) we formalize the analysis of change dynamics in language with a causal framework; @xcite we propose a semantic change metric that builds upon contextualized word representations; and (iii) we discover interesting insights about slang words and semantic change -e.g., showing that the change dynamics of slang words are different from those of nonslang words, with slang words exhibiting both more rapid frequency fluctuations and less semantic change. ## causal methodology for change dynamics examining change dynamics through a causal lens helps determine the existence of direct causal effects, by modeling the interactions between variables. for example, it allows us to conclude whether word type directly influences semantic change, or rather influences polysemy, which in turn causes semantic change. in this section, we first give a short overview of relevant work on causality, before presenting how we apply these concepts to word change dynamics. ## slang and nonslang word selection we select 100 slang words and 100 nonslang words for our study, presented in appendix e. in the tradeoff between statistical significance and time spent on computation and data collection, we found that a set of 200 words was enough to get highly significant results. the slang words are randomly sampled from the online slang dictionary, 3 which provides well-maintained and curated slang word definitions as well as a list of 4,828 featured slang words as of june 2021. we limit the scope of our study to only encompass single-word expressions, and in so doing we filter out 2,169 multi-word expressions. to further clean the data, we also delete words with only one character and acronyms. lastly, we limit the causal analysis to words that are exclusively either slang or nonslang, excluding "hybrid" words with both slang and nonslang meanings, such as "kosher" or "tool." including words of this type would have interfered with the causal analysis by creating a hardcoded dependency between word type and polysemy, as these words by definition are polysemous. we do however perform a separate analysis of the hybrid words in appendix c. ## data collection we curate a twitter dataset from the years 2010 and 2020, which we select as our periods of reference, and collect the following variables: ## discussion we analyze the dynamics of frequency shift and semantic change in slang words, and compare them to those of nonslang words. our analysis shows that slang words change slower in semantic meaning, but adhere to more rapid frequency fluctuations, and are more likely to greatly decrease in frequency. our study is the first computational approach to confirm this property in slang words @xcite . to ensure that this is the result of a causal effect, and not mediated through another variable or subject to confounders, we model the data with a causal dag, by also considering the potential interacting variables polysemy, frequency and pos. we discover that there is no influence of confounders, nor are there mediators between a word's type and its semantic change or its frequency shift, which confirms a direct causal effect. this means that if we could intervene on a word's type, i.e., by setting it to be slang instead of nonslang or vice versa, we would expect its change dynamics to differ. our results are consistent with those of dubossarsky et al. @xcite , which found that neither the law relating semantic change to frequency, polysemy @xcite nor prototypicality @xcite were found to be as strong as previously thought after a case-control study using a scenario without semantic change. indeed, there is no directed path from polysemy or frequency to semantic change in our causal graph, but they are both influenced by word type. we leave for future research to explore whether other word categorizations, e.g., related to specific domains, languages or phonetic aspects, sustain this result. in addition, our analysis does not support the claim that pos could underlie semantic change @xcite . we note however that as our vocabulary contains 50% slang words, the results need not be consistent with results obtained with a word sample drawn from standard language. moreover, in the causal structure we discover that word polysemy has a direct effect on word frequency, which is in line with previous linguistic studies showing that a word's frequency grows in an s-shaped curve when it acquires new meanings @xcite , as well as a known positive correlation between polysemy and frequency @xcite . we emphasize that this relationship is not merely an artifact of contextualized word representations being affected by frequency @xcite , since our polysemy score does not rely on word representations as in @xcite . our approach is however not without drawbacks -the polysemy variable is collected from dictionaries, which may be subjective in their assignments of word senses. our study, along with previous work on the dynamics of semantic change, is limited by mainly considering distributional factors. linguists have suggested that sociocultural, psychological and political factors may drive word change dynamics @xcite , and slang words are not an exception. although challenging to measure, the influence of such factors on slang compared to nonslang words would be interesting to examine in future work. in conclusion, we believe that a causal analysis as we have presented here provides a useful tool to understand the underlying mechanisms of language. complementing the recent emergence of research combining causal inference and nlp @xcite , we have shown that tools from causality can also be beneficial for gaining new insights in diachronic linguistics. ## conclusion in this work, we have analyzed the diachronic mechanisms of slang language with a causal methodology. this allowed us to establish that a word's type has a direct effect on its semantic change and frequency shift, without mediating effects from other distributional factors. a appendix -fine-tuning with urban dictionary data a.1 preprocessing the full urban dictionary data contains 3, 534, 966 word definitions. in the dataset provided by @xcite , each entry contains a definition, examples in which the word occurs, number of upvotes & downvotes from website visitors, username of the submitter and a timestamp. as the data is crowd-sourced, many of these entries are noisy and of low quality. we therefore filter the lower quality definitions out before fine-tuning roberta. after performing data exploration, we came up with two criteria that we found the most indicative of a definition's quality: the number of upvotes it got, and its upvote/downvote ratio. the distribution of upvotes, downvotes and the upvote/downvote ratios in the dataset can be seen in figure 6 below. we also note that the number of submissions to urban dictionary is relatively well-spread, see figure 5 . this implies that we do not have a strong bias towards more recently popularized slang terms in the dataset, and that we do have representation of the entire time span of interest; 2010 -2020. we keep the entries having more than 20 upvotes and an upvote/downvote ratio of at least 2. this leaves us with 488, 010 urban dictionary entries, out of which we randomly sample 100, 000 to reduce the computation time in the fine-tuning process. we use both the definitions and the word usage examples for fine-tuning, producing a final dataset of 200, 000 sequences.