Detecting Contact-Induced Semantic Shifts: W hat Can Embedding-Based Methods Do in Practice? (2021)

Abstract: This study investigates the applicability of semantic change detection methods in descriptively oriented linguistic research. It specifically focuses on contact-induced semantic shifts in Quebec English. We contrast synchronic data from different regions in order to identify the meanings that are specific to Quebec and potentially related to language contact. Type-level embeddings are used to detect new semantic shifts, and token-level embeddings to isolate regionally specific occurrences. We introduce a new 80-item test set and conduct both quantitative and qualitative evaluations. We demonstrate that diachronic word embedding methods can be applied to contact-induced semantic shifts observed in synchrony, obtaining results comparable to the state of the art on similar tasks in diachrony. However, we show that encouraging evaluation results do not translate to practical value in detecting new semantic shifts. Finally, our application of token-level embeddings accelerates manual data exploration and provides an efficient way of scaling up sociolinguistic analyses.

## introduction a growing body of work has investigated semantic change detection, mainly relying on word embedding methods to model differences in meaning over time. this has been presented as a promising new approach for descriptive linguistics, providing the means to test theoretical hypotheses and discover new cases of semantic change on a large scale @xcite . however, it has also been pointed out that existing work has largely focused on generic research questions and datasets, using them as a training ground for proof-of-concept studies @xcite . as a result, many reported instances of semantic change are either long established (e.g. gay shifting from 'happy' to 'homosexual') or are of otherwise limited importance (e.g. proper names reflecting sociohistorical context). but how useful are semantic change detection methods when applied to a precisely defined descriptive issue? we use them to investigate contactinduced semantic shifts, focusing specifically on quebec english. semantic influence of french is documented in the sociolinguistic literature on quebec (e.g. deception 'disappointment', cf. fr. déception), but mostly from a qualitative standpoint, with the importance of the phenomenon still under debate. this reflects known methodological challenges in studying lexical variation, which is precisely what our approach aims to overcome. we follow common sociolinguistic practice and use synchronic data, contrasting quebec with predominantly english-speaking canadian regions. drawing on a large twitter corpus, we implement a two-pronged approach: we use type-level embeddings to detect new semantic shifts, and token-level embeddings to isolate regionally specific occurrences of target lexical items. we conduct standard quantitative evaluation and extensive qualitative analyses, focusing on the descriptive relevance of the results and challenges arising from empirically occurring data. we provide the following contributions: this paper is organized as follows. we first provide a brief overview of related work ( § 2) and introduce the corpus and the test set ( § 3). we then present two experiments, the first using type-level embeddings to detect new cases of semantic change ( § 4), and the second using token-level embeddings to investigate the individual senses of target lexical items ( § 5). we finally outline key conclusions and directions for future work ( § 6). ## related work semantic change detection methods developed in recent years have often relied on vector space models @xcite , where meaning is represented as a vector reflecting the target word's cooccurrence patterns. the general approach consists in training models specific to different time periods and then quantifying the distance between a word's vectors in different models. the same principle can be applied to semantic differences across regions, text types, and so on. specific implementations include creating separate count-based models while maintaining the same linguistic contexts across time periods @xcite or training neural word embeddings in a way that that captures meaning change while ensuring comparability between vector spaces @xcite @xcite . semantic change is usually quantified using the cosine distance between a word's vectors in different models, but alternative measures have also been proposed @xcite . more recently, pretrained neural networks have been used to obtain contextual embeddings and detect change by computing average representations for different time periods or measuring differences in clustering patterns @xcite . similar work has been done on synchronic data, but it is considerably more limited. it has focused on issues such as semantic variation across online communities of practice @xcite ), text types @xcite , large dialect regions @xcite , and different languages @xcite . evaluation of these methods is challenging and is often limited to a qualitative analysis of a restricted number of examples. quantitative evaluations have involved background information from lexicographic sources @xcite or synthetic corpora @xcite . while their focus is on the top semantic change candidates output by a model, an alternative approach consists in using manually annotated datasets containing (usually several dozen) words whose meaning is either stable or subject to change @xcite @xcite @xcite . the words are associated with binary labels or semantic change scores, and the models are evaluated on a ranking or binary classification task. descriptively oriented studies have used these methods to test theoretical hypotheses, as in the evaluation of two competing laws of semantic change by @xcite , or to facilitate exploratory analyses by domain experts. for instance, @xcite used vector space models for ancient greek to detect semantic shift candidates and then analyzed their nearest neighbors to establish meaning change. de pascale (2019) studied regional lexical variation in dutch using token-level models to identify cases where competing words were used with equivalent meanings. computational work on sociolinguistic issues uses social media posts or other large corpora to detect lexical innovations and dialect regions @xcite @xcite or to examine the impact of sociodemographic factors @xcite @xcite . language contact has been addressed in terms of codeswitching @xcite , language choice @xcite , and semantic integration of loanwords @xcite @xcite . related work on canadian english is limited to corpus construction @xcite and descriptive analyses of online communication @xcite . we are unaware of any computational studies of regional variation in canadian english, or of contact-induced semantic shifts in inherited (i.e. non-borrowed) lexical items. ## data for contact-induced semantic shifts in quebec english the presence of contact-induced semantic shifts in quebec english has been reported in a series of sociolinguistic surveys @xcite @xcite and manual corpus analyses @xcite @xcite . they have described around 60 english words at-tested in quebec with meanings typical of formally or semantically similar french words. this phenomenon involves the introduction of a new sense into a word's polysemic structure through innovative meaning change @xcite driven by semantic interference in bilinguals (cf. @xcite . while this process can be observed over time, we adopt a synchronic perspective. our assumption is that contact-induced semantic shifts correspond to words whose sense distributions are different in quebec, a majority french-speaking province, compared to predominantly english-speaking canadian regions. this is a typical comparative sociolinguistic approach @xcite : we contrast speech communities to identify linguistic behaviors which mirror the differences in their composition. in this section, we introduce the data needed to implement our approach: a large corpus of tweets allowing us to analyze different canadian regions, and a new test set used to tune our models and validate their performance. ## detecting semantic shift candidates we first examine contact-induced semantic shifts from an exploratory perspective. our aim is to determine which words, within the entire vocabulary, have the most different overall meanings in montreal compared to toronto and vancouver. this section introduces the experimental setup, presents the results of an evaluation used to find the best performing model, and examines the results it obtains on the detection of semantic shift candidates. ## characterizing semantic shifts in context a key limitation of the approach we introduced in the previous section is that it is sense-agnostic: all contexts in which a word appears are grouped into a single vector. although it can capture meaningful general trends, it is limited in accounting for rare phenomena, which may be troublesome when analyzing language change. this process is not abrupt; rather, it involves a period of coexistence of a new linguistic behavior with the old one @xcite , which for semantic change implies gradual modifications to the polysemic structure of the word @xcite . we therefore expect contact-related senses to appear in a limited number of occurrences per word. we further expect them to be attested in similar immediate contexts, but to be dispersed throughout the corpus. these assumptions, coupled with the previously outlined issues with type-level analyses, suggest that manual inspection of corpus data remains necessary to reliably identify semantic shifts. in order to facilitate this process, we implement a system which allows us to automatically group semantically similar occurrences of a target word and identify the uses that are specific to montreal. this enables us to manually analyze batches of occurrences all at once, rather than examining them one at a time, which streamlines both the discovery of contact-related uses and the exclusion of false positives. in the remainder of this section, we present the experimental setup, a sample analysis of clustered data, and the overall patterns for 40 semantic shifts that were manually annotated using this approach. ## conclusion in this study, we applied embedding-based semantic change detection methods to a new set of experimental conditions. focusing on the descriptive relevance of the results, we developed an 80-item test set for english-french language contact, and manually annotated corpus data for 40 lexical items. our results demonstrate that current type-level methods are of limited practical use in detecting new semantic shifts: despite robust performance on standard quantitative evaluation, they obtain very low precision on the discovery task. we further implemented a token-level analysis of the individual senses with which target items are used. this accelerated manual data exploration, leading to a detailed account of the issues impacting type-level methods. these results more generally represent the first quantitative corpus-based analysis of contact-induced semantic shifts in quebec english. our work also has more general implications, which reaffirm the central role of evaluation practices and corpora in advancing computational analyses of semantic change @xcite . the comparison of evaluations on the test set and on the discovery task underscored the stark difference between the two approaches. this should be taken into account when choosing evaluation methods, especially where the aim is to establish practical usability. and while some reported issues are specific to our corpus, similar problems may affect other semantic change studies, as noisy datasets and complex sense distributions are not unique to our work. finally, the comparison of type-level and token-level analyses highlighted diverging trends in the data which indicate that semantic shifts involve multiple dimensions of varia-