Using Synchronic Definitions and Semantic Relations to Classify Semantic Change Types (2024)

Abstract: There is abundant evidence of the fact that the way words change their meaning can be classified in different types of change, highlighting the relationship between the old and new meanings (among which generalisation, specialisation and co-hyponymy transfer).In this paper, we present a way of detecting these types of change by constructing a model that leverages information both from synchronic lexical relations and definitions of word meanings. Specifically, we use synset definitions and hierarchy information from WordNet and test it on a digitized version of Blank’s (1997) dataset of semantic change types. Finally, we show how the sense relationships can improve models for both approximation of human judgments of semantic relatedness as well as binary Lexical Semantic Change Detection.

## introduction at any point in time, a word can have several meanings. often, these meanings share a certain degree of relationship with one another and the word is then said to display polysemy. diachronically, polysemy is a result of semantic change, the process in which a word change its meaning/s. when new meanings are derived, the relationship to existing meanings is determined by the type of change that takes place. examples include metaphorical extensions (the arm of the sea) and specialization of usage (sand that used to mean the shore as well as the grain of sand, and now only the latter). several taxonomies of semantic change have been proposed, from @xcite to @xcite , that describe both the type of change as well as the causes of it. however, the computational community, which has spent the past 15 years developing computational models for automatically detecting semantic change in corpora, has disregarded those taxonomies and processes proposed in the past @xcite . sparked by the invention of neural word embedding techniques, the focus has been on quantifying the degree of change, or determining when change has taken place, without the use of sense-inventories, dictionaries or knowledgebases. currently, the standard recipe for lexical semantic change detection (lsc) is form-based @xcite : each usage of a word is represented by its contextualized embedding, and the embeddings of different time periods are compared using cosine similarity. the comparison is done either by averaging pairwise similarities, or first averaging the embeddings of each time periods and then comparing the averages. when sensebased models are employed, (a) the embeddings are clustered to derive sense representations and then clusters are compared over time, and optionally (b) the clusters are labeled to ease interpretation. so far, no computational work has been done to detect the qualitative type of change that words experience. nor have these change types been utilised when building models for change detection. our work therefore presents a first approach to addressing both of these tasks. we rely directly on generated definitions and sidestep the standard recipe, i.e., to represent each usage of a word with its contextualized embedding. we consider a word's meaning, i.e., sense, as a latent construct and the definition as one way to capture that latent construct. the use of definitions to precisely describe the meaning of a word offers an ideal level of sense representation, that combines both precision and abstraction, as well as a way to summarize multiple usages. thus, the use of definitions serves as a dimension reduction technique, projecting n×m contextualized embeddings in two time periods to a significantly smaller set of n×m definitions. by using a single computational step rather than two (steps (a) and (b) above), we reduce the complexity and decrease the uncertainty. to classify semantic change types, we train a classifier on a synchronic sense repository with def-initions and semantic relations, namely wordnet. the rationale behind this choice is that each of the type of semantic change targeted in this paper corresponds to a synchronic relationship between senses. this distant training procedure via synchronic information is necessary as there are no available resources to date for change type classification that surpass a few hundred examples. next, we test this classifier on an unseen dataset of historical semantic changes. we therefore extend the dataset provided by @xcite , in which for a given word and a pair of senses there is an associated type of semantic change, with concise definitions for each sense. finally, we incorporate change type classification in a state-of-the art model for detection of semantic change, and show an improvement in the model performance. contributions: in this paper, we show that 1. definitions of word senses can be used to detect semantic change type; 2. we can classify the type of semantic change by training on synchronic sense relationships using sense definitions; and that 3. the use of type information can improve models for both graded word-in-context (wic) as well as semantic change detection. ## lsc-ctd benchmark one of the most comprehensive and recent classification of semantic change is blank's taxonomy @xcite . in this work, about 650 cases of semantic change are classified covering the vocabulary of the romance languages and to some extent also german and english. blank tries to reinterpret traditional types of classification from a cognitivelinguistics point of view. for the lsc cause-type-definitions (lsc-ctd) benchmark foot_3 we digitize the cases reported by blank and create a benchmark of definitions manually curated by an historical linguist assisted by chatgpt foot_4 . the format of the dataset, the dataset statistics and the annotation procedure are described further in appendix a. blank's type classification include specialization, generalization, co-hyponymous transfer, autoantonym, metaphor, antiphrasis, metonymy, autoconverse, ellipsis, folk-etymology, analogy, meaning dilution, meaning reinforcement, and doubtful cases. for our study, we use the following types of change for which we can find a counterpart in the synchronic lexical database. foot_5 generalization: the old meaning is a sub-case of the new meaning (or: the new meaning has at least one less definitional feature than the old meaning); other terms used to refer to this type of change are broadening and widening. an example is the (non-attested) vulgar latin *adripare, which originally meant 'to reach the shore (by vessel)' but in contemporary romance languages has generalised to 'arrive (by any means of transport)' (e.g.: french arriver, italian arrivare) specialization: the new meaning is a sub-case of the old meaning (or: the new meaning has at least one extra definitional feature than the old meaning); other terms used to refer to this type of change are narrowing and restriction. an example is latin necare, which used to refer to 'to kill (by any means)' but whose french derivation noyer is now restricted to 'to kill by drowning' co-hyponymous transfer: the change that occurs due to naming confusion, that is, when a word is used for new referents that are (wrongly) thought to be similar if not identical to the old referents. for instance, in some romance languages the word for 'rat' is also used (in a derived form) to refer to 'mouse' or vice versa (piedmontese: rat for 'rat' and rató for 'mouse'; spanish: ratón for 'rat' and rata for 'mouse') the relations of hyperonymy, hyponymy, cohyponymy and antonymy can all be considered the synchronic counterpart of the semantic change types introduced in section 3. below we illustrate how we map the taxonomical relationship between the sense glosses of synsets in wordnet to each considered change type. change types in blank's taxonomy that cannot be modelled through the wordnet hierarchy are disregarded for this work. for hierarchical relationships of hyponymy, hyperonymy, and co-hyponymy, we consider paths of maximum length 1, meaning pairs for which there is a direct relationship. generalization and specialization: for every synset, we extract the list of associated hyperonymous synsets, and form pairs of a synset sense gloss with its hyperonym sense gloss. viewed from the perspective of the child synset to the parent synset, we obtain instances of generalization. similarly, by exploiting the inverse relationship (from parent to child synset), we obtain examples of specialization. co-hyponymy: we define a co-hyponymy relationship as two separate synset nodes with a common synset parent, in which the two child synsets represent two different and more specialized cases of the same general concept. by pairing the sense glosses of the child synsets, we generate the same sense relationship that lies at the basis of the diachronic co-hyponymous transfer. antonymy: wordnet also encodes the relationship between synsets that express opposite concepts, often between adjectives, but also found between other word classes such as nouns (father vs. mother) and verbs (to buy vs. to sell). for each synset, we therefore extract the synset, or the lemma within the synset, labeled as antonym, and pair the sense glosses of these two synsets. homonymy: finally, we randomly generate pairs of synsets with the same part-of-speech that do not lead to hyperonymous/hyponymous, cohyponynic and antonymic pairings of sense glosses. the potential unrelatedness between the senses should be a good proxy for homonymy, defined as the occurrence of completely unrelated senses (for the same word form). while it is possible that related senses of the same word are paired in the random pairing, the risk is extremely small (i.e., on average 3-4 out of the 30-40,000 randomly sampled synsets can belong to the same word). homonymy is not treated on the same par as the other types of semantic change by @xcite , even though its importance as a driver of language change is uncontroversial. for this reason it has been included in the experiments. ## model to classify the relationship between senses, we utilize the data collected in section 4 and train a classification model foot_6 that takes as input the two definitions of the respective senses and outputs the corresponding class, i.e. {homonymy, hyperonymy, hyponymy, antonymy}. specifically, we concatenate the textual definitions δ 1 and δ 2 together with special tokens into a single string δ = <@xmath0, which is then tokenized and encoded using roberta-large @xcite . from this, we obtain the vectors corresponding to the sub-tokens of the original string v 1 , .., v k . the vector v 1 , corresponding to the initial token, is used for classification. we apply a linear transformation to reduce the vector's dimension to the label space, such that v * 1 = w v 1 , where @xmath1. finally, we use cross-entropy to optimize the model. to counteract class imbalance, examples in each batch are drawn proportionally from each class. we use adamw for optimization with a weight decay of 0.1 and a learning rate of 1e-6. early stopping is employed to determine when to halt training based on accuracy on the development set. additionally, we assess the outcomes using two baselines. the results for these baselines are detailed in appendix b. ## evaluation of the type classification we first evaluate the model on the test portion of the synchronic relations found in wordnet. next, we test the model on a historical, novel data, namely the lsc-ctd benchmark. ## evaluation on semantic change we will use our classification of semantic change type, introduced in section 5, to test if we can improve on state-of-the-art lsc detection. in this section, we use flan-t5 xl ft presented by @xcite to generate definitions, which is a flan-t5 xl (chung et al., 2022) model finetuned on wordnet @xcite , the oxford dataset @xcite and the codwoe dataset @xcite . ## conclusion with this paper, we present a first computational approach to detecting the type of semantic change that a word has experienced. we release the lsc-ctd benchmark, a first dataset for computational modeling of semantic change that is annotated by type of change. the benchmark is based on a digitized version of blank's taxonomy extended with concise sense definitions. because the resource is limited in size, it can only be used for testing purposes. to train a model for classifying change types, we used wordnet with synchronic sense definitions and relations between the synsets. the use of wordnet as a foundational resource for training our classifier enabled us to leverage a vast repository of semantic relationships and meanings. our evaluation is grounded both in linguistic theory and computational metrics. we prove that homonyms are perceived by human annotators as the most distant semantic difference class in blank's continuum of semantic proximity. we applied our classifier to extend semantic change detection models with type information and tested it on a standard evaluation benchmark, namely semeval-2020 task 1. we aimed to provide a more refined and accurate model for semantic change detection. we found that by incorporating change types, in particular the notion of homonymy, we improved the state-of-the-art results for lsc on the graded task. for the binary task, we also saw encouraging results. our findings underscore the importance of distinguishing between different types of semantic changes, such as generalization, specialization, and co-hyponymous transfer, in understanding the dynamics of semantic change. in future work, we will extend our resources with more examples of change types, as well as additional synchronic resources.