Word sense extension (2023)

Abstract: Humans often make creative use of words to expressnovel senses. A long-standing effort in natural language processing hasbeen focusing on word sense disambiguation (WSD), but little has been explored about how the sense inventory of a word may be extended toward novel meanings. We present a paradigm of word sense extension (WSE) thatenables words to spawn new senses toward novel context. We develop a framework that simulates novel word sense extension by first partitioning a polysemous word type into two pseudo-tokens that mark its different senses, and then inferring whether the meaning of a pseudo-token can be extended to convey the sense denoted by the token partitioned from the same word type. Our framework combines cognitivemodels of chaining with a learning scheme that transforms a language model embedding space to supportvarious types of word sense extension. We evaluate our frameworkagainst several competitive baselines and show that it is superior in predicting plausible novel senses for over 7,500 English words. Furthermore, we show that our WSE framework improves performance over a range of transformer-based WSD models in predicting rare word senses with few or zero mentions in the training data.

## introduction humans make creative reuse of words to express novel senses. for example, the english verb arrive extended from its original sense "to come to locations (e.g., to arrive at the gate)" toward new senses such as "to come to an event (e.g., to arrive at a concert)" and "to achieve a goal or cognitive state (e.g., to arrive at a conclusion)" (see figure 1 ). the extension of word meaning toward new context may draw on different cognitive processes such as metonymy and metaphor, and here we develop a general framework that infers how words extend to plausible new senses. ## computational framework our framework of word sense extension involves three interrelated components: 1) a procedure for partitioning polysemous words in the lexicon into new pseudo-tokens that signify their different senses; 2) a probabilistic, chaining-based formulation of word sense extension for lexical choice making under novel linguistic context; and 3) a learning algorithm for a transformed semantic space to learn flexible extensions of word senses. ## conclusion we have presented a framework for word sense extension that supports lexical items to extend to new senses in novel context. our results show that chaining provides a general mechanism for automated novel sense extension in natural context, and learning a transformed sense-extensional space enables systematic generalization to a certain degree. we also show that word sense extension improves the performance of transformer-based wsd models particularly on rare word senses. future work may extend our framework in several ways, such as how to better model systematic word sense extension, and do so over time and in different languages.