Word class flexibility: A deep contextualized approach (2020)

Abstract: Word class flexibility refers to the phenomenon whereby a single word form is used across different grammatical categories. Extensive work in linguistic typology has sought to characterize word class flexibility across languages, but quantifying this phenomenon accurately and at scale has been fraught with difficulties. We propose a principled methodology to explore regularity in word class flexibility. Our method builds on recent work in contextualized word embeddings to quantify semantic shift between word classes (e.g., noun-to-verb, verb-to-noun), and we apply this method to 37 languages. We find that contextualized embeddings not only capture human judgment of class variation within words in English, but also uncover shared tendencies in class flexibility across languages. Specifically, we find greater semantic variation when flexible lemmas are used in their dominant word class, supporting the view that word class flexibility is a directional process. Our work highlights the utility of deep contextualized models in linguistic typology.

## introduction in natural languages, lexical items can often be used in multiple word classes without overt changes in word form. for instance, the word buru in mundari can be used as a noun to denote 'mountain', or as a verb to denote 'to heap up' @xcite . known as word class flexibility, this phenomenon is considered one of the most challenging topics in linguistic typology @xcite . we present a computational methodology to quantify the regularity in word class flexibility across languages. there is an extensive literature on how languages vary in word class flexibility, either directly @xcite @xcite or through related notions such as word class conversion (with zeroderivation) @xcite @xcite @xcite . however, existing studies tend to rely on analyses of small sets of lexical items that may not be representative of word class flexibility in the broad lexicon. critically lacking are systematic analyses of word class flexibility across many languages, and existing typological studies have only focused on qualitative comparisons of word class systems. we take to our knowledge the first step towards computational quantification of word class flexibility in 37 languages, taken from the universal dependencies project @xcite . we focus on lexical items that can be used both as nouns and as verbs, i.e., noun-verb flexibility. this choice is motivated by the fact that the distinction between nouns and verbs is the most stable in word class systems across languages: if a language makes any distinction between word classes at all, it will likely be a distinction between nouns and verbs @xcite @xcite . however, our understanding of cross-linguistic regularity in noun-verb flexibility is impoverished. we operationalize word class flexibility as a property of lemmas. we define a lemma as flexible if some of its occurrences are tagged as nouns and others as verbs. flexible lemmas are sorted into noun dominant lemmas, which occur more frequently as nouns, and verb dominant lemmas that occur more frequently as verbs. our methodology builds on contextualized word embedding models (e.g., @xcite and bert @xcite ) to quantify semantic shift be-tween grammatical classes of a lemma, within a single language. this methodology can also help quantify metrics of flexibility in the lexicon across languages. we use our methodology to address one of the most fundamental questions in the study of word class flexibility: should this phenomenon be analyzed as a directional word-formation process similar to derivation, or as a form of underspecification? derived words are commonly argued to have a lower frequency of use and a narrower range in meaning compared to their base @xcite . if word class flexibility is a directional process, we should expect that flexible lemmas are subject to more semantic variation in their dominant word class than in their less frequent class. we also test the claim that noun-toverb flexibility involves more semantic shift than verb-to-noun flexibility. while previous work has explored these questions, it remains challenging to quantify semantic shift and semantic variation, particularly across different languages. we present a novel probing task that reveals the ability of deep contextualized models to capture semantic information across word classes. our utilization of deep contextual models predicts human judgment on the spectrum of noun-verb flexible usages including homonymy (unrelated senses), polysemy (different but related senses), and word class flexibility. we find that bert outperforms elmo and non-contextual word embeddings, and that the upper layers of bert capture the most semantic information, which resonates with existing probing studies @xcite . ## conclusion we use contextual language models to examine shared tendencies in word class flexibility across languages. we find that the majority class often exhibits more semantic variation than the minority class, supporting the view that word class flexibility is a directional process. we also find that in english, noun-to-verb flexibility is associated with more semantic shift than verb-to-noun flexibility, but this is not the case for most languages. our probing task reveals that the upper layers of bert contextual embeddings best reflect human judgment of semantic similarity. we obtain similar results in different datasets and language models in english that support the robustness of our method. this work demonstrates the utility of deep contextualized models in linguistic typology, especially for characterizing cross-linguistic semantic phenomena that are otherwise difficult to quantify.