Disambiguate Words like Composing Them: A Morphology-Informed Approach to Enhance C hinese Word Sense Disambiguation (2024)

Abstract: In parataxis languages like Chinese, word meanings are highly correlated with morphological knowledge, which can help to disambiguate word senses. However, in-depth exploration of morphological knowledge in previous word sense disambiguation (WSD) methods is still lacking due to the absence of publicly available resources. In this paper, we are motivated to enhance Chinese WSD with full morphological knowledge, including both word-formations and morphemes. We first construct the largest and releasable Chinese WSD resources, including the lexico-semantic inventories MorInv and WrdInv, a Chinese WSD dataset MiCLS, and an out-of-volcabulary (OOV) test set. Then, we propose a model, MorBERT, to fully leverage this morphology-informed knowledge for Chinese WSD and achieve a SOTA F1 of 92.18% in the task. Finally, we demonstrated the model’s robustness in low-resource settings and generalizability to OOV senses. These resources and methods may bring new insights into and solutions for various downstream tasks in both computational and humanistic fields.

## introduction word sense disambiguation (wsd) aims to identify the sense of a polysemous word in a specific context. it has become critical for accurate language understanding and has proven effective in various downstream tasks, including information extraction @xcite , test summarization @xcite and machine translation @xcite , etc. previously, mainstream methods typically regarded the target word as an indecomposable lexico-semantic unit, utilizing various interword and word definition information @xcite , to improve the model's performance. these methods achieved remarkable results, surpassing the f1 score of 80% on standardized english datasets @xcite , which is the estimated human performance @xcite . recently, large language models (llms) have demonstrated remarkable performance across various tasks, but they still struggle to make improvements on wsd. even chatgpt, one of the top-performing models, only achieves an f1 score of 73.30% @xcite . however, when applied to chinese, these top-performing models (without morphological knowledge) fail to reach an f1 value of 80% @xcite . this decrease in accuracy may be attributed to the characteristics of the chinese language. in a parataxis language like chinese, word meanings are highly correlated with morphological knowledge, which includes word-formations and morphemes as two aspects @xcite . wordformations designate how characters (as morphemes) are combined to compose lexical semantics by specific patterns of morphemes. while word-formation has been proven to be effective in chinese wsd @xcite , the role of morphemes may be equally significant for the task. in chinese, morphemes are defined as the smallest semantic and sound-bearing unit @xcite and play a fundamental role in the process of word formation with their characteristics @xcite . despite the vast and dynamic set of chinese would-be words, morphemes remain relatively stable in number and meaning @xcite ). an overwhelming majority of them maintain their meanings in the process of word formation @xcite , and as a result, the composition of morpheme senses can largely represent the meaning of words @xcite . combining morphemes under word-formations can yield even greater efficacy. as shown in figure 1 , the polysemous word "满月" holds two senses: "满 月 1 " and "满 月 2 ", which are composed of different morphemes ('满 3 月 2 " vs. "满 4 月 1 ') under different word-formation rules (verb-object vs. modifier-head). such semantic compositionality of morphemes, with intuitive clarity, can therefore enhance wsd and improve its performance. what's more, this motivated strategy may bring about further advantages that largely benefit outof-vocabulary (oov) issues previously entangled with wsd. figure 1 : the word formation process of the polysemous word "满月", with word-formation and morpheme senses specified, can denote the word's meaning. in recent years, researchers have explored the application of morphological knowledge. zheng et al. (2021b) leveraged word-formation in chinese wsd and achieved a high f1 of 87.62% on ficls. this is largely attributed to the introduction of word-formation knowledge. while the importance of word-formations in chinese wsd has been explored and yielded notable improvements, the introduction and application of morphemes are still neglected due to the absence of publicly available resources and annotations. with morphologyinformed resources and models, the potential of full morphological knowledge can be stimulated, which may further enhance chinese wsd and improve its performance. this specific strategy may bring about further advantages, which can largely benefit oov sense representations and disambiguation. in real chinese corpora, the oov issues are quite common, and even more popular than in other languages foot_1 . for example, the word "千万(ten million/be sure to)" may mean "ten million" in texts, representing a direct combination of the morphemes "千 1 (thousand)" and "万 1 (ten thousand)", but it is unambiguous in the contemporary chinese dictionary (ccd) foot_2 with only one sense "be sure to". these "unambiguous" words also need disambiguation and can therefore enhance wsd by extending the coverage and accuracy of word sense representation @xcite . currently, existing chinese wsd datasets cannot cover or handle such cases. this urges for a new kind of wsd resources that may underline and address these problems. facing these scenarios of chinese and inspired by the process of word formation, in this paper, following the previous works @xcite , we are motivated to further leverage morpheme features and fully explore the potential of morphological knowledge to enhance wsd. we first construct two lexico-semantic inventories morinv and wrdinv, a morphology-informed chinese lexical sample dataset(micls), and an oov test set. then, we propose morbert to explicitly incorporate chinese morphological knowledge into a bert-based wsd model. experimental results show that our method brings consistent and substantial performance improvements to chinese wsd with high accuracy in morpheme prediction. analysis shows that our method can be greatly generalized to low-resource settings and oov senses. these resources and methods may bring new insights into and solutions for various downstream tasks in both computational and humanistic fields. in summary, this paper is committed to fully leveraging morphological knowledge to enhance chinese wsd. the main contributions are as follows: chinese wsd resources: compared to english, resources for chinese wsd are still relatively lacking. semeval-2007 task 5 @xcite and chinese word sense tagging corpus (stc) @xcite used the corpora of people's daily foot_3 (pd, in months of @xcite and annotated senses for 40 and 966 polysemous words, respectively. they are currently the most commonly used chinese wsd datasets, but are limited to low coverage, and the corpora lack timeliness. word sense annotated corpus for teaching chinese as second language @xcite and ficls dataset @xcite annotated senses for 1181 polysemous words in chinese textbooks and 7064 polysemous words in chinese wikipedia, respectively, with large scale and corpora exhibiting strong timeliness. however, both datasets are annotated based on ccd and are therefore not publicly available due to copyright issues. modern chinese word sense annotated dataset @xcite annotated senses for 1083 polysemous words in 19082 sentences collected from different sources. it is now publicly available, but the size is still relatively small and without providing morphological knowledge. due to restrictions on intellectual property, chinese wsd still faces the challenges of obtaining large-scale, high-quality datasets. it is necessary to construct resources for public use (and adaption to computing applications) through methods such as paraphrasing to facilitate sharing with both the academic and industrial communities. chinese morphology-related resources: existing chinese morphology-related resources fo-cus on morpheme definition and morpheme-based word-formation features. for morpheme inventory, @xcite introduced a morpheme knowledge-base by manually describing 17,470 morphemes for 6,763 characters used for wordformation analysis. @xcite further connected such morphemes to cilin @xcite , an influential chinese thesaurus. lin and liu (2019) described morphemes with part-ofspeech (pos) and inter-morpheme relations, covering a comprehensive set of 20,855 morphemes for 8,515 characters originating from ccd. based on it, for morphology-informed datasets, zheng et al. (2021a) manually annotated word-formations and morphemes for 45,311 words for definition generation (dg). these resources can serve as the basis for the construction of our resources for public use. ## resources current chinese wsd resources face challenges in achieving high coverage and restrictions on public use due to intellectual property.in light of these, we aim to construct large-scale, releasable resources for wsd, which can also benefit handling the oov problems. in this section, we first introduce morpheme inventory (morinv) and word inventory (worinv) as the lexico-semantical basis. then, we construct the micls dataset for wsd. additionally, we provide a small oov test set for testing chances of generalizability to oov senses. ## analysis 6.1 analysis on the high f1 score the f1 score of 92.18 is significantly higher than that of top-performing wsd models on other datasets. we believe that the relatively high f1 of the models on micls dataset value may be related to the following factors: ## conclusion in this paper, we propose to enhance chinese wsd with full morphological knowledge, including word-formation and morphemes. we first construct large-scale and releasable wsd resources, including lexico-semantic inventories morinv and wrdinv, chinese wsd dataset micls, and oov test set. then, following the previous formbert, we propose morbert to fully leverage morphological knowledge and achieve a sota f1 of 92.18% on micls dataset. analysis reveals that our model is characteristic of robustness in low-resource settings and generalizability to oov senses. this may bring new insights into and solutions for various downstream tasks in both computational and humanistic fields. in the near future, we plan to continuously enlarge the wsd datasets, expand them to other similar languages for further improvements in wsd, and apply them to the aforementioned downstream tasks, thereby contributing to a deeper understanding of languages.