Try to Substitute: An Unsupervised C hinese Word Sense Disambiguation Method Based on H ow N et (2020)

Abstract: Word sense disambiguation (WSD) is a fundamental natural language processing task. Unsupervised knowledge-based WSD only relies on a lexical knowledge base as the sense inventory and has wider practical use than supervised WSD that requires a mass of sense-annotated data. HowNet is the most widely used lexical knowledge base in Chinese WSD. Because of its uniqueness, however, most of existing unsupervised WSD methods cannot work for HowNet-based WSD, and the tailor-made methods have not obtained satisfying results. In this paper, we propose a new unsupervised method for HowNet-based Chinese WSD, which exploits the masked language model task of pre-trained language models. In experiments, considering existing evaluation dataset is small and out-of-date, we build a new and larger HowNet-based WSD dataset. Experimental results demonstrate that our model achieves significantly better performance than all the baseline methods. All the code and data of this paper are available at https://github.com/thunlp/SememeWSD .

## introduction word sense disambiguation (wsd) is a long-standing natural language processing task which aims to identify the correct sense of a polysemous word in the context @xcite . wsd is fundamental to natural language understanding @xcite and has been proven to be beneficial to many other tasks such as machine translation @xcite , information extraction @xcite and information retrieval @xcite . there are two main kinds of wsd, namely supervised disambiguation and unsupervised knowledgebased disambiguation. supervised wsd requires large amounts of sense-annotated training corpora that are difficult to obtain @xcite . in contrast, unsupervised knowledge-based wsd relies on only an external lexical knowledge base (lkb) as the sense inventory and thus has wider practical use. existing unsupervised knowledge-based wsd approaches mainly comprise gloss-based and graphbased methods. the gloss-based methods utilize glosses (sense definitions) to conduct disambiguation. lesk algorithm @xcite ) is a seminal gloss-based method which disambiguates a word by selecting the sense whose gloss overlaps most with the context. there are many subsequent methods based on lesk algorithm @xcite @xcite . the graph-based methods is the other major type of knowledge-based wsd approaches, which exploit the structures of the lkb for disambiguation @xcite @xcite . besides, a recent method uses both glosses and structural information of lkbs in knowledge-based wsd and achieves state-of-the-art performance @xcite . in chinese wsd, hownet @xcite is the most widely used lkb @xcite . different from other lkbs, hownet contains neither glosses nor structures of different senses. instead, hownet defines a sense by a set of predefined sememes, the minimum semantic units in linguistics @xcite . therefore, both gloss-based and graph-based methods cannot work in hownet-based wsd. to the best of our knowledge, only a few studies focus on unsupervised wsd based on hownet. @xcite propose a representative statistical method which utilizes the co-occurrence of sememes of the target word and context to conduct disambiguation. @xcite learn sememe and sense embeddings and disambiguates a word by choosing the sense that has the closest embedding similarity with the context. these methods work well but far from perfectly. in this paper, we propose a new unsupervised hownet-based wsd model with the help of large pretrained language models. previous studies have shown that pre-trained language models such as bert @xcite incorporate much sense information @xcite , which can be utilized in wsd. their pre-training task of masked language model (mlm) is supposed to predict appropriate words for a specified position in the context. in other words, a word with higher mlm prediction score is more suitable for the given context and should have more similar meaning to the original word. based on this assumption, we design our lexical substitution-based wsd model. for each sense of the target polysemous word, we can find a set of substitution words that involve a sense annotated with the same sememes as the target sense. we calculate the mlm prediction score for each substitution word, and the average of prediction scores of a sense's all substitution words can reflect the probability that the target word conveys this sense in the context. the idea of lexical substitution has been applied to wsd in @xcite . different from our model, it uses a statistical language model to calculate substitution word scores and more importantly, it is not fully unsupervised and requires some sense-annotated corpora in the wsd procedure. besides, our model resembles the end-to-end bert-based lexical substitution model in @xcite . however, it is not aimed at wsd and has different calculation methods of substitution word score from us. in experiments, considering existing hownet-based wsd dataset is unavailable and based on an outdated version of hownet, we build a new and larger hownet-based wsd dataset for evaluation. experimental results demonstrate that our model significantly outperforms all the baseline methods and achieves state-of-the-art performance. ## methodology in this section, we elaborate on our hownet-based unsupervised wsd model. before description of the model, we first give a brief introduction to hownet. ## experiments in this section, we evaluate our model on the newly built hownet-based wsd dataset by us. ## conclusion and future work in this paper, we propose an unsupervised hownet-based word sense disambiguation method, which exploits the masked language model task of large pre-trained language models. in addition, we build a new and larger hownet-based word sense disambiguation dataset. in evaluation, experimental results demonstrate that our model achieves obviously better performance than all the baseline methods. in the future, we will try to extend our model to other unsupervised knowledge-based word sense disambiguation tasks, e.g., wordnet-based english word sense disambiguation. besides, we will consider making explorations into supervised word sense disambiguation based on our method.