Word Sense Disambiguation with Knowledge-Enhanced and Local Self-Attention-based Extractive Sense Comprehension (2022)

Abstract: Word sense disambiguation (WSD), identifying the most suitable meaning of ambiguous words in the given contexts according to a predefined sense inventory, is one of the most classical and challenging tasks in natural language processing. Benefiting from the powerful ability of deep neural networks, WSD has achieved a great advancement in recent years. Reformulating WSD as a text span extraction task is an effective approach, which accepts a sentence context of an ambiguous word together with all definitions of its candidate senses simultaneously, and requires to extract the text span corresponding with the right sense. However, the approach merely depends on a short definition to learn sense representation, which neglects abundant semantic knowledge from related senses and leads to data-inefficient learning and suboptimal WSD performance. To address the limitations, we propose a novel WSD method with Knowledge-Enhanced and Local Self-Attention-based Extractive Sense Comprehension (KELESC). Specifically, a knowledge-enhanced method is proposed to enrich semantic representation by incorporating additional examples and definitions of the related senses in WordNet. Then, in order to avoid the huge computing complexity induced by the additional information, a local self-attention mechanism is utilized to constrain attention to be local, which allows longer input texts without large-scale computing burdens. Extensive experimental results demonstrate that KELESC achieves better performance than baseline models on public benchmark datasets.

## introduction word sense disambiguation (wsd) is to identify a proper sense with an ambiguous word in a given context according to a predefined sense inventory, which is one of the most typical and challenging tasks in natural language processing (nlp) and play a critical role for human language understanding @xcite . for instance, the noun word plant conveys different senses in industrial plant and plant seeds. wsd has been able to determine accurate meanings of ambiguous words, which is beneficial to a variety of downstream nlp applications, such as machine translation, information extraction and retrieval @xcite . in recent years, with the rapid development of deep learning, the performance of wsd with neural networks-based methods has great improvement. the early neural networks-based models have cast wsd as a multi-label classification task, which disambiguated all polysemous words with a unified classier @xcite . however, these models have focused on modeling contexts containing ambiguous words from sense-labeled training data, which ignored the rich semantic knowledge in lexical resources, such as wordnet and babelnet @xcite , and resulted in their inability to outperform the traditional word expert supervised methods @xcite . due to the semantic knowledge in a lexical dictionary including sense definitions (glosses), examples, relations, etc., defined by professional lexicographers, which is essential and valuable for wsd, some works @xcite have attempted to integrate gloss information into neural wsd models in order to leverage the lexical knowledge. gas @xcite has incorporated glosses into wsd, which jointly encoded glosses and contexts, and captured their relations with a memory network. gloss-bert @xcite has utilized glosses in wordnet together with the annotated data to construct context-gloss pairs, reformulated wsd as a text matching task. bem @xcite ) has been a bi-encoder method that encodes the target word and candidate glosses inde-pendently and optimizes the encoders in the same representation space. although these works considered the gloss information in wordnet, they neglected to explore the knowledge contained in semantic relations such as hypernyms. therefore, ewiser @xcite has been proposed to enhance wsd by integrated synset embeddings and semantic relations including hypernyms and hyponyms. esr @xcite has further enhanced sense representations by incorporating synonyms, example sentences and sense glosses of hypernyms. although the methods mentioned above have achieved great successes, they have treated wsd as multi-label classification or text matching tasks, which focused on modeling the relations between a context and each specified candidate sense. none of them considers all candidate senses of an ambiguous word simultaneously, which is not consistent with human behaviors, as humans always justify the right sense by comparing all possible senses with the context. in order to simulate the cognitive process of human, esc @xcite has reformulated wsd as a text span extraction task, called extractive sense comprehension, which accepted a context of an ambiguous word together with all definitions of its candidate senses. although esc demonstrated the superiority over the competitive methods, it merely relied on a short definition (gloss) to represent a sense, which was insufficient to learn an ideal sense representation and inevitably hinder the improvement of wsd performance. to address the above-mentioned limitations, we propose a novel wsd method with knowledge-enhanced and local self-attention-based extractive sense comprehension (kelesc), inspired by esr @xcite . specifically, a knowledge-enhanced method is proposed to enrich semantic representation by incorporating additional examples and definitions of the related senses in wordnet. then, in order to avoid the huge computing complexity induced by the additional information, a local self-attention mechanism is utilized to constrain attentions to be local, which allows longer input texts without large-scale computing burdens @xcite . extensive experiments have been conducted to verify the effectiveness of the proposed model on public benchmark datasets. in summary, this paper makes the following contributions: the existing works on wsd can be categorized into three groups: knowledge-based, supervised and neural-based methods. ## methodology in this section, we first give the task definition. then, we detail our proposed model, knowledge-enhanced and local self-attentionbased extractive sense comprehension (kelesc) for wsd. ## conclusion and future work in this paper, we proposed a novel wsd method with knowledge-enhanced and local self-attentionbased extractive sense comprehension. specifically, a knowledge-enhanced method was devised to enrich semantic representation by incorporating additional examples and definitions of the related senses in wordnet. then, in order to avoid the huge computing complexity induced by the additional information, a local self-attention mechanism was utilized to constrain attentions to be local, which allowed longer input texts without large-scale computing burdens. extensive experimental results had demonstrated the effectiveness of the proposed model on public benchmark datasets. although our model achieved better performance, it still could be improved. currently, we utilized example sentence and hypernym gloss. there are many other unexplored semantic relations in wordnet and babelnet. we leave it as future work to explore more semantic relations in more lexical resources to further enhance wsd performance. besides, a detailed qualitative analysis on rare senses and frequent ones should be considered. we will attempt to evaluate the performance on