An Iterative Approach for Unsupervised Most Frequent Sense Detection using W ord N et and Word Embeddings (2018)

Abstract: Given a word, what is the most frequent sense in which it occurs in a given corpus? Most Frequent Sense (MFS) is a strong baseline for unsupervised word sense disambiguation. If we have large amounts of sense-annotated corpora, MFS can be trivially created. However, sense-annotated corpora are a rarity. In this paper, we propose a method which can compute MFS from raw corpora. Our approach iteratively exploits the semantic congruity among related words in corpus. Our method performs better compared to another similar work.

## introduction word sense disambiguation (wsd) remains to be one of the relatively hard problems in the field of natural language processing. machine learning approaches to wsd can be broadly classified into two categories: supervised and unsupervised. supervised techniques rely on learning patterns from sense-annotated training data. however, such data are hard to come by. semcor, one of the most common sense-annotated corpus in english language, contains around 700k tokens, 200k of which have been sense-annotated. it is really small as compared to raw corpora such as ukwac, where the number of tokens is close to 2 billion. on the other hand, unsupervised techniques do not require sense-annotated corpora. a strong baseline for unsupervised wsd is the most frequent sense (mfs) baseline. while performing sense disambiguation, the baseline completely ignores the context, and simply assigns the most frequent sense to the target word. inspite of ignoring context, which is one of the main source of information for performing sense disambiguation, the mfs baseline gives re-ally strong results. this is because of the inherent skew in the sense distribution of the data. computing mfs baseline is trivial, if one has access to large amounts of sense-annotated corpora. however, that is not the case as explained earlier. thus there is a need for uncovering mfs from raw data itself. word embeddings collectively refers to the set of language modelling and feature learning techniques, which maps words to real valued vectors @xcite @xcite @xcite @xcite . do note that most word embedding models only output one embedding per word, instead of the ideal case of outputting one embedding per sense of a word. though, some models do exist, which provide one embedding per sense of a word by inferring number of senses either through context clustering approaches @xcite , or by using sense inventory @xcite . for the rest of this paper, we mean one embedding per word models, when we use the phrase word embeddings. the field of natural language processing is increasingly seeing the use of word embeddings for various problems, and mfs is no exception. @xcite showed that pretrained word embeddings can be used to compute most frequent sense. in this paper, we propose an iterative approach for extracting most frequent sense of words in a raw corpus. the approach uses word embeddings as an input. thereby, in order to obtain mfs from some raw corpus, one need to apply the following two steps: the key points of this paper are: the rest of the paper is organized as follows: section 2 describes the related work. section 3 explains our approach. section 4.1 details our experimental setup and results. section 5 provides some error analysis, followed by conclusion and future work. ## approach our approach relies on the semantic congruity of raw text. consider the following example: consider the word cricket having two senses sport and insect, and the word bat having two senses sport instrument and reptile. then, if in our corpus, we already know that bat is in sport instrument sense for most cases, then in order for the corpus to be semantically congruent, the most frequent sense of cricket has to be sport. so, in order to find most frequent sense of all words in the vocabulary of the corpus, we start with the word whose sense is already known. so, the approach begins with monosemous words, for which mfs is trivial. next, it moves on to bisemous words, and uses the monosemous words sense information to detect most frequent sense. then it moves on to trisemous words, and use the hitherto resolved words for detecting most frequent sense, and so on. thus the approach iterates over the degree of polysemy, and uses the computed mfs of words with degree of polysemy 1 to n -1 to compute the mfs of words with degree of polysemy n. at any point of time, we call the words whose mfs is already established as tagged words. for a given word whose mfs is to be computed, we enumerate all senses, and then compute the vote for each senses by the top k nearest neighbors who are already tagged. the vote is a product of two measures: the cosine similarity (w i ) between the embedding of the current tagged word and the target word, and the wordnet similarity (s i ) between the mfs of current tagged word (which would have been established in the previous iteration), and the sense for which the vote is being computed. the votes are summed over, and the sense with the highest sum is considered to be the most frequent sense of the target word. the basic flow is illustrated in figure 1 . the major parameters in our approach are: our experiments found the average of normalized wu palmer and lin similarity performs slightly better than other combinations. ## discussion even though our approach performs better than @xcite , we are not able to cross semcor and wfs results. the following are the reasons for the same: ## conclusion in this paper, we proposed an iterative approach for unsupervised most frequent sense detection in raw corpus. the approach uses word embeddings. our results bears similar trends to those of @xcite , thereby strengthening the claim that word embeddings do indeed capture most frequent sense. through 2 different categories of experiments, we established that our method is better than theirs. since there are no language specific restrictions, we believe that our approach should be easily applicable to other languages. in the future, we would like to experimentally validate this claim.