Detecting Most Frequent Sense using Word Embeddings and B abel N et (2016)

Abstract: Since the inception of the SENSEVAL evaluation exercises there has been a great deal of recent research into Word Sense Disambiguation (WSD). Over the years, various supervised, unsupervised and knowledge based WSD systems have been proposed. Beating the first sense heuristics is a challenging task for these systems. In this paper, we present our work on Most Frequent Sense (MFS) detection using Word Embeddings and BabelNet features. The semantic features from BabelNet viz., synsets, gloss, relations, etc. are used for generating sense embeddings. We compare word embedding of a word with its sense embeddings to obtain the MFS with the highest similarity. The MFS is detected for six languages viz., English, Spanish, Russian, German, French and Italian. However, this approach can be applied to any language provided that word embeddings are available for that language.

## introduction word sense disambiguation or wsd refers to the task of computationally identifying the sense of a word in a given context. it is one of the oldest and toughest problems in the area of natural language processing (nlp). wsd is considered to be an aicomplete problem @xcite i.e., it is one of the hardest problems in the field of artificial intelligence. various approaches for word sense disambiguation have been explored in recent years. two of the widely used approaches for wsd aredisambiguation using the annotated training data called as supervised wsd and disambiguation without the annotated training data called as unsupervised wsd. mfs is considered to be a very powerful heuristics for word sense disambiguation. even with sophisticated methods, it is difficult to outperform its baseline. the mfs baseline for english language is created with the help of a sense annotated corpus wherein the frequencies of individual senses are learnt. it is found that, only 5 out of 26 wsd systems submitted to senseval-3, were able to beat this baseline. the success of the mfs baseline is mainly due to the frequency distribution of senses, with the shape of the sense rank versus frequency graph being a zipfian curve. unsupervised approaches were found very difficult to beat the mfs baseline, while supervised approaches generally perform better than the mfs baseline. in our paper, we have extended the work done by @xcite . they used word embeddings along with features from wordnet for the detection of mfs. we used word embeddings and features from babelnet for detecting mfs. our approach works for all part-of-speech (pos) categories and is currently implemented for six different languages viz., english, spanish, russian, german, french and italian. this approach can be easily extended to other languages if word embeddings for the specific language are available. the paper is organized as follows: section 2 briefs the related work. section 3 explains babelnet. our approach is given in section 4. experiments are presented in section 5 followed by conclusion. ## babelnet babelnet @xcite is a multilingual encyclopedic dictionary, with lexicographic and encyclopedic coverage of terms, and a semantic network. it connects concepts and named entities in a very large network of semantic relations, made up of more than 13 million entries, called babel synsets. each babel synset represents a given meaning and contains all the synonyms which express that meaning in a range of different languages. babelnet v3.0 covers 271 languages and is obtained from the automatic integration of: babelnet provides api for java, python, php, javascript, ruby and sparql. ## our approach we propose an approach for detecting the mfs which is an extension of the work done by @xcite . our approach follows an iterative procedure to detect the mfs of any word given its pos and language. it works for six different languages viz., english, spanish, russian, german, french and italian. we used babelnet as a lexical resource, as it contains additional information as compared to wordnet. this approach uses pre-trained google word embeddings foot_6 for english language, and for all other languages polyglot foot_7 word embeddings are used. the steps followed by our approach as shown in figure 1 are as follows -1. the system takes a word, pos and language code as an input. 2. for every sense of a word, features such as synset members, gloss, hypernym, etc. are extracted from babelnet. 3. sense embeddings or sense vectors are calculated by using this feature set. 4. cosine similarity is computed between word vector (word embedding) of an input word and its sense vectors. 5. sense vector which has maximum cosine similarity with the input word vector is treated as the mfs for that word. ## experiment and results we used pre-trained google's word vectors as word embedding for english language, for all other languages polyglot's word embeddings are used. due to lack of availability of gold data, we could not compare our results with mfs results obtained from babelnet. upon considering princeton wordnet as gold data, we cannot equate our results with it because they might be semantically similar but not syntactically. ## conclusion we proposed an approach for detecting the most frequent sense for a word using babelnet as a lexical resource. babelnet is preferred as a resource since it incorporates data not only from princeton wordnet but also from sources. hence the volume of ambiguity is reduced by a significant proportion. our approach follows an iterative procedure until a suitable context is found to detect the mfs of a word. it is currently working for english, russian, italian, french, german, and spanish languages. however, it can be easily ported across multiple languages. an api is developed for detecting mfs using babelnet which can be publically made available in future.