Multilingual W ordnet sense Ranking using nearest context (2018)

Abstract: In this paper, we combine methods to estimate sense rankings from raw text with recent work on word embeddings to provide sense ranking estimates for the entries in the Open Multilingual WordNet (OMW). The existing Word2Vec pre-trained models from Polygot2 are only built for single word entries, we, therefore, re-train them with multiword expressions from the wordnets, so that multiword expressions can also be ranked. Thus this trained model gives embeddings for both single words and multiwords. The resulting lexicon gives a WSD baseline for five languages. The results are evaluated for Semcor sense corpora for 5 languages using Word2Vec and Glove models. The Glove model achieves an average accuracy of 0.47 and Word2Vec achieves 0.31 for languages such as English, Italian, Indonesian, Chinese and Japanese. The experimentation on OMW sense ranking proves that the rank correlation is generally similar to the human ranking. Hence distributional semantics can aid in Wordnet Sense Ranking.

## introduction most of the existing word-net sense rankings @xcite use document level statistics to find the prominent sense of the given word. @xcite showed that predominate senses could be learned from a sufficiently large corpus, and this work has since been extended by various researchers. words that appear nearest to the given word convey the context/meaning of a word @xcite @xcite @xcite , and this can be used to estimate the most frequently used senses. this proposed work uses nearest context words to predict the senses and computes the frequency of occurrence of these senses within the corpus. since most of the existing wsd systems utilize the most frequent sense (mfs) as a baseline, it is important to rank the wordnet senses in a meaningful way. two well-known software packages used to train word embeddings, are word2vec @xcite and glove model @xcite . polyglot @xcite is a natural language pipeline that supports many nlp based tasks such as tokenization, language detection, named entity recognition, part of speech tagging, sentiment analysis, word embeddings, morphological analysis and transliteration for many languages. this work utilizes their word embeddings. existing polyglot word embeddings @xcite support 137 languages. we have planned to use the word embeddings for the 35 handbuilt wordnets currently in omw @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite . we use corpus based frequencies for five of these languages (english, chinese, japanese, italian and indonesian) from the ntu multilingual corpus (ntu-mc: @xcite and use them to evaluate the learned sense rankings. our major contribution is training and testing on large numbers of multiword expressions, which are often neglected in the word embedding literature. we identify the multi-word expressions found in the handbuilt lexicons and train our own model for them using word2vec @xcite and glove @xcite . this paper is structured as follows. section 2 discusses the related work in word embedding and its application in wordnet synset ranking. section 3 describes the data, methods, and section 4 discusses the evaluation of results obtained from word embedding and its effect in wordnet sense ranking. finally, section 5 concludes with the findings and future plans to improve the results. ## related work word embedding techniques have been popular in recent years in word sense disambiguation (wsd) research. similar to this proposed work, @xcite computes word embeddings with the help of pretrained word2vec @xcite and matches with the sense embeddings obtained from the wordnet features. they have attempted wordnet sense ranking for hindi and english. since the word2vec @xcite model is based on the words frequency of occurrence in the corpus, finding the nearest context words that occur infrequently in the corpus is difficult. @xcite compares sense embeddings of adagram @xcite with ba-belnet @xcite synsets and proved that sense embeddings can be retrieved by automatically learned sense vectors. sense embeddings for a given target word are identified by finding the similarity between the adagram word embeddings list with the babelnet synsets words list. @xcite proposed an approach that takes word embeddings as input and produces synset, lexeme embeddings without retraining them. they used wordnet lexical resource to improve word embeddings. @xcite showed that word vectors can capture polysemy and word vectors can be thought of as linear superpositions of each sense vector. they have attempted discourse analysis to find the cluster of sense vectors. although the basic idea of word embeddings is not tied to any one languages, the preprocessing steps are language specific. kang et al. (2016) presented a cross-lingual word embedding for english and chinese word sense disambiguation (wsd). they have experimented with the performance of wsd using different word embeddings such as word2vec @xcite and glove model. @xcite compared word embeddings obtained from the word2vec @xcite model and the sense embedding obtained from the wordnet for english and hindi languages and restricted to nouns. they used various wordnet features similar to this proposed work to find the predominant sense. their approach outperforms semcor baseline for words with the frequency below five. in this research context words are identified with the help of polyglot(al-rfou et al., 2013) word embeddings. ## methodology in this work, we use word embeddings to find the nearest context of a given word and compare it with the senses obtained from the omw to find the most frequently used senses. our aim is to rank the senses obtained from the omw with the help of the context words and their frequency of occurrence. initially, we use the pretrained polyglot word embedding model @xcite to retrieve the nearest context words and found multiwords are unidentified. hence in this work, we have trained our own model similar to polyglot for both single and multi-words. our aim is to train this model for all 35 languages supported by omw, for this paper we present only the results for the five languages for which we have evaluation data: english, chinese, japanese, italian and indonesian. ## results and evaluation in this section, the word embedding models such as (glove: al-rfou et al., 2013) and (word2vec: @xcite have been evaluated on two different tasks such as wordsense ranking of wordnet and query expansion for clinical texts, then we present some examples of word embeddings for intuitive comprehension. the word sense ranking and trained word embeddings have been tested for 5 languages english, chinese, japanese, indonesian and italian languages of semcor dataset for the words with more than one sense. the polyglot2 word embedding model have been trained with the context window size as 14, initial learning rate as 0.025, hidden layer size as 32 and minimum word count as 2 @xcite . glove word embedding model has been trained with the minimum word count as 2, vector size as 100, maximum iteration as 100 and context window size as 14 @xcite . we use two metrics to measure the efficiency of the baseline and the proposed word embedding model. ## conclusion omw has over 150 languages with word-nets built automatically, ranging from major languages like german or korean for which there are no free word-nets, to smaller languages such as volapuk. for all languages for which polyglot has data (which is most of them) we will learn rankings and incorporate them into omw, so that the lexicon is maximally useful for speakers of as many languages as possible. in future, we planned to extend this work to identifying missing senses by comparing the trained model over the sense-annotated corpus with the existing pre-trained models like polyglot. since the glove model is based on co-occurrence context, it gave better results even for a tiny corpus, hence we have planned to extend our model to sentence embedding using glove model for finding nearest context sentences for a given synset example sentence to further improve our wordnet ranking.