Semi-automatic W ord N et Linking using Word Embeddings (2018)

Abstract: Wordnets are rich lexico-semantic resources. Linked wordnets are extensions of wordnets, which link similar concepts in wordnets of different languages. Such resources are extremely useful in many Natural Language Processing (NLP) applications, primarily those based on knowledge-based approaches. In such approaches, these resources are considered as gold standard/oracle. Thus, it is crucial that these resources hold correct information. Thereby, they are created by human experts. However, manual maintenance of such resources is a tedious and costly affair. Thus techniques that can aid the experts are desirable. In this paper, we propose an approach to link wordnets. Given a synset of the source language, the approach returns a ranked list of potential candidate synsets in the target language from which the human expert can choose the correct one(s). Our technique is able to retrieve a winner synset in the top 10 ranked list for 60% of all synsets and 70% of noun synsets.

## introduction wordnets @xcite have been useful in different natural language processing applications such as word sense disambiguation @xcite , machine translation @xcite etc. linked wordnets are extensions of wordnets. in addition to language specific information captured in constituent wordnets, linked wordnets have a notion of an interlingual index, which connects similar concepts in different languages. such linked wordnets have found their application in machine translation @xcite , cross-lingual information retrieval @xcite , etc. given the extensive application of wordnets in different nlp applications, maintenance of wordnets involves expert involvement. such involvement is costly both in terms of time and resources. this is further amplified in case of linked wordnets, where experts need to have knowledge of multiple languages. thus, techniques that can help reduce the effort needed by experts are desirable. recently, deep learning has been extremely successful in a wide array of nlp applications. this is primarily due to the development of word embeddings, which have become a crucial component in modern nlp. they are learned in an unsupervised manner from large amounts of raw corpora. @xcite were the first to propose neural word embeddings. many word embedding models have been proposed since then @xcite @xcite . they have been efficiently utilized in many nlp applications: part of speech tagging @xcite , named entity recognition @xcite , sentence classification @xcite , sentiment analysis @xcite , sarcasm detection @xcite made a particularly interesting observation about the structure of the embedding space of different languages. they noted that there is a linear mapping between such spaces. in this paper, we address the following question: we demonstrate that this is true at least in the case of english and hindi wordnets. we propose an approach to link them using word embeddings. given a synset of the source language, the approach provides a ranked list of target synsets. this makes the overall linking task easy for human experts, as they have to choose from a relatively small set of potential candidates. our evaluation shows that our technique is able to retrieve a winner synset in the top 10 ranked list for 60% and 70% of all synsets and noun synsets respectively. ## background and related work princeton wordnet or the english wordnet was the first wordnet and inspired the development of many other wordnets. eurowordnet (vossen and others, 1997) is a linked wordnet comprising of wordnets for european languages, viz, dutch, italian, spanish, german, french, czech and estonian. each of these wordnets is structured in the same way as the princeton word-net for english @xcite ) -synsets (sets of synonymous words) and semantic relations between them. each wordnet separately captures a language-specific information. in addition, the wordnets are linked to an inter-lingual-index, which uses princeton wordnet as a base. this index enables one to go from concepts in one language to similar concepts in any other language. such features make this resource helpful in crosslingual nlp applications. indowordnet @xcite is a linked wordnet comprising of wordnets for major in-dian languages, viz, assamese, bengali, bodo, gujarati, hindi, kannada, kashmiri, konkani, malayalam, manipuri, marathi, nepali, oriya, punjabi, sanskrit, tamil, telugu, and urdu. these wordnets have been created using the expansion approach using hindi wordnet as a pivot, which is partially linked to english wordnet. previously, @xcite come up with a heuristic based measure where they use bilingual dictionaries to link two wordnets. they combine scores using various heuristics and generate a list of potential candidates for linked synsets. @xcite discuss a method to improve the current status of hindi-english linkage and present a generic methodology i.e., manually creating bilingual mappings for concepts which are unavailable in either of the languages or not present as a synset in the target wordnet. their method is beneficial for culture-specific synsets, or for non-existing concepts; but, it is cost and time inefficient, and requires a lot of manual effort on the part of a lexicographer. our approach is mainly geared towards reducing effort on the part of the lexicographers. ## problem statement given wordnets of two different languages e and f with sets of synsets {s 1 e , s 2 e , . . . , s m e } and {s 1 f , s 2 f , . . . , s n f } respectively, find mappings of the @xmath0, s j @xmath1. ## approach we adapted the technique of translating words in @xcite to translate synsets (see fig 1 ). in order to do so, however, we need "synset embeddings". we computed the same by assigning to a synset-id, the average of the "word embeddings" of its synset-members. to the best of our knowledge, this is a first attempt at solving this problem using word embeddings. the following is a detailed description of the technique. let e and f be two languages. let |e| and |f | be the number of synsets in wordnets of e and f respectively. let s i e and s j f be the i th and j th synsets of e and f respectively, with let v e p α be the word embedding corresponding to e p α . then we estimate v s i e , the embedding for synset s i e , as given links of the form s i e , s j f , we learn w such that the error err now, to find a mapping for a new synset s k e , one needs to our hypothesis is that for a given synset-id, the noise added to its representative embedding by a highly polysemous synset-member will be canceled out, while the actual information content pertaining to that synset-id will be enhanced, due to contribution from other, relatively less polysemous, synset members. ## experiments datasets we applied our technique to link hindi and english wordnets. we obtained a dataset of mappings between english and hindi wordnets from the developers of indoword-net. these mappings are of the form hindi synset id, english synset id, link type , where link @xmath2, hypernymy, etc.}. for this experiment, we focused solely on direct links. there are a total of 6,883 such mappings, the distribution among classes of which is mentioned in for the english language, we used the pretrained word embeddings published by google that were trained on part of google news dataset (about 100 billion tokens). these embeddings are of dimension 300, and are created using cbow model with negative sampling. for the hindi language, we trained word embeddings on bo-jar hindmonocorp dataset @xcite . @xcite suggests that the input embeddings' dimension should be at least 2.5 to 4 times that of the output dimension. but we also wanted to check what happens when they are equal. therefore, we trained two sets of embeddings, one of dimension 300, and the other of dimension 1200. ## conclusion and future work in this paper, we described an approach to link wordnets. it entails creating synset embeddings using the word embeddings of the synset members, and learning a function to map the embedding of a synset from the source language to an embedding in the space of target language, and returning the nearest neighbors as potential candidates for linking. our evaluation shows that our technique is able to retrieve a winner synset in the top 10 ranked list for 60% and 70% of all synsets and noun synsets, respectively. although, it did not achieve significantly good results for other classes, especially verbs. we discussed the possible reasons for poor performance and suggested mechanisms to address the same. in future, we plan to continue this work, and explore each of the above possible reasons for poor performance, in order to mitigate them. we will also evaluate it in an active learning setting. eventually, we aim to integrate our work with tools such as the ones created by @xcite , etc. so that our work can be used by lexicographers and researchers alike.