Mapping it differently: A solution to the linking challenges (2016)

Abstract: This paper reports the work of creating bilingual mappings in English for certain synsets of Hindi wordnet, the need for doing this, the methods adopted and the tools created for the task. Hindi wordnet, which forms the foundation for other Indian language wordnets, has been linked to the English WordNet. To maximize linkages, an important strategy of using direct and hypernymy linkages has been followed. However, the hypernymy linkages were found to be inadequate in certain cases and posed a challenge due to sense granularity of language. Thus, the idea of creating bilingual mappings was adopted as a solution. A bilingual mapping means a linkage between a concept in two different languages, with the help of translation and/or transliteration. Such mappings retain meaningful representations, while capturing semantic similarity at the same time. This has also proven to be a great enhancement of Hindi wordnet and can be a crucial resource for multilingual applications in natural language processing, including machine translation and cross language information retrieval.

## introduction wordnets are online lexical resources which are easily accessible, free to use, and fairly accurate. they play a dominant role in the field of text processing applications, such as machine translation, information extraction, information retrieval and natural language understanding systems. among the indian language wordnets, hindi wordnet foot_0 was the first one to come into existence from the year 2000 onwards. it was inspired by the english wordnet foot_1 which contains nouns, verbs, adjectives and adverbs organized into synonym sets called synsets, each representing one underlying lexical concept @xcite . different relations like hypernymy, hyponymy, etc. link the synonym sets to each other. soon, other indian language wordnets started getting created, with hindi wordnet as the pivot, inheriting all the relations. hindi wordnet is linked to the english wordnet and the other indian language wordnets are linked to hindi wordnet, in turn. this has led to the creation of a wide grid of shared concepts, thus creating an important knowledge base for the nlp community. to achieve maximum linkage between the english and hindi wordnets, the policy of having direct and hypernymy linkage @xcite has been adopted. however, it was observed that the hypernymy linkage does not lead to an accurate word and concept in all cases. thus, to overcome this challenge the idea of creating complimentary bilingual mappings in english came up. the roadmap of the paper is as follows: section 2 presents a comprehensive view of related work done earlier, while in section 3 the need for this approach is discussed. section 4 deals with the methodology. section 5 presents the qualitative analysis of the challenges encountered and the solutions put forth. in section 6 the interface used for this task is discussed. the overall statistics is given in section 7 and section 8 mentions some of the words sent to the english wordnet. section 9 winds up the paper with the conclusion and future work. ## related work wordnets have been built for around 100 different languages. efforts towards mapping synsets across wordnets have been going on for a while in various parts of the world. many languages have been trying to link their wordnets to the english wordnet for a universal set of linked concepts, enabling translation on the lexical level as well as cross-lingual wsd and other applications. usually, a concept in one wordnet is directly linked to a similar one in the english wordnet, but in many cases, some kind of mapping is required. corenet @xcite has made one such effort. it is a multilingual lexicosemantic network constructed in kaist for the korean, chinese, and japanese languages and many of its words/concepts have been linked to the english wordnet. to ameliorate translation problems between corenet (mostly written in korean) and the english wordnet and to enhance recall of wordnet equivalents, the two are partially indirectly linked via @xcite to the english wordnet. when this kind of indirect mapping is also not available, the concepts in corenet are manually mapped to the concepts in the english wordnet. in euroword-net @xcite foot_2 , multilinguality is achieved by storing the language-specific wordnets in a central lexical database in which equivalent word meanings across the languages are linked to a so-called inter-lingual-index (ili) to get a comprehensive conceptual match of concepts across languages. another effort towards wordnet linking can be found in the mul-tiwordnet @xcite foot_3 which aligns the italian and the english language wordnets. another such effort to create a multilingual wordnet is wwds @xcite . a similar task was performed for basque wordnet @xcite . bilingual mappings are a special case of wordnet linkage by which hindi wordnet deals with this problem. here, the concepts are translated, and, at times, the synset members are translated / transliterated, in english and this task is carried out manually. to the best of our knowledge this is a novel method and has not been implemented elsewhere. ## motivation -need for bilingual mapping the task of linking synsets of hindi wordnet to those of the english wordnet has been undertaken to create valuable parallel data for various nlp applications. however, languages are mirrors of the society in which they develop and are used. they are, therefore, unique and specific to particular geographical regions and cultural milieus. when two languages, which are as far set apart as hindi and english, have to be linked at the conceptual level, along with word transfers, it is bound to throw up the challenge of lexical and conceptual gaps. to overcome these challenges, the idea of having two types of linkagesdirect and hypernymyhas been followed although the hindi concept is very specific, yet it has been linked to the synset of charity which means 'an activity or gift that benefits the public at large' and is marked as a hypernymy linkage. however, consider the example of the synset छे दन (chhedan), which means -छे द करने की क्रिया -make hole act -chhed karane kii kriyaa -act of piercing. in the absence of a matching synset in the same pos category in the english wordnet, it had been given a hypernymy linkage to the english synset of deed, which means something that people do or cause to happen. what this would have implied was that each time the hindi word छे दन would occur in the corpus, the parallel english word deed would be its lexical counterpart in english. this is not only too far-fetched but may prove to be insufficient in the translation process. it is in such cases that the hypernymy linkage was substituted for a bilingual mapping. thus, for छे दन the lexical item was kept as piercing and the gloss as the act of piercing. moreover, the proper nouns in hindi wordnet, also pose a problem with hypernymy linkage. it is because of these issues that the idea of creating bilingual mappings of the hindi synset into english was adopted. here, mapping indicates the linking of two data-sets, in this case, between hindi wordnet and the data set containing english translations of glosses and words. ## method the method adopted for creating bilingual mappings is translation / transliteration of the synset member and the translation of the gloss in english. for this we search various lexical resources and look for valid usages on the internet. after verifying, we create the bilingual mapping. as far as possible, we do not coin words for this purpose, but there are some exceptions. the mapping is created in a dialogue box where the lexicographer manually types the required text, which is then stored in the database of hindi wordnet. users can see the mappings on the online hindi wordnet interface by querying for the english linkage. in cases where the hypernymy linkage is too distant then it is removed but where the hypernymy is close-enough, it is retained along with the mapping (see screenshot 2). the retention of hypernymy linkage is also motivated by the fact that it may prove useful for the general users, who may not be familiar with the language and the culture it represents. ## challenges and solutions for the creation of mappings we have divided words into four major categories based upon the problems faced. these categories and their treatment are as follows: ## bilingual mapping interface hindi and english wordnets are in both mysql and file format (text). hindi wordnet is accessible via an online interface, which provides a login facility to administrators, thus enabling features like adding / editing of a bilingual mapping between hindi and english over the web interface itself. we currently store the mappings based on hindi wordnet ids as pivot. mapping ids are provided serially. while querying the database for a linkage, the interface also looks for a mapping, which, if present, is shown on the interface. the bilingual mapping is stored in the database in the following format: <word1, word2, ... , @xmath0;<@xmath1 the lexicographers are familiar with this format, and update the database accordingly. following are some screenshots of the bilingual mapping interface for mapping / addition and querying of bilingual mapping. ## statistics the above statistics show that maximum numbers of synsets have direct linkages with the english wordnet. although, around 8,000 hypernymy linkages have been done, yet these are under review. some of these have been converted to bilingual mappings. since assigning mappings is a very recent activity, the numbers are likely to go up as the task proceeds. a bulk of this would comprise of proper nouns which consists mainly of names of persons, places and organizations. ## words sent to the english wordnet for inclusion as a corollary to the linkage task, it was observed that there are many english language concepts that are missing in the english word-net and can be easily assimilated therein. these concepts are available in other english dictionaries. we have sent lists of such words to the english wordnet team and have received assurance that these would be looked into. as and when such senses would be made available in the english wordnet, they will be utilized for the hindi-english linkage task. some examples of such words are given below: 1. page (computer science) -a quantity of memory storage equal to between 512 and 4,096 bytes. 2. flying -the piloting or navigation of an aircraft or spacecraft 3. occupier -one who seizes possession of and maintains control over forcibly or as if by conquest. 4. crisp -conspicuously clean or new 5. shakingly -in a shaking manner 6. in hand -owned by or in possession 9 conclusion and future work in this paper we have discussed the process of creating bilingual mappings of the synsets of hindi wordnet into english, the methods adopted and the tool used in creating them. it was observed that the problems occurred due to conceptual and lexical gaps between hindi and english languages. the main problem areas are the following: words/ concept not available in english wordnet  required sense missing in the english wordnet  culture specific words  language specific words -causative verbs -be' form of conjunct verbs -idioms  words for which hypernymy relation unavailable -adjectives -adverbs