A picture is worth a thousand words: Using O pen C lip A rt library for enriching I ndo W ord N et (2016)

Abstract: WordNet has proved to be immensely useful for Word Sense Disambiguation, and thence Machine translation, Information Retrieval and Question Answering. It can also be used as a dictionary for educational purposes. The semantic nature of concepts in a WordNet motivates one to try to express this meaning in a more visual way. In this paper, we describe our work of enriching IndoWordNet with image acquisitions from the OpenClipArt library. We describe an approach used to enrich WordNets for eighteen Indian languages. Our contribution is three fold: (1) We develop a system, which, given a synset in English, finds an appropriate image for the synset. The system uses the OpenclipArt library (OCAL) to retrieve images and ranks them. (2) After retrieving the images, we map the results along with the linkages between Princeton WordNet and Hindi WordNet, to link several synsets to corresponding images. We choose and sort top three images based on our ranking heuristic per synset. (3) We develop a tool that allows a lexicographer to manually evaluate these images. The top images are shown to a lexicographer by the evaluation tool for the task of choosing the best image representation. The lexicographer also selects the number of relevant images. Using our system, we obtain an Average Precision (P @ 3) score of 0.30.

## introduction our goal is to enrich the semantic lexicon of various indian languages by mapping it with images from the openclipart library @xcite . india is currently experiencing a major enhancement in the digital education sector with its vision of the 'digital india' program 1 . in this paper, we introduce an approach to enrich the indowordnet foot_1 @xcite , with images, which can help students and language enthusiasts alike. we envision the use of wordnets in the education sector to promote language research among young students, and provide them with a multilingual resource which eases their study of languages. wordnets have proven to be a rich lexical resource for many nlp sub tasks such as machine translation (mt) and cross lingual information retrieval. india has 22 official languages, written in more than 8 scripts. when a user reads a concept in a language that is not known to them, and moreover in an unknown script, an image can provide helpful insight into the concept. language learners in a multilingual country like this often face difficulty mainly due to: (a) not being able to find a mapping of the concept in the language being studied and their native language and (b) not being able to decipher the script in the language being learnt. in such cases a pictorial representation of a concept will be very useful. finally, systems for automatic image captioning and real time video summarization can leverge the power of image enriched word-nets. ## our architecture the following section gives the detailed architecture of our system. a diagrammatic representation is shown in figure 1 . also, we discuss the structure of the indowordnet and talk about how we link it to the retrieved set of images. ## retrieval procedure and scoring we use the openclipart api 6 to retrieve a set of results using the head word from a synset as the query, since openclipart is a free to use resource, unlike google search results which might retrieve copyright data. the api provides a json output which can be easily parsed using any programming language. we use java for this purpose. the result for each image provides the following data: to rank the results, we calculate a score based on overlaps between the synsets and image meta-data. the score is derived as a weighted overlap between the words in the title and tags of the result image with the words of the synset. words from each part are given a different weight owing to how useful the feature is in describing the image. for example, words from the title are given a higher weight as compared to words from the image tags. the algorithm increases the score if an overlap occurs and decrements the score otherwise. the magnitude of this increase and decrease depends on the weights of the words being compared. our system allows for all these weights to be tweaked. after the result images are scored, they are sorted based on this score. only the top three scoring images are downloaded. these downloaded images are then evaluated by lexicographers. ## results using the methodology described above, we map several synsets of the indian language end for 14: end procedure wordnets to the available images. a total of 8,183 hindi synsets for directly linked nouns were mapped to their corresponding images. we perform manual evaluation of the data using the tool mentioned above and have evaluated approx. 3,000 synsets for each of the languages. we continue with the manual evaluation for mapping as of now. the top three images are shown to a trained linguist who decides the winner image and also calculates the precision (p@3) of results for that synset. over a set of 8183 images, we obtain a precision (p@3) of 0.30. ## evaluation tool we create a php foot_2 based interface, and provide it to lexicographers and linguists for evaluation of the images obtained. the tool uses mysql foot_3 database at the back-end to store both hindi and english wordnet databases, and uses synset id as a pivot to display the images obtained. the tool provides with a hindi synset words, its concept, and the english words to help the lexicographer identify its proper sense. the lexicographer chooses a winner image out of the top three, or none of ## qualitative analysis in this section, we explain the work done to evaluate the resultant images and the analysis of the results. ## conclusion and future work we successfully identified images for synsets of indian languages and described our work on enriching indowordnet. many synsets could not be linked due to the lack of appropriate image availability on ocal. we also created a tool for manual evaluation of the data, or any other such work in the future. we evaluated the images obtained, and reported the highest precision score as 0.30. as a future work, we aim to try to retrieve these images using other open source image databases, and utilizing gloss and examples for finding overlaps. also, the concept of content based image retrieval (cbir) appears to be a viable option of several indian language synsets which cannot be directly linked to a single corresponding english synset. using cbir, we can harness resources of several untagged image databases, and thus further enich indowordnet as a resource.