pyiwn: A Python based API to access I ndian Language W ord N ets (2018)

Abstract: Indian language WordNets have their individual web-based browsing interfaces along with a common interface for IndoWordNet. These interfaces prove to be useful for language learners and in an educational domain, however, they do not provide the functionality of connecting to them and browsing their data through a lucid application programming interface or an API. In this paper, we present our work on creating such an easy-to-use framework which is bundled with the data for Indian language WordNets and provides NLTK WordNet interface like core functionalities in Python. Additionally, we use a pre-built speech synthesis system for Hindi language and augment Hindi data with audios for words, glosses, and example sentences. We provide a detailed usage of our API and explain the functions for ease of the user. Also, we package the IndoWordNet data along with the source code and provide it openly for the purpose of research. We aim to provide all our work as an open source framework for further development.

## introduction wordnets are extensively used in many sub-tasks for natural language processing (nlp) @xcite . they are a rich semantic lexicon which are accessible, freeto-use and fairly accurate. they have been used in cross-lingual information retrieval @xcite , word sense disambiguation @xcite , question answering @xcite etc. princeton wordnet @xcite or the english wordnet was the first to come into existence. eurowordnet @xcite followed with a common structure for 12 european languages. indian language wordnets originated with the advent of hindi wordnet @xcite and based on an expansion approach, the rest of them were created. they form a common lexico-semantic resource called indoword-net (iwn) @xcite . india has more than 22 languages and 18 of these have constituent wordnets under a common roof -indowordnet. a considerable effort has gone into the creation of a perfect application programming interface (api) for english wordnet and many of these are available for use, publically. although, we do not see that kind of push for a common api for indian language wordnets. nlp research for indian languages has seen tremendous growth in the recent past and wordnets are a crucial resource especially in the context of nlp methodologies based on knowledge bases. additionally, this will help emerging wordnets acquire our open-source framework and adapt to it for creating a simple python based api, thus helping nlp for their own language. ## motivation efforts to create a lexical semantic network for indian languages began with hindi wordnet foot_0 @xcite , and based on the concept of pivotal expansion, indowordnet foot_1 (bhattacharyya, 2010) was created. nlp for indian languages is gaining traction among the computer scientists in india, and a robust framework which is readily available for use is much needed. we believe that such an api bundled with the indowordnet data could be really helpful to the nlp community. princeton wordnet or the english wordnet api is available for use via nltk foot_2 @xcite in python. @xcite collaborate many wordnets and provide open access for using the wordnet data aligned with them. these wordnets from all over the world are linked via their linkages to english wordnet. indian language wordnets are also linked to english, but only 25000 out of 40000+ synsets. until the time all the synsets are unlinked; a separate api for browsing through indian languages is required, and a common api might not sufficiently cover the data available with indowordnet. hence, we build this api with an aim that in-dowordnet data should also readily available in an easy-to-use framework. python facilitates prebuilt libraries and datasets for nlp via nltk. ten-sorflow by google @xcite is also built on python, and other classic machine learning algorithms are available for use via the sci-kit learn (sklearn) library @xcite . hence, we choose python for implementing the api and build a framework using it. ## related work the java wordnet library foot_3 has been extensively used for research across various domains in nlp @xcite @xcite . extjwnl foot_4 extend jwnl and provides command-line support, and maven foot_5 support among many other features in their api. emerging wordnets like sinhala wordnet @xcite employ jwnl to create an api for their wordnet. java api for wordnet searching (jaws) @xcite is another such implementation. the mit java wordnet interface (jwi) foot_6 is also available for the same purposes and is available under the creative commons 4.0 license foot_7 . finlayson (2014) presents an extensive evaluation of the apis available in java for accessing princeton wordnet. all of the work above has been done for java, and is available for princeton wordnet. a python based toolkit, estnltk @xcite includes esto-figure 1 : basic flow of the pyiwn api nian wordnet developed under the eurowordnet project @xcite . previously, efforts had been made to create an api for indowordnet but they are not python based. @xcite describe a two-layered architecture of a web-based api created using php. it requires one to download data separately and is inconvenient to deploy; we also come across hard-coded paths while trying to deploy their api. a java-based api foot_8 is available for download on the hindi wordnet web interface, and also requires one to separately download the database for hindi wordnet. @xcite claim to have built an api for wordnets universally but their work is not publicly accessible, and no references to their implementation could be found. hence, we work on an api which would contain nltk like functionality, should be robust, readily available, and more importantly easy-touse. ## api design we choose python for implementation due to its widespread use in the nlp community and aim to align our work with nltk. with this in mind, we keep the design of our api similar to that of nltk wordnet interface foot_9 . our api provides access to synsets and their relational connections such as hypernymy, hyponymy, meronymy, etc. with other synsets for all the languages mentioned in the in each file, on each line, there is a synset with its unique identifier, the synset, gloss, examples, and the part of speech of the synset. there are 4 more files for each language for each of the part of speech: noun, verb, adjective, and adverb. these files contain the synsets for the respective part of speech of the synset. apart from the textual data, we also specifically augment hindi wordnet with speech data for all the words, glosses and example sentences. the speech data is in waveform audio file format (wav). this data is obtained using indic tts, a text-to-speech synthesis system for indian languages @xcite . our system provides access to iwn data for the languages mentioned in ## conclusion & future work we provide an api for accessing indowordnet using python. wordnet package in nltk is already widely used amongst nlp researchers; we provide them with a similar functionality for indian language wordnets and bundle the data along with. we also provide audio data in a separate package for the hindi language. currently, we only provide with functionalities such as displaying synset data, browsing through relational connections of a synset with other synsets, and access to speech data for the hindi language. we believe our work will help the nlp community by providing them with a robust and easy-to-use framework for indian languages. in future, we plan to add functionalities like getting the top-level relational synset, the path-length of longest and shortest relational synset, finding all parent/child synsets with respect to a lexicosemantic relation. we also plan to create voice models using a speech synthesis system for other indian languages or use pre-built voice models to generate audios for indian languages and augment our api with the audio data. in addition to this, we aim to provide morphological analysis for more languages other than hindi and marathi as a feature for the ability to search concepts using the inflectional form of a word. we hope our work helps the indian nlp diaspora further their research and gain more insights.