Intrinsically Interlingual: The Wn Python Library for Wordnets (2021)

Abstract: This paper introduces Wn, a new Python library for working with wordnets. Unlike previous libraries, Wn is built from the beginning to accommodate multiple wordnets — for multiple languages or multiple versions of the same wordnet — while retaining the ability to query and traverse them independently. It is also able to download and incorporate wordnets published online. These features are made possible through Wn’s adoption of standard formats and methods for interoperability, namely the WN-LMF schema (Vossen et al., 2013; Bond et al., 2020) and the Collaborative Interlingual Index (Bond et al., 2016). Wn is open-source, easily available, and well-documented.

## introduction wordnet is a popular tool for natural language processing, and there are interfaces in many programming languages. for python alone, there are 99 packages that mention wordnet in the python package index. 3 many of them provide an interface to a single language, like romanian @xcite , or multiple languages from the same project, like @xcite for the indian languages. of course, there are many interfaces for english, of which the natural language tool kit's implementation is very widely used @xcite . the nltk has a very well documented and clear interface to the princeton wordnet @xcite , with several distance metrics also implemented. the interface makes some design decisions that simplify wordnet structure, such as treating the word ↔ sense ↔ synset triad as a lemma ↔ synset dyad. the wn library introduced by this paper differs from the existing python packages in several ways. it is not tied to any particular wordnet and does not immediately include any wordnet data, but instead it can read and use any wordnet published in the wn-lmf format @xcite . as a convenience, dozens of such wordnets hosted online are indexed by wn (see appendix a) for easy downloading. wn also differs from the nltk in that it uses the triadic structure, but in many ways it is deliberately similar to the nltk's interface in order to help smooth the transition for new users. in 2015, the nltk was extended to cover the wordnets in the open multilingual wordnet (omw 1.0; @xcite . 4 omw 1.0 was made to deal with wordnets produced by the expand method which take the structure of an existing wordnet, in this case pwn 3.0, and extend it by adding lemmas in a new language to existing synsets @xcite . most wordnets are built in this manner @xcite . the main advancement of omw 1.0 was in the production of multiple wordnets, all in the same format, under open licenses. 5 this structure where all synsets are shared among all languages has several advantages, such as the straightforward translation of words and the implicit sharing of structure which makes smaller wordnets more useful. it also has immediate disadvantages, the most prominent being that synsets not in pwn cannot be included. some disadvantages are more subtle: as the omw structure is the union of the structure of all the wordnets, new paths could become available when another wordnet is added, which has ramifications for reproducibility in research. in practice, for omw 1.0, all structure came from pwn 3.0 and non-english wordnets contributed no relations so this was never an issue, but we are anticipating future developments to omw which may cause such problems. to allow for wordnets with different structures and synsets not in pwn, a new version of the omw (2.0) is under development. it uses the collaborative interlingual index (cili; @xcite to link synsets. this allows wordnets to define their own synset structure while maintaining interlingual linking through the shared, resource-agnostic index. the software for the open multilingual wordnet 2.0 6 is released as open-source software with the aim of making it easily available for everyone. however, its primary goals were to allow the browsing of the unified resource and to facilitate the validation, addition, and management of new and historical wordnets and cili entries-not to assist the individual researcher with downloading and using particular wordnets from its collection. as such, the software is not optimized for loading just one or two wordnets, and while it can run locally, it is expected to run as a web service. in contrast, wn does fewer checks and assumes that the wordnets are generally well-formed. this assumption should hold if the wordnets come from a source that performs these checks, such as the omw. wn allows a user to only load the wordnets they need and to access them distinctly. it is designed for wordnet users running things locally. this paper describes a new python interface for modeling wordnet data, including those from omw 2.0, designed to replace the existing nltk interface in a researcher's workflow. wn is the first python module designed from the beginning to use the collaborative interlingual index to link separate wordnets. we discuss the desiderata for the software further in section 2. we then briefly discuss the design of the system in section 3. in section 4 we give a brief tour of wn's functionality. finally we discuss a couple of aspects of why we think wn is an improvement over previous implementations in section 5 and conclude in section 6. ## desiderata the main goals of wn are as follows: resource independence: each lexicon loaded into wn is treated as a distinct resource and 6 https://github.com/globalwordnet/omw/ may be added and removed without affecting other lexicons. wn-lmf compliance: wordnets in the modern wn-lmf format are fully supported and information is not lost upon loading or exporting wordnets. precise modeling: all information in a wordnet is available to and discoverable by the user through intuitive structures. notably, word senses have first-class status, just like words and synsets. interlingual queries: queries may traverse multiple wordnets, or not, depending on what the user specifies. user convenience: data sources and query results are readily available; the user does not need to comprehend the complexity of the software to use it. ## design here we discuss several aspects of wn's design, from the low-level database design in section 3.1 to the user-facing python data structures in section 3.2 and the methodology for performing interlingual queries in section 3.3. to support the distribution of wordnets as individual resources or in collections, formats for packaging wordnets are described in section 3.4. ## usage in this section we give a brief tour of wn's programming interface. ## discussion here we discuss how wn improves over previous offerings for users and researchers. 11 every ili should have only one synset in a lexicon.