pl W ord N et 3.0 – Almost There (2016)

Abstract: It took us nearly ten years to get from no wordnet for Polish to the largest wordnet ever built. We started small but quickly learned to dream big. Now we are about to release plWordNet 3.0-emo – complete with sentiment and emotions annotated – and a domestic version of Princeton WordNet, larger than WordNet 3.1 by nearly ten thousand newly added words. The paper retraces the road we travelled and talks a little about the future.

## assumptions we based the development of plwordnet on several unique assumptions, formulated a priori. they have been discussed at length in previous publications, notably @xcite , so we will only recapitulate them briefly just to ease into the further discussion. first and foremost, we believe that lexicosemantic systems of different languages differ in deep -and interesting -ways. that is why plwordnet, meant as a precise description of the polish lexical system, had to be built in a way that avoided widespread influence of the material and structure of other wordnets. we were aware of the high cost of not simply translating princeton wordnet, the only resource large enough for our ambition, but it felt most important to be faithful to the complex reality of our language. 1 when the project began, there were no publicdomain and no open-licence large electronic lexico-semantic resource for polish. 2 we opted for a corpus-based wordnet development process. a very large corpus, the main knowledge source, is supplemented by a variety of linguistic substitution tests, mono-lingual dictionaries and other semantic language resources, encyclopaedias, discussions among linguists, and the wordnet editors' linguistic and lexicographic intuition. corpus-based work, unaided by specialised software, would necessarily be rather slow. we assumed large-scale software support for semiautomatic wordnet construction, predicated on the availability of support tools for editing. such tools were designed and built (and then perfected over the years) in parallel with fully manual construction of a small wordnet core to serve as the springboard for further expansion. this ensured much reduced workload for the editors and improved exploration of the corpus data. in many cases, the editor needs only to conform the support system's suggestions. 3 it soon became clear that there were significant problems with making the usual synset definition operational, and with the consistency of the editors' decisions. we chose a smaller-grain basic element for plwordnet: the lexical unit. foot_3 a synset was then defined indirectly as a set of lexical units which share a number of constitutive lexicosemantic relations and features @xcite . relations between synsets are a notational abbreviation for the shared relations between lexical units grouped into those synsets. constitutive relations, which define the structure of the wordnet, are complemented by relations which only link lexical units. three categories of constitutive features are lexical registers, semantic classes of verbs and adjectives, and aspect. in this model, synonymy is also a derived concept: constitutive relation-and feature-sharing lexical units grouped into a synset are understood to be synonymous. finally, in the construction of plwordnet we tried to follow the principle of a minimal commitment, that is to say, to keep the number of assumptions small, to make plwordnet transparent to linguistics theories of meaning, and to shape it in a close relation to language data. ## statistics wordnets are treated as basic lexical resources, so their sizes matter a lot for potential applications. see moreover, plwordnet has become larger than all modern dictionaries of general polish in terms of the entries included: 130k @xcite , 125k [180k lexical units] @xcite , 100k [150k lexical units] @xcite , 45k [100k lexical units] @xcite . one of the main reasons is that those dictionaries do not contain many specialised words and senses from science, technology, culture and so on. such material, however, is appropriate for a wordnet due to its applications in processing of texts of many genres coming from different sources, including the internet. we could also observe that lemma lists added to plwordnet (based on the corpus) included quite a few words that are now frequent, but not described in those dictionaries. the largest ever polish dictionary, from the early 1900s, has 280k entries @xcite @xcite and is still much larger than plwordnet, but it contains many archaic words, perhaps useful in the processing of texts from specialised domains. the achieved size of plwordnet has already exceeded the target size estimated for it considering a corpus of 1.8 billion words @xcite . lexico-semantic relations are the primary means of description of lexical meanings represented in a wordnet by synsets. the average number of relation links per synset, which is called relation density, tells us about the average amount of information provided by the wordnet for a single lexical meaning. if a wordnet is treated as a reference source, we expect to find in it most of the lemmas from the processed text. the complete coverage is not possible, but the higher it is, the more information a wordnet provides for the analysed text. the average polysemy -the ratio of lexical units to lemmas -is higher in plwordnet than in wordnet both for nouns (1.32 vs 1.24) and adjectives (1.71 vs 1.38). the difference is lower than in plwordnet 2.1: we added more specific monosemous lemmas as a result of the focus given to lexical units and the tendency to describe exhaustively all existing lexical units for a given lemma. for verbs we have 1.83 vs 2.17, maybe because of aspect and rich derivation in polish verbs. the comparison of hypernymy path lengths did not change much from plwordnet 2.1 @xcite ). wordnet's much longer paths are caused by the elaborate topmost part of its hypernymy hierarchy; plwordnet has ≈100 linguistically motivated hypernymy roots. 14 ## applications wordnet-building costs a lot of public money, so as a rule the effect should be free for the public use. this good rule, grounded in princeton word-net's practice, is central for languages other than english, still less resourced. the availability of plwordnet on the wordnet-style open licence has stimulated, over the years, many interesting applications in linguistic research, language resources and tools, scientific applications, commercial applications and education. the plwordnet web page and web service have had tens of thousands of visitors, and hundreds of thousands of searches. there are over 100 citations and over 700 users, individual and institutional, who optionally registered when downloading the plwordnet source files. most of the registered users described the intended use of plword-net, and a rich tapestry it is. the limited space only allows us to single out a handful in citations. first of all, plwordnet has been applied in linguistic research: valency frame description and automated verb classification; verb analysis for semantic annotation in a corpus of referential gestures; contrastive/comparative studies, etc. increasingly often, plwordnet is treated as a large monolingual and bilingual dictionary, e.g., in text verification during editing or as a source of meta-data for publications. @xcite included plwordnet among the dictionaries in a proofreading tool and as a knowledge source for an open polish-english dictionary, which many translators and translation companies say they use. open multilingual wordnet @xcite now includes plwordnet. it is referred to in several other projects on wordnets and semantic lexicons 14 they do not have hypernyms according to the definitions assumed in plwordnet. @xcite @xcite @xcite . practical machine translation systems use plwordnet. we are aware of applications in measuring translation quality and building the mt component embedded in an application supporting english teaching to children. there are more research and commercial projects, both under way and announced by plwordnet users. they include ontology building and linking, information retrieval, question answering, text mining, semantic analysis, terminology extraction, word sense disambiguation (wsd), text classification, sentiment analysis and opinion mining, automatic text summarisation, speech recognition, or even the practice of aphasia treatment. ## the lexicographer's work is never done when in 2012 we established the target size of plwordnet 3.0, we were convinced that we would go to limits of the polish lexical system. we now see that -even if major paths have been exploredwe are discovering numerous smaller paths going deeper into the system. the polish side of plwordnet could have more relation links per synset. the constitutive relations do not differentiate all hypernymy leaves yet. there are cross-categorial relations, more numerous than in many other wordnets, but still not enough for wsd or semantic analysis. the connection to the valency lexicon could be tighter. the description of verb derivation (as highly productive in polish as in other slavic languages) needs much more work, and so do some relations, e.g., meronymy. more information useful for wsd could be introduced, e.g., further glosses or links to external sources like wikipedia. finally, for applications in translation (manual and machine-based) we must not only complete the mapping to wordnet, but also go inside synsets, i.e., map lexical units. we are fortunate to have so much more intriguing work to do.