pl W ord N et in Word Sense Disambiguation task (2016)

Abstract: The paper explores the application of plWordNet, a very large wordnet of Polish, in weakly supervised Word Sense Disambiguation (WSD). Because plWordNet provides only partial descriptions by glosses and usage examples, and does not include sense-disambiguated glosses, PageRank-based WSD methods perform slightly worse than for English. However, we show that the use of weights for the relation types and the order in which lexical units have been added for sense re-ranking can significantly improve WSD precision. The evaluation was done on two Polish corpora (KPWr and Składnica) including manual WSD. We discuss the fundamental difference in the construction of both corpora and very different test results.

## introduction large wordnets are often treated as sense inventories that describe and enumerate word senses. if we want to process texts at the level of wordnet senses, a very useful operation, we first must map text words to those senses, i.e. to perform word sense disambiguation (henceforth wsd). this is only trivial for monosemous words. wsd methods built upon supervised machine learning achieve good accuracy but are intrinsically impractical in their dependence on corpora that have been manually disambiguated with respect to word senses. needless to say, such corpora are very laborious to annotate. weakly supervised wsd methods that use a wordnet as the basic knowledge source, but do not depend on a manually annotated corpus, can fully utilise wordnet senses, i.e. they can in theory assign any sense stored in a wordnet to words in text. so, in spite of their lower precision they seem to be noteworthy as a potentially practical solution. most wordnet-based weakly supervised wsd methods are based on the idea of spreading activation in the wordnet graph, where the initial activation comes from the words in a textual context. several methods based on this general scheme were proposed. a short overview is presented in section 2. most such methods were developed and tested on princeton wordnet (pwn) @xcite that is slightly different than plwordnet @xcite , currently the world largest wordnet. first attempts to transfer the methods with good performance on pwn to plwordnet @xcite were encouraging; the performance is relatively close to the performance of the supervised methods observed for polish on limited test sets @xcite . in addition to the differences between both wordnets, pwn has been enriched with various other resources in order to obtain better performance of unsupervised wsd. first of all, additional links between synsets were created on the basis of the manually disambiguated semcore corpus @xcite . such links have contributed significantly to the increase of wsd performance. there is no polish corpus similar to semcore. the goal of the work presented here is to explore the structure and specific properties of plwordnet in order to improve the precision of the wsd methods based on the spreading activation in the wordnet graph, here the plwordnet graph. in the rest of the paper, first we will briefly overview the existing wordnet-based unsupervised wsd methods, including their known applications to plwordnet. next, the plwordnet model will be discussed and compared with pwn from the perspective of utilising different features in wsd method. on this basis, several possible versions of unsupervised wsd will be introduced. finally, we will present data sets used in the evaluation and the results achieved for different settings used in wsd methods. based on the results, we will analyse the the specific properties of plwordnet and its development process and its influence on wordnetbased unsupervised wsd methods for polish. lesk's algorithm @xcite ) can be applied to textual definitions constructed on the basis on of synsets, e.g. from glosses, examples and synset members. the definitions are next compared with the occurrence contexts of words. different similarity measures can be applied. the main problems are limited lenghts of the constructed definitions and high computational complexity, because many word sets must be compared. weakly supervised wordnet-based wsd algorithms assume that if we map words senses pertaining to a text fragment onto the wordnet graph, we can expect that the "hits" are located in short distances (in terms of paths) from each other in the wordnet graph. moreover, we can use a kind of spreading activation algorithm in order to move this information along the wordnet graph, analyse the "hot" areas and identify word sense, i.e. lexical units (lus), 1 located in them or close to them. those lus should be the most likely senses for words in the text. there are several parameters to set in this general scheme: the initial activation (text words vs lus), spreading algorithm (topology and relations) and identification of association between "hot" areas and lus to be chosen. various methods propose a range of decisions. weakly supervised wsd methods are mostly based on the pagerank algorithm @xcite for spreading. @xcite proposed application of the original pagerank to wsd called static pagerank. page rank algorithm (henceforth pr) is an iterative method for ranking nodes in the graph g. in wsd the nodes in g represent synsets and the 1 see section 3 for more on lus. m n ×n ins the adjacency matrix of the wordnet graph with n nodes (synsets), where m @xmath0, 0 otherwise; d i is degree of the node s i (representing the synset i); where c is the damping factor; v n×1 is the vector of the initial scores for nodes and p n×1 is a vector of node scores updated in every iteration. in static pagerank (spr) all values in v are equal 1/n . agirre and soroa (2009 ), agirre et al. (2014) proposed a modified version called personalised pagerank (ppr) in which the values in v, called personalised vector, depends on the text context of the disambiguated word. the non-zero score values are assigned to those nodes which are contextually supported. in ppr all words from the context are disambiguated at once. the v values are equal to: in @xcite , spr algorithm for polish was based on plwordnet 2.1. the graph consisted of synsets linked by edges representing a selected subset of the synset relations. the precision on nouns (43%) and verbs (28%) was low in comparison to the works for english. the algorithm was evaluated on the kpwr corpus of polish discussed in section 5. in the second version, a measure of semantic relatedness was utilised to add links to plwordnet. the measure had been extracted automatically from a large corpus of 1.8 billion words. however, there was no improvement: the precision for nouns was 37% and 27% for verbs. nevertheless, we observed that even a wsd method of limited precision can be helpful in improving the performance of text clustering. next we adapted several algorithms: spr, ppr and ppr_w2w -to polish resources @xcite . plwordnet 2.2 was used with all synset relations for the edges. due to the lack of word-sense disambiguation of glosses, no additional synset links could be added. the achieved precision (on kpwr) was in the range 42.79%-50.73% for nouns and 29.79%-32.94% for verbs. ppr_w2w produced the best results. we also tested different variants of combining plword-net with the suggested upper merged ontology (sumo) @xcite on the basis of the mapping constructed in @xcite . all three pr-based algorithm were evaluated. a slight improvement of the precision for nouns up to 50.89% for ppr_w2w could be observed when the two joined graphs were treated as one large graph. ## plwordnet properties the detailed description of the model underlying plwordnet can be found in @xcite , below we present only a concise overview due to the space limit. lus that share a set of constitutive lexico-semantic relations are grouped into synsets that are considered to consists of near synonyms. synset relations are notational abbreviations for the relations shared between lus from the linked synsets. the relations are the basic means of describing word senses. different types of relations express different semantic associations, and provide different semantic information. this properties can be explored in wsd to improve the use of knowledge during spreading activation in the graph. plwordnet provides as well some additional means of semantic description: stylistic registers, glosses and use examples. stylistic registers signal pragmatic constraints on the use of lus. however, such subtle differences are difficult to explore in wsd methods, so we have not done it. glosses in plwordnet are comments to the lus (not to synsets like in pwn) provided for a human reader in order to explain the motivation behind the given word sense and clarify its difference from other senses of the same lemma. glosses are short descriptions but they are not proper lexicographic definitions and are much less elaborated from the point of view of their application in lesk's algorithm @xcite . glosses are intended to be secondary and additional to the lexico-semantic relations that are the primary tool for the description of the lexical meanings in plwordnet, e.g. the genus information is expressed by hypernymy and should not be provided in a gloss. as such they have been added only to a subset of lus. in addition to glosses, lu can be described by one or more use examples. they are also focused on human readers, but they can be used in wsd as an additional source of information. there have been not attempts so far to disambiguate word senses in the plwordnet glosses and examples. finally, there is no information about the frequency or salience of lus, e.g. in comparison to other lus of the same lemma. numerical identifiers of lus and the order of synsets in the plword-net database mostly originate from the order in which editors introduced them into the database. ## exploring plwordnet in wsd taking as a starting point the work of @xcite and the observations in the previous section, we explored several ways of using the knowledge present in plwordnet to improve wsd performance. ## results and evaluation evaluation was based on applying the analysed algorithms to a corpus with manually disambiguated lus (word senses). as a main criterion for evaluation we used the precision, calculated by comparing the lus assigned by annotators and the algorithms, see equation ( 3 ): ## conclusions weakly supervised wsd methods based on plwordnet have slightly lower precision in tests on polish wsd corpora than similar pwn-based methods. however, plwordnet does not provide glosses for all lus and the existing glosses are not disambiguated. instead we looked into utilisation of other features. we showed that except glosses and examples, we can explore relation types by weighting them for the needs of wsd and the order in which lus have been added to plwordnet. both resulted in the increased precision of wsd on one of the test corpora -the one that seems to be closer to the practical applications. while the positive influence of the relations weights on pagerank-based wsd algorithm had been expected, the positive influence of the lus adding order is a surprise, as the wordnet editors were not asked to use any specific order in introducing new lus into plwordnet. thus they have to share some idea of the salience or frequency of the individual lus for the given lemma. this effect may not be visible when we analyse lists of lus of individual lemmas, but it seems to be the most probable explanation for the results wsd algorithms using this order as a knowledge source. in future work we plan to develop more sophisticated system of weights assigned to relations for wsd and to work on combining different knowledge sources in one complex wsd algorithm.