Towards a principled approach to sense clustering – a case study of wordnet and dictionary senses in D anish (2018)

Abstract: Our aim is to develop principled methods for sense clustering which can make existing lexical resources practically useful in NLP – not too fine-grained to be operational and yet finegrained enough to be worth the trouble. Where traditional dictionaries have a highly structured sense inventory typically describing the vocabulary by means of mainand subsenses, wordnets are generally fine-grained and unstructured. We present a series of clustering and annotation experiments with 10 of the most polysemous nouns in Danish. We combine the structured information of a traditional Danish dictionary with the ontological types found in the Danish wordnet, DanNet. This constellation enables us to automatically cluster senses in a principled way and improve inter-annotator agreement and wsd performance.

## lexical resources and word sense disambiguation (wsd) dealing with finegrained lexical sense inventories in nlp is a challenging task. selecting the correct sense in a specific context is incredibly hard when word meaning is richly described with subtle and detailed sense distinctions as found in most wordnets and lexica. to this end, coarse-grained word-sense disambiguation has become a well-established discipline over the years. one way to obtain a coarsegrained sense inventory is to cluster existing inventories either manually or automatically @xcite @xcite @xcite . in recent years, also so-called supersense tagging has become popular where wordnet's first beginners 1 are applied as a cross-lingual sense inventory. in recent experiments on danish cor-1 cf. https://wordnet.princeton.edu/man/lexnames.5wn.html pora we achieved state of the art results in both annotator agreement and automatic supersense tagging @xcite @xcite . nevertheless, our experiments also demonstrated that the inventory was not particularly well suited for our purpose. first of all, the inventory proved too coarse in a considerable number of cases (see @xcite for a discussion), and secondly, the set did not facilitate annotations across part-of-speech as in the case of de-verbal nouns resulting in unbalanced annotations between nouns and verbs. in the present work, we pursue a slightly different path by returning to the monolingually and corpus-defined sense inventory of our monolingual lexical resources, the danish wordnet, dannet, and the danish dictionary (den danske ordbog, ddo) on the basis of which dannet was originally compiled @xcite ). our aim is to further examine the potential of a principled method for sense clustering to be performed automatically on existing fully-fledged sense inventories. the basic idea is to combine the structured information of a traditional danish dictionary with the ontological types found in the danish wordnet, dannet, and to develop clustering methods on this basis. for our lexical sample study, we select 10 of the most polysemous nouns in danish; we study how the senses are organized in ddo and dan-net and how they can be automatically clustered following two different principles: one allowing for clusters only within the same main sense, and one where also clustering of main senses are allowed except for the cases of homographs. for both sense inventories we perform manual annotation and word sense disambiguation using the liblinear package and compare the results. ## establishment of clusters following the line of the discussion in section 2, it does not seem appropriate just to collapse all ddo subsenses with its main sense; this would leave all metaphorical senses (which are indeed very frequent in our corpus) very poorly represented. we combine the information types from both resources: the ddo and dannet and to this end, we perform three annotation experiments:  experiment 1 ('regular') where all main and subsenses are maintained.  experiment 2 ('clustered') where subsenses are clustered if they are of the same ontological type, and  experiment 3 ('clustered reduced') where also main senses are clustered if they are of the same ontological type. even if the ontology enables groupings of synsets which are ontologically similar (for instance artifact/part of artifact artifact/group of artifacts, person/groups of persons), we have in these experiments adopted a rather conservative approach and only clustered senses with the exact same ontological type. often a narrowed or an extended sense will have the same ontological type, in other cases a similar one. in contrast, figurative senses are typically of a completely different ontological type and are preserved with this method. the texts selected for annotation have been extracted from the 45 million words clarin reference corpus @xcite . this corpus comprises a wide variety of text types and domains: blog, chat, forum, magazine, parliament debates (written down by professionals), and newswire, of which the latter constitutes 48 % of the entire corpus. in line with the senseval approach ( www.senseval.org ), the number of annotated sentences for each noun varies according to the number of ddo senses of the noun (100 + 15*no. of senses), resulting in 177 to 535 sentences per noun. it turned out that the otherwise very frequent nouns that we selected are not very frequent in social media texts, and since it is important for the project to have all text types including social media represented in the annotated data, all sentences from this text type that contained the noun in question were extracted from the corpus. still to reach the specified number of sentences for each noun, we ended up with a majority of sentences from newswire texts. for the annotation task we used the tool webanno @xcite , which facilitates calculation of the inter-annotator quality and adjudication of the annotated files. for each occurrence of the word to be annotated, the annotators select a sense from the list of clustered senses in a drop down menu, see fig. 6 . ## word sense disambiguation using the liblinear package we also perform an experiment to see how empirical methods can perform in such hard tasks. the task is to disambiguate some specific words in a sentence (lexical sample task), and to see if there is any significant improvement of the prediction accuracies, when using clustered word senses. the features that we use include a bag of lemmas of the whole sentence. we also include the next and previous four lemmas. these last elements are devised to disambiguate idiomatic expressions whose structure is mostly fixed. as currently the data includes information from several annotators, training and evaluating machine learning classifiers is not straightforward. the main problem is the evaluation of a model. if two or more annotators have tagged a word in a sentence with diverging sense cluster tags, we consider it correct if an ml classifier classifies that instance as one of those sense clusters (either of them). this corresponds well to the fact that most divergences are caused by underspecified corpus examples. for learning, if two different annotators have tagged an instance, we consider it to be two different instances, resulting in some cases where we can have two instances with the same attributes, but with different outputs. as the amount of data is limited, we decided to perform a 5-fold cross-validation to check if the classifiers work sufficiently. we train a linear support vector machine for its robustness when used with a high number of features. the toolkit that we employ is the well-known liblinear package 3 @xcite , included in the module scikit-learn @xcite ) from python. accuracies of the word disambiguation tasks with the three types of sense inventories compared to a baseline are provided in figure 8 . on average, reduced clusters can be seen to outperform the experiments with the more fine-grained sense inventories. ## concluding remarks in this paper we have examined how we can cluster noun senses in a principled way based on dictionary and wordnet information in combination (main and sub-senses versus ontological typ ing). we have dealt with some of the hardest and most polysemous nouns in danish. we have further examined how systematically clustered noun senses influence inter-annotator agreement and automatic word sense disambiguation in a positive way, resulting in our last experiment (reduced clusters) in a sense inventory which seems actually manageable and well-functioning for both the annotators and the automatic disambiguation system. how our method will apply to verbs and adjectives is still an open question; for these word classes other information types than ontological typing may be more crucial. finally, only little space has however been left to discuss to which extent the meaning distinctions that are established by our clustering methods are actually relevant. relevance depends on our purpose and on the kind of language technology service we are aiming at, where translation generally demands a high degree of detail, information search quite less, and question answering maybe something in between. in future work we would like to include relevance criteria as a more dominant feature encompassing also elements such as sense frequency and predominance information of senses; information which is however not directly accessible for danish at the current stage.