How do We Treat Systematic Polysemy in Wordnets and Similar Resources? – Using Human Intuition and Contextualized Embeddings as Guidance (2023)

Abstract: Systematic polysemy is a well-known linguistic phenomenon where a group of lemmas follow the same polysemy pattern. However, when compiling a lexical resource like a wordnet, a problem arises regarding when to underspecify the two (or more) meanings by one (complex) sense and when to systematically split into separate senses. In this work, we present an extensive analysis of the systematic polysemy patterns in Danish, and in our preliminary study, we examine a subset of these with experiments on human intuition and contextual embeddings. The aim of this preparatory work is to enable future guidelines for each polysemy type. In the future, we hope to expand this approach and thereby hopefully obtain a sense inventory which is distributionally verified and thereby more suitable for NLP.

## introduction systematic polysemy, also called regular polysemy, is a well-known linguistic phenomenon where a group of lemmas follow the same polysemy pattern @xcite @xcite and several others). for instance, the lemmas chicken and school belong to the patterns animal/food and location/institution due to their inherently dual meanings with different ontological types. the phenomenon is challenging to describe in theoretical linguistics as well as in practical lexicography where decisions need to be made regarding whether to split regular polysemous lemmas into several senses, or whether to see the meaning of these lemmas as inherently complex, 1 the danish abbreviation of 'the central word register'. with the individual context simply highlighting one or the other meaning. at times, a context does not specify any of the meanings and may highlight both equally. this kind of underspecification @xcite thus invokes two ontological types simultaneously, as seen in sentence a), where taste highlights a food reading of salmon, while lived a good life draw attention to the animal reading: a) you can taste if the salmon has lived a good life. in lexicons, systematic polysemy can be dealt with in two ways @xcite . first, a sense enumeration lexicon can be established where different readings of a lexical item are listed under a single dictionary entry. in the case of salmon, such an approach would list both the animal and food sense. this method is typically used in traditional dictionaries. alternatively, it can be treated with a onerepresentation approach motivated by the fact that it is impossible in praxis to list all existing meanings of a lexical item. instead, the lexicon describes regular patterns of sense alternations which also predict senses in a systematic way. a well-known example of the one-representation approach is provided in the generative lexicon @xcite . according to this approach, the salmon would be considered a complex type that denotes both the living animal and its corresponding meat. this paper describes the challenges of achieving a homogenous approach to represent systematic polysemy in lexical resources and discusses when to rely on a sense enumeration approach and when to underspecify. we perform our studies within the framework of the cor 1 lexicon, which is based on previous lexical resources that were not consistent in their treatment of systematic polysemy. overall, cor aims towards a restricted sense inventory where only distributionally 'verified' senses are maintained. the new lexicon is primarily based on the corpus-based monolingual danish dictionary: den danske ordbog (ddo). even though the dictionary mostly follows a sense enumeration approach, it occasionally uses a joint sense description for instances of systematic polysemy, typically in the case of less frequent lemmas in the corpus. in the cor lexicon, we rely heavily on our experience from compiling two other resources based on the ddo dictionary. first, in the danish wordnet project dannet @xcite , in which we took steps towards expanding the representations for specific systematic polysemy patterns, see @xcite . later, we compiled a danish thesaurus based on senses in ddo and dannet @xcite . we also take inspiration from alonso (2013), who examines expert and laymen annotations of the underspecified sense, however only on selected number of patterns. in the cor project we aim at a homogenous treatment of similar polysemy patterns throughout the whole vocabulary, and with specific information on the type of pattern as part of the lexical semantic information. we adopt a similar idea to @xcite who suggests a method for systematic polysemy detection through lexical resources. the strategy is based on the initial hand annotations of a set of polysemous lemmas in ddo, which are again informed with information from dannet. thereby, we examine the vocabulary both bottom-up and top-down to establish a typology of danish systematic polysemy patterns. the registered patterns lead to a set of rules stating whether the senses of a certain pattern must be reflected as either one or two cor lexicon senses. a subset of these rules is supplemented by two additional investigations, namely i) surveys on the human intuition, and ii) distributional investigations using a large, contextualised embedding model (bert). the idea of evaluating systematic polysemy by use of multiple information sources originates from the work of @xcite , who investigate an integrative method for distinguishing senses. they treat the sense distinction problem by including four perspectives: formal, cognitive, distributional, and multilingual. in our case, the combination of a formal semantic resource (dannet), a study of the human intuition, and a distributional analysis, allows us to analyse systematic polysemy from different angles, including how the patterns are perceived by humans and used in texts. for instance, we investigate whether cases of systematic polysemy are conceptualised by humans as one or multiple senses by asking informants whether context pairs invoke the same or different senses. by using a distributional approach, we examine whether the ontological types in a pattern are represented in texts. this is particularly relevant in the application of nlp, as texts do not necessarily reveal the metonymic relationship between the senses in systematic polysemy, and distributional models may not be able to distinguish such senses. the representation of systematic polysemy in lexical resources has been explored and discussed before @xcite . although, to our knowledge, this is the first study to use both language models and informants to analyse systematic polysemy to compile valid encoding guidelines for a practical resource, in our case the cor lexicon. the study also gives valuable feedback to the treatment of systematic polysemy in dannet and the ddo dictionary. the structure of the paper is as follows. section 2 introduces a typology of danish systematic polysemy patterns. in section 3 and 4, we present a preliminary study that analyses a selection of patterns in two ways, first using a survey of human intuition, then using a distributional model (bert). in section 5, we discuss the interaction of the different approaches, and discuss how the treatment of systematic polysemy in lexical resources can benefit from the results. ## a typology of danish systematic polysemy patterns in our annotation work, we have identified 28 danish systematic polysemy patterns based on the compilation of the central vocabulary in the cor lexicon @xcite . the project is initiated by the annotation of ~3,300 polysemous lemmas in the ddo dictionary. we consider this a core vocabulary of danish since they all have at least one sense which is linked to a core concept in the princeton wordnet (pwn) @xcite . in total, more than 15,000 senses are annotated. in the dataset, all patterns of systematic polysemy are identified based on information in the sense definitions in ddo and the taxonomies in dannet. the different patterns are analysed and discussed, resulting in a list of the most prominent systematic polysemy patterns in danish. as briefly mentioned above, an overall goal of the cor-project is not to reflect the fine-grained ddo sense inventory 1:1, but to compile a more coarse-grained sense inventory for danish which is suitable for ai purposes and computational applications. by identifying the patterns, we can apply a homogenous analysis across multiple lemmas with the same patterns. the starting point is the patterns registered in the projects dannet @xcite and the danish thesaurus @xcite , e.g., process/result, plant/ food, and animal/ food. however in contrast to these projects, we consider the entire lemma information including all senses, and not just concepts represented as standalone ddo senses. this allows us to detect patterns of polysemy in a systematic way, lemma by lemma. for instance, it is typical for the lemmas that hold the pattern location/institution to have 'building/ location' senses with similar definitions, which are typically listed under the same main sense as the 'institution' senses. by looking into dannet, we can also compare the ontological types and thereby detect the patterns top-down. during the discussion of the initially identified patterns, we questioned whether some patterns were actually cases of systematic polysemy or rather a case of the annotators being too eager to register patterns. therefore, we include a pattern in the typology only if it fulfils the following three criteria: a) at least five instances of the pattern can be found in the cor-dataset of ~3300 polysemous core lemmas. b) the danish dictionary (ddo) or dannet must distinguish between both senses of the pattern for most of the identified lemmas. c) each sense in a pattern must have distinct ontological types. the criteria a) and b) ensure that a pattern is prominent in danish by taking frequency and previous sense descriptions into account. if the pattern is systematic in danish, we assume that it would be reflected in the core polysemous part of the danish vocabulary. in the case of b), we must consider that the ddo in some cases prefers a single sense description. this is partly due to space limitations in the originally printed dictionary. the ddo typically uses sense enumeration when the lemma is frequent and a central simplex lemma, e.g., bog ('book'), while for compound nouns (e.g., kogebog ('cooking book')) as well as more rare semantic divisions @xcite . for each pattern, we decide whether the sense descriptions should be enumerated or combined. a combined sense gets the ontological type of the most prominent sense, unless both senses are evaluated as being equally important. in that case, the merged sense will be assigned two ontological types. in all cases, the pattern is labelled explicitly in the lexicon the decisions are based on the available information from dannet and ddo and supplemented by introspection and searches in corpora. we further partition the 28 patterns into five groups based on lyons' semantic divisions @xcite . thus, patterns that only include semantically concrete types fall into one group, while patterns that include a mix of concrete and abstract types fall into another. the groups are shown in ## humans' intuition on systematic polysemyan experiment to support our set of polysemy rules, we first, examine the phenomenon by including investigations on the human intuition. ## a distributional analysis with bert according to the distributional hypothesis, we can estimate the senses of a lemma from its distribution in language @xcite .. we investigate the distribution by performing a clustering experiment using the dataset described in section 0. the idea is to cluster the representations of a contextualised embedding model that has been pretrained on a large amount of textual data. if a systematic polysemy pattern is distinguishable in text, then the result will show separate clusters for each sense. ## discussion in this section, we discuss how the formal, the intuition-based, and the distributional approaches, respectively, contribute to our understanding of the different cases of systematic polysemy. we start by analysing the how each pattern is formally represented in the lexical resource, dannet. animal/food: all three approaches support that we separate our sense descriptions into an animal and food sense. in dannet, the pattern is consistently distinguished when both senses occur in ddo. each sense has its own synset with non-overlapping taxonomic structures. in the survey, the participants are also able to recognize the contrast between the living animal and its meat. 5 the python implementation is available here: https://hdbscan.readthedocs.io/en/latest/index.html the distributional approach separates the senses of the pattern for the frequent target laks ('salmon') but does not separate the less frequent target jomfruhummer ('langoustine'). we can explain the difference with the frequency of the food sense of jomfruhummer. since the bert model is trained on mostly web crawled texts, we expect a high number of recipes and food reviews in the text collection. therefore, the model might not have seen enough clear animal examples to create distinct representations. unfortunately, we do not have access to the exact training data and cannot confirm this hypothesis. however, we do know that our lexical resources contain this missing realworld knowledge, although for the infrequent lemmas, we see a mismatch between the sense descriptions and the language use. in dannet, the ddo genus proximum dyr 'animal' has led to only one sense, the animal sense, and the food sense has not been included even though the example is food. ## future work the approach described in this paper provides new insights into how to treat four frequent systematic polysemy patterns in the cor lexicon. a noticeable finding is that, as in the case of many other lexical phenomena, the patterns, and to some degree also lemmas within a pattern, tend to dispose quite individual properties. we would like to carry out similar investigations on the remaining part of the patterns in the typology we have presented, both in order to examine the prototypical behaviour for each pattern, and how this should correspondingly be represented in cor, but also to reveal the deviant cases. we think that by including information from both a survey among informants and statistical methods, we will be able to treat the many cases of systematic polysemy across the danish vocabulary in a more appropriate manner.