Studying Taxonomy Enrichment on Diachronic W ord N et Versions (2020)

Abstract: Ontologies, taxonomies, and thesauri have always been in high demand in a large number of NLP tasks. However, most studies are focused on the creation of lexical resources rather than the maintenance of the existing ones and keeping them up-to-date. In this paper, we address the problem of taxonomy enrichment. Namely, we explore the possibilities of taxonomy extension in a resource-poor setting and present several methods which are applicable to a large number of languages. We also create novel English and Russian datasets for training and evaluating taxonomy enrichment systems and describe a technique of creating such datasets for other languages.

## introduction nowadays, construction and maintenance of lexical resources (ontologies, knowledge bases, thesauri) have become essential for the nlp community. in particular, enriching the most acknowledged lexical databases like wordnet @xcite and its variants for almost 50 languages foot_0 or collaboratively created lexical resources such as wiktionary is crucial. resources of this kind are widely used in multiple nlp tasks: word sense disambiguation, entity linking @xcite , named entity recognition, coreference @xcite . there already exist several initiatives on wordnet extension, for example, the open english wordnet with thousands of new manually added entries or plwordnet @xcite which includes a mapping to an enlarged princeton wordnet. however, the manual annotation process is too costly: it is time-consuming and requires language or domain experts. on the other hand, automatically created datasets and resources usually lag in quality compared to manually labelled ones. therefore, it would be beneficial to assist manual work by introducing automatic annotation systems to keep valuable lexical resources up-to-date. in this paper, we analyse the approaches to automatic enrichment of wordnets. formally, the goal of the taxonomy enrichment task is as follows: given words that are not included in a taxonomy (hereinafter orphan words), we need to associate each word with the appropriate hypernyms from it. for example, given an input word "duck" we need to provide a list of the most probable hypernyms the word could be attached to, e.g. "waterfowl", "bird". a word may have multiple hypernyms. semeval-2016 task 14 @xcite was the first effort to evaluate this task in a controlled environment, but it provided an unrealistic task scenario. namely, the participants were given definitions of words to be added to taxonomy, which are often unavailable in real world. the majority of presented methods heavily depended on these definition. in contrast to that, we present a resource-poor scenario and solutions which conform to it. our contributions are as follows: this work is licensed under a creative commons attribution 4.0 international licence. licence details: http:// creativecommons.org/licenses/by/4.0/ . ## related work the existing studies on the taxonomies can be divided into three groups. the first one addresses the hypernym discovery problem @xcite : given a word and a text corpus, the task is to identify hypernyms in the text. however, in this task, the participants are not given any predefined taxonomy to rely on. the second group of works deals with the taxonomy induction problem @xcite @xcite , in other words, creation of a taxonomy from scratch. finally, the third direction of research is the taxonomy enrichment task: the participants extend a given taxonomy with new words. our methods tackle this task. unlike the former two groups, the latter garners less attention. until recently, the only dataset for this task was created under the scope of semeval-2016. it contained definitions for new words, so the majority of models solving this task used the definitions. for instance, @xcite computed definition vector for the input word, comparing it with the vector of the candidate definitions from wordnet using cosine similarity. another example is taln team @xcite which also makes use of the definition by extracting noun and verb phrases for candidates generation. this scenario may be unrealistic for manual annotation because annotators are writing a definition for a new word and adding new words to the taxonomy simultaneously. having a list of candidates would not only speed up the annotation process but also identify the range of possible senses. moreover, it is possible that not yet included words may have no definition in any other sources: they could be very rare ("apparatchik", "falanga"), relatively new ("selfie", "hashtag") or come from a narrow domain ("vermiculite"). thus, following russe-2020 shared task @xcite we stick to a more realistic scenario when we have no definitions of new words, but only examples of their usage. the organisers of the shared task provided a baseline and training and evaluation datasets based on ruwordnet @xcite . the task exploited words which were recently added to the latest release of ruwordnet and for which the hypernym synsets for the words were already identified by qualified annotators. the participants of the competition were asked to find synsets which could be used as hypernyms. the participants of this task mainly relied on vector representations of words and the intuition that words used in similar contexts have close meanings. they cast the task as a classification problem where words need to be assigned one or more hypernyms @xcite or ranked all hypernyms by suitability for a particular word @xcite . they also used a range of additional resources, such as wiktionary @xcite , dictionaries, additional corpora. interestingly, only one of the well-performing models @xcite used context-informed embeddings (bert). however, the best-performing model (denoted as yuriy in the workshop description paper) extensively used external tools such as online machine translation (mt) and search engines. this approach is difficult to replicate because their performance for different languages can vary significantly. thus, we exclude features which relate to search engines and compute the model performance without them. the same applies to pre-trained word embeddings to some extent, but in this case we know which data was used for their training and can make more informed assumptions about the downstream performance. moreover, the embeddings are trained on unlabelled text corpora which exist in abundance for many languages, so training high-quality vector representations is an easier task than developing a well-performing mt or search engine. therefore, we would like to work out methods which do not depend on resources which exist only for a small number of well-resourced languages (e.g. semantic parsers or knowledge bases) or data which should be prepared specifically for the task (descriptions of new words). at the same time, we want our methods to benefit from the existing data (e.g. corpora, pre-trained embeddings, wiktionary). ## diachronic wordnet datasets we build two diachronic datasets: one for english, another one for russian based respectively on princeton wordnet @xcite and ruwordnet taxonomies. each dataset consists of a taxonomy and a set of novel words not included in the taxonomy. ## taxonomy enrichment methods our method is based on the baseline distributional model from russe-2020 shared task extended with ranking of synset candidates and use the information from wiktionary and various types of embeddings. ## conclusion in this work, we deal with the taxonomy enrichment task: the automatic extension of an existing taxonomy with new terms. the novelty of our work is the use of diachronic versions of wordnets in two languages: this setting reflects the real process of development of the resources in time and allows us to study if machines could perform similar taxonomy completion task automatically. toward this end, we present a simple method for solving the task that can be applied to multiple languages. its results are close to the state-of-the-art for russian and stable across languages. an interesting finding is that the task does not benefit from context-informed embeddings,whereas context-free vector representations like fasttext often successfully identify hypernyms. on the other hand, the availability of wiktionary substantially boosts the performance, yet models based solely on embeddings still yield competitive results. error analysis reveals that some groups of words (e.g. named entities) are easier to find a hypernym for, and polysemous words are significantly more challenging. at the same time, our models were able to identify some cases of polysemy which were not reflected in wordnets. promising directions of future work are (i) application of the developed methods to automate work of lexicographers constructing wordnets and (ii) further improvement of results for verbs.