Named Entity layer in E stonian UD treebanks (2023)

Abstract: In this paper we will introduce two new language resources, two NE-annotated corpora for Estonian: Estonian Universal Dependencies Treebank (EDT, 440,000 tokens) and Estonian Universal Dependencies Web Treebank (EWT, 90,000 tokens). Together they make up the largest publicly available Estonian named entity gold annotation dataset. Eight NE categories are manually annotated in this dataset, and the fact that it is also annotated for lemma, POS, morphological features and dependency syntactic relations, makes it more valuable. We will also show that dividing the set of named entities into clear-cut categories is not always easy.

## introduction named entity recognition (ner) is an important sub-task of information extraction. in order to build a ner tagger, one first needs to annotate a corpus for named entities (ne). in this paper we introduce two ne-annotated corpora for estonian: estonian universal dependencies treebank foot_0 (edt) and estonian universal dependencies web treebank foot_1 (ewt). by annotating these two resources for ne, we have aimed at broad coverage of genres, writing styles and correct vs relaxed compliance to the estonian spelling rules. although there are previous ne-annotated resources for estonian, we regard enriching existing ud corpora with ne annotation an important effort as ud annotations can support both manual annotation and help to build better ner models. we were also encouraged by the reports on similar efforts for finnish @xcite , danish @xcite and norwegian @xcite . in this paper, we first describe the underlying estonian ud treebanks in section 2.1. section 2.2 introduces the ne categories that are distinguished in the dataset and discusses some gray areas between these classes. corpus statistics is presented in section 2.3 and a brief overview of related work is given in section 3. the ne annotations are included in the release 2.12 of estonian ud treebanks. there are two estonian ud treebanks: estonian universal dependencies treebank edt and estonian universal dependencies web treebank ewt. edt contains ca 440,000 tokens in ca 30,000 sentences and its texts cover three central text types of normed written language: fiction, journalism and scientific writing. the text types of the treebank are not balanced: journalism with ca 270,000 tokens makes up more than half of the treebank, whereas fiction (ca 68,000 tokens) and scientific texts (ca 95,000 tokens) comprise the other half. ewt consists of texts from blog posts, online comments and discussion forums, it contains ca 90,000 tokens in ca 7000 sentences. universal dependencies annotation is described thoroughly on their website 4 . for the task of ne annotation it is relevant to point out that there is a special pos-tag for proper nouns (propn) and a special syntactic relation 'flat', that is used for exocentric (headless) structures, also for multiword names. so it is relatively easy to pre-annotate the majority of nes automatically, using these ud annotations. however, there are still some nes that don't include a proper noun, so an annotator still has to go through the entire text carefully. also, the exact extent of a named entity and its category have to be marked manually. ## conclusions and future directions we have presented two manually annotated ner datasets for estonian. the annotated texts represent the core text types of normed written language as well as several text types of the user-generated content of the web. the annotated nes fall into eight categories: persons, locations, geo-political entities, organizations, products, events, other nes that can't be classified into aforementioned categories and nes of unknown category. the category of geo-political entities is a hybrid category between location and organization. although we noticed that there exist also other cases of systematic metonymy besides using a location name to note the people connected with this location, we did not introduce more ne types as we did not want to divide the nes into too many too small categories. obviously the next step would be building ner models using this dataset. also, as the web treebank ewt is being developed further, annotating new genres of web texts with ud annotations, we plan to add the new texts into our dataset.