Transfer-based Enrichment of a H ungarian Named Entity Dataset (2021)

Abstract: In this paper, we present a major update to the first Hungarian named entity dataset, the Szeged NER corpus. We used zero-shot cross-lingual transfer to initialize the enrichment of entity types annotated in the corpus using three neural NER models: two of them based on the English OntoNotes corpus and one based on the Czech Named Entity Corpus finetuned from multilingual neural language models. The output of the models was automatically merged with the original NER annotation, and automatically and manually corrected and further enriched with additional annotation, like qualifiers for various entity types. We present the evaluation of the zero-shot performance of the two OntoNotes-based models and a transformer-based new NER model trained on the training part of the final corpus. We release the corpus and the trained model.

## introduction 1.resources named entity recognition is a fundamental nlp task that has played an important role in tasks like information extraction, document deidentification, conversational models, etc. following the annotation scheme used in the conll 2002/2003 ner annotation tasks, legacy named entity corpora usually contain annotation of four entity types: organizations (org), persons (per), locations (loc) and general entity category covering all the rest (misc). this is the case for all named entity corpora available for hungarian, the szeged ner corpus @xcite , the silver-standard hungarian hunnerwiki corpus @xcite automatically derived from wikipedia, and the recently published nerkor corpus. 1 the english ontonotes 5 corpus @xcite , on the other hand, contains a richer set of entities. geopolitical entities (gpe: countries, settlements, etc.) and facilities (fac: buildings, roads, airports etc.) are differentiated from geographical locations like continents or bodies of waters. within the misc category, products (prod), laws and other norms (law), events (event) and titles of works of art (work of art) are differentiated. in addition, the ontonotes ner tagset also encompasses time and numerical expressions distinguishing dates and times, cardinal and ordinal numbers, quantities, percentages and amounts of money. in addition, other categories covering nonentities like languages (language) and nationalities, religions and political affiliations (norp 'nationality/other/religion/political') are covered, presumably just because english orthography happens to prescribe capitalization for words (adjectives in the case of norp) belonging to this category. some resources in languages other than english also use ner tagsets richer than the basic fourclass tagset. although the nosta-d resource used in the germeval2014 shared task targeting german ner @xcite maintains a four-class distinction, words (especially adjectives) derived from names as well as compounds containing them are marked as such. this corpus, similarly to other resources like the genia corpus @xcite containing biomedical entities and the spanish and catalan newspaper text corpus ancora @xcite , also features nested named entities. one of the most richly annotated ner corpora is the czech named entity corpus @xcite . it contains both a rather rich set of entity types and nested entities. ## method in the project presented in this paper, we significantly enriched the annotation in the first hungarian named entity dataset, the szeged ner corpus @xcite . ## models and performance we tested the zero-shot performance of the original ontonotes-based models on the final corpus disregarding entity types not covered by the ontonotes annotation. the flair model achieves f 1 = 75.20 with a great proportion of the errors coming from erroneously included definite articles. considering all tag types, the performance is f 1 = 67.46. a simple fix of the definite article problem boosts performance on common tags to f 1 = 87.91, and f 1 = 80.63 on the full tagset. the deeppavlov model trained on the same dataset fared much worse achieving only f 1 = 58.26 on common tags and f 1 = 53 on the full tagset. we trained a vanilla neural sequence tagger using the huggingface transformers library @xcite fine-tuning the monolingual hungarian hubert language model (nemeskey, 2021) using a 9:1 train:test split of the corpus. it achieved f 1 = 92.69, performing significantly better than the zero-shot models. ## conclusion in this paper, we presented the procedure we followed to enrich the annotation in a legacy hungarian ner resource by applying ner models based on multilingual language models and fine-tuned on ner corpora in other languages. we then made a significant effort identifying errors and correcting the annotation using automatic and semi-automatic methods, providing a solid base for the final manual annotation correction. we trained a neural sequence tagger on the final corpus achieving a solid f 1 = 92.69 performance.