D a N +: D anish Nested Named Entities and Lexical Normalization (2020)

Abstract: This paper introduces DAN+, a new multi-domain corpus and annotation guidelines for Dan-ish nested named entities (NEs) and lexical normalization to support research on cross-lingualcross-domain learning for a less-resourced language. We empirically assess three strategies tomodel the two-layer Named Entity Recognition (NER) task. We compare transfer capabilitiesfrom German versus in-language annotation from scratch. We examine language-specific versusmultilingual BERT, and study the effect of lexical normalization on NER. Our results show that 1) the most robust strategy is multi-task learning which is rivaled by multi-label decoding, 2) BERT-based NER models are sensitive to domain shifts, and 3) in-language BERT and lexicalnormalization are the most beneficial on the least canonical data. Our results also show that anout-of-domain setup remains challenging, while performance on news plateaus quickly. Thishighlights the importance of cross-domain evaluation of cross-lingual transfer.

## introduction named entity recognition (ner) is the task of finding entities in text, such as locations, organizations, and persons. ner is a key step towards natural language understanding, for instance for question answering and information extraction. the task has received a substantial amount of attention, particularly for english. most research so far, including for danish, focused on newswire data and flat entities. it ignores nested entities, like 'australian open' (illustrated in figure 1 ) being both an event and a location-derived entity. there is also little prior work on transfer learning for nested ner. in this paper we introduce dan+, a novel resource for danish nested named entities and lexical normalization, covering texts from canonical data from newswire and non-canonical social media sources. danish bears interesting challenges for ner similar to german, which we capture by drawing inspiration from the nosta-d @xcite ner annotation scheme. in particular, location adjectives like 'dansk' (danish) or 'hollandske' (dutch) are not capitalized, and there are tokens which are only partially named entities, like 'baltica-aktierne' (the baltica shares). such entities were mostly ignored so far. full annotation guidelines for both tasks are provided in the appendix. contributions we present 1) dan+, a new multi-domain dataset for nested ner and lexical normalization; 2) an evaluation of various models for danish nested ner, including bert variants and in-language versus cross-language experiments; 3) first experiments of lexical normalization on danish and its downstream impact on ner. 1 2 related work nested nes have received less research focus in contrast to flat entities @xcite @xcite . this has been attributed to technological complexity @xcite and limited data availability @xcite . existing nested ne data mostly spans newswire and biomedical data for english @xcite and german news @xcite , for example. interest in nested ner is re-emerging @xcite , with many new recent neural approaches @xcite @xcite . to facilitate research, a fine-grained nested ner annotation on top of the penn treebank has been released recently @xcite . to facilitate research on a less-resourced language, namely danish, @xcite introduced publicly available evaluation data of flat ner on top of danish ud @xcite , providing annotations for approximately 20% of the data. the study also first benchmarked existing ner tools and evaluated the feasibility of transfer for danish. @xcite recently independently annotated the entire danish ud data for flat ners, though with different guidelines, annotating also adjectives, for example. before these two recent studies, danish ner data was behind a paywall or available tools were not benchmarked. to the best of our knowledge, dan+ is the first danish nested ner dataset beyond newswire. domain shift is a pressing issue in nlp. one solution is to normalize the input text before detecting nes, which is a mitigation strategy particularly suitable for social media @xcite . previous work has evaluated lexical normalization for a variety of languages-but not for danish-with varying degrees of success @xcite @xcite @xcite . most works do not evaluate the normalization model intrinsically, which is often restricted to a simple rule-based approach which is unlikely to transfer well. dan+ provides also data to study lexical normalization for danish. to the best of our knowledge, there is very little prior work on cross-lingual and cross-domain transfer for nested (or overlapping) entities. contemporary work includes english-arabic @xcite . ## data and annotation this section depicts the data sources and annotation. ## experimental setup for nested ner, we use bert @xcite with fine-tuning implemented in machamp (van der goot et al., 2020). 5 we evaluate three decoding strategies: we first evaluate all ner models on danish, both within news and on the three out-of-domain (ood) varieties. we further compare to transfer from german: 1) zero-shot transfer, fine-tuning only on german; and 2) union of the danish and german data for fine-tuning. we compare multilingual bert (mlbert) versus training with danish bert (danishbert). 6 even though both are trained on danish data, for mlbert this is wikipedia data, whereas danishbert is trained on wikipedia, common crawl, danish debate forums, and danish subtitles. for machamp, we use the proposed default parameters (van der goot et al., 2020) shown to work well across tasks. we tune early stopping and τ on danish news dev data, and set τ = 0.9. we compare our final model to the boundary-aware model @xcite , a state-of-the-art nested ner model which was also evaluated on germeval 2014. we train it with bilingual danish and german polyglot embeddings obtained via procrustes alignment @xcite . for evaluation we use the official germeval script @xcite with strict span-based f1 over both entity levels. 7 for normalization, we choose to use monoise (van der goot, 2019), since it is open-source and is the only model that has shown to reach competitive performance across multiple languages. monoise ## conclusions this paper contributes to the limited prior work on cross-lingual cross-domain transfer of nested ner. we provide a new resource for danish, dan+, with baselines on nested ner and lexical normalization, using two bert variants and training on danish, german or both. our results show that bert-based variants are sensitive to domain shift for cross-domain nested ner, whereas they can cope relatively well with missing in-language data. results on normalization show that it helps in case of very non-standard data only, for which automatic normalization improves danish nested ner performance.