S mart S pan NER : Making S pan NER Robust in Low Resource Scenarios (2023)

Abstract: Named Entity Recognition (NER) is one of the most fundamental tasks in natural language processing. Span-level prediction (SpanNER) is more naturally suitable for nested NER than sequence labeling (SeqLab). However, according to our experiments, the SpanNER method is more sensitive to the amount of training data, i.e., the F1 score of SpanNER drops much more than that of SeqLab when the amount of training data drops. In order to improve the robustness of SpanNER in low resource scenarios, we propose a simple and effective method SmartSpanNER, which introduces a Named Entity Head (NEH) prediction task to SpanNER and performs multi-task learning together with the task of span classification. Experimental results demonstrate that the robustness of SpanNER could be greatly improved by SmartSpanNER in low resource scenarios constructed on the CoNLL03, Few-NERD, GENIA and ACE05 standard benchmark datasets.

## introduction ner is a fundamental information extraction task and plays an essential role in natural language processing applications such as information retrieval, question and answering, machine translation and knowledge graphs @xcite . the goal of ner is to extract named entities (nes) into predefined categories, such as person (per), location (loc), organization (org) and geo-political entity (gpe). with the rapid evolution of neural architectures @xcite @xcite and large pretrained models @xcite @xcite , recent years have seen the paradigm shift of ner systems from sequence labeling (chiu and nichols, 2016; luo conll03 few-nerd genia ace05 0 10 20 30 40 50 60 70 figure 2 : averaged f 1 scores of spanner and smartspanner methods over 10 independent runs on the test datasets of conll03, few-nerd, genia and ace05, where the models are trained on 1,000 sentences randomly sampled from the training data with the maximum epoch set to 10. @xcite to span prediction @xcite @xcite . it is pointed out that nested entities are very common in the field of ner @xcite , i.e., a named entity can contain or embed other entities as illustrated in fig. 1 . compared with seqlab, spanner has an obvious advantage: all candidate entities can be easily found with different sub-sequences, which is straightforward for nested ner @xcite . however, our experiments reveal that the performance of spanner drops much more than that of seqlab when the amount of training data drops, which makes spanner hard in low resource scenarios. in order to mitigate this problem, we propose a novel method smartspanner in this paper. by introducing a named entity head (neh) prediction task for each word in given sentences, we perform multi-task learning together with the task of span classification for ner. we conduct experiments on both flat and nested standard benchmark datasets (conll03, few-nerd, genia and ace05). experimental results demonstrate that smartspanner could improve the robustness in low resource scenarios significantly as shown in fig 2 . our contributions are summarized as follows: ## related work there has been a long history of research involving ner @xcite . traditional approaches are based on hidden markov model (hmm; @xcite or conditional random field (crf; @xcite . with the development of deep learning technology @xcite , seqlab methods such as lstm-crf @xcite and bert-lstm-crf @xcite achieve very promising results in the field of ner. however, these methods cannot directly handle the nested structure because they can only assign one label to each token. as it is pointed out that named entities are often nested @xcite , various approaches for nested ner have been proposed in recent years @xcite . one of the most representative directions is span-based methods that recognize nested entities by classifying sub-sequences of a sentence @xcite @xcite @xcite nesses of spanner have been systematically investigated by @xcite , its performance in low resource scenarios is not discussed. to the best of our knowledge, we are the first to highlight that the performance of spanner drops much more than that of seqlab when the amount of training data drops, which poses a challenge to the robustness of spanner in low resource scenarios. in order to address this challenge, we propose a novel method smartspanner. it should be noted that the low resource scenarios in this paper refer to those with at least 1,000 labeled sentences, different from the settings for fewshot ner @xcite and more common in real-world applications. ## analysis we have shown the robustness of smartspanner in low resource scenarios. in this section, we aim to take a deeper look and understand what contributes to its final performance. ## conclusion in this paper, it is found that the spanner method is sensitive to the amount of training data, i.e., the performance of spanner is worse than seqlab in low resource scenarios. in order to alleviate this problem, smartspanner is proposed by introducing the neh prediction task into spanner in a multi-task learning manner. the comparison results of experiments designed on the conll03, few-nerd, genia and ace05 datasets demonstrate that smartspanner is much more robust in low resource scenarios than spanner, and greatly reduces the running time of training and inferring. in addition, the reasons for the strong robustness of smartspanner are analyzed in depth on the dataset ace05.