Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition (2021)

Abstract: Named entity recognition (NER) is a well-studied task in natural language processing. Traditional NER research only deals with flat entities and ignores nested entities. The span-based methods treat entity recognition as a span classification task. Although these methods have the innate ability to handle nested NER, they suffer from high computational cost, ignorance of boundary information, under-utilization of the spans that partially match with entities, and difficulties in long entity recognition. To tackle these issues, we propose a two-stage entity identifier. First we generate span proposals by filtering and boundary regression on the seed spans to locate the entities, and then label the boundary-adjusted span proposals with the corresponding categories. Our method effectively utilizes the boundary information of entities and partially matched spans during training. Through boundary regression, entities of any length can be covered theoretically, which improves the ability to recognize long entities. In addition, many low-quality seed spans are filtered out in the first stage, which reduces the time complexity of inference. Experiments on nested NER datasets demonstrate that our proposed method outperforms previous state-of-the-art models.

## introduction named entity recognition (ner) is a fundamental task in natural language processing, focusing on identifying the spans of text that refer to entities. ner is widely used in downstream tasks, such as entity linking @xcite and relation extraction @xcite . previous works usually treat ner as a sequence labeling task, assigning a single tag to each token in a sentence. such models lack the ability to identify nested named entities. various approaches for nested ner have been proposed in recent years. some works revised sequence models to support nested entities using different strategies @xcite @xcite and some works adopt the hyper-graph to capture all possible entity mentions in a sentence @xcite . we focus on the span-based methods @xcite @xcite , which treat named entity recognition as a classification task on a span with the innate ability to recognize nested named entities. for example, @xcite exhausts all possible spans in a text sequence and then predicts their categories. however, these methods suffer from some serious weaknesses. first, due to numerous low-quality candidate spans, these methods require high computational costs. then, it is hard to identify long entities because the length of the span enumerated during training is not infinite. next, boundary information is not fully utilized, while it is important for the model to locate entities. although some methods @xcite have used a sequence labeling model to predict boundaries, yet without dynamic adjustment, the boundary information is not fully utilized. finally, the spans which partially match with entities are not effectively utilized. these methods simply treat the partially matched spans as negative examples, which can introduce noise into the model. different from the above studies, we observed that ner and object detection tasks in computer vision have a high degree of consistency. they both need to locate regions of interest (rois) in the context (image/text) and then assign corresponding categories to them. furthermore, both flat ner and nested ner have corresponding structures in the object detection task, as shown in figure 1 . for the flat structure, there is no overlap between entities or between objects. while for nested structures, finegrained entities are nested inside coarse-grained entities, and small objects are nested inside large objects correspondingly. in computer vision, the two-stage object detectors @xcite @xcite @xcite are the most popular object detection algorithm. they divide the detection task into two stages, first generating candidate regions, and then classifying and fine-tuning the positions of the candidate regions. inspired by these, we propose a two-stage entity identifier and treat ner as a joint task of boundary regression and span classification to address the weaknesses mentioned above. in the first stage, we design a span proposal module, which contains two components: a filter and a regressor. the filter divides the seed spans into contextual spans and span proposals, and filters out the former to reduce the candidate spans. the regressor locates entities by adjusting the boundaries of span proposals to improve the quality of candidate spans. then in the second stage, we use an entity classifier to label entity categories for the number-reduced and quality-improved span proposals. during training, to better utilize the spans that partially match with the entities, we construct soft examples by weighting the loss of the model based on the iou. in addition, we apply the soft non-maximum suppression (soft-nms) @xcite algorithm to entity decoding for dropping the false positives. our main contributions are as follow: ## model figure 2 illustrates an overview of the model structure. we first obtain the word representation through the encoder and generate seed spans. among these seed spans, some with higher overlap with the entities are the proposal spans, and others with lower overlap are the contextual spans. in the span proposal module, we use a filter to keep the proposal spans and drop the contextual spans. meanwhile, a regressor regresses the boundary of each span to locate the left and right boundaries of entities. next, we adjust the boundaries of the span proposals based on the output of the regressor, and then feed them into the entity classifier module. finally, the entity decoder decodes the entities using the soft-nms algorithm. we will cover our model in the following sections. ## case study examples of model predictions are shown in ## conclusion in this paper, we treat ner as a joint task of boundary regression and span classification and propose a two-stage entity identifier. first we generate span proposals through a filter and regressor, then classify them into the corresponding categories. our proposed model can make full use of the boundary information of entities and reduce the computational cost. moreover, by constructing soft samples during training, our model can exploit the spans that partially match with the entities. experiments illustrate that our method achieves state-of-the-art performance on several nested ner datasets. for future work, we will combine named entity recognition and object detection tasks, and try to use a unified framework to address joint identification on multimodal data. makoto miwa and mohit bansal. 2016. end-to-end relation extraction using lstms on sequences and tree structures. in proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1105-1116, berlin, germany. association for computational linguistics. aldrian obaja muis and wei lu. 2017. labeling gaps between words: recognizing overlapping mentions with mention separators. in proceedings of the 2017 conference on empirical methods in natural language processing, pages 2608-2618, copenhagen, denmark. association for computational linguistics. tomoko ohta, yuka tateisi, and jin-dong kim. 2002. the genia corpus: an annotated research abstract corpus in molecular biology domain. in proceedings of the second international conference on human language technology research, hlt '02, page 82-86, san francisco, ca, usa. morgan kaufmann publishers inc. nanyun peng and mark dredze. 2015. named entity recognition for chinese social media with jointly trained embeddings. in proceedings of the 2015 conference on empirical methods in natural language processing, pages 548-554, lisbon, portugal. association for computational linguistics.