Nested Named Entity Recognition via Explicitly Excluding the Influence of the Best Path (2021)

Abstract: This paper presents a novel method for nested named entity recognition. As a layered method, our method extends the prior second-best path recognition method by explicitly excluding the influence of the best path. Our method maintains a set of hidden states at each time step and selectively leverages them to build a different potential function for recognition at each level. In addition, we demonstrate that recognizing innermost entities first results in better performance than the conventional outermost entities first scheme. We provide extensive experimental results on ACE2004, ACE2005, and GENIA datasets to show the effectiveness and efficiency of our proposed method.

## introduction named entity recognition (ner), as a key technique in natural language processing, aims at detecting entities and assigning semantic category labels to them. early research @xcite @xcite proposed to employ deep learning methods and obtained significant performance improvements. however, most of them assume that the entities are not nested within other entities, so-called flat ner. inherently, these methods do not work satisfactorily when nested entities exist. figure 1 displays an example of the nested ner task. recently, a large number of papers proposed novel methods @xcite for the nested ner task. among them, layered methods solve this task through multi-level sequential labeling, in which entities are divided into several levels, where the term level indicates the depth of entity nesting, and sequential labeling is performed repeatedly. as a special case of layered method, shibuya and hovy (2020) force the next level entities to locate on the second-best path of the current level search space. hence, their algorithm can repeatedly detect inner entities through applying a conventional conditional random field (crf) @xcite and then exclude the obtained best paths from the search space. to accelerate computation, they also designed an algorithm to efficiently compute the partition function with the best path excluded. moreover, because they search the outermost entities first, performing the second-best path search only on the spans of extracted entities is sufficient, since inner entities can only exist within outer entities. however, we claim that the target path at the next level is neither necessary nor likely to be the second-best path at the current level. instead, those paths sharing many overlapping labels with the current best path are likely to be the second-best path. besides, @xcite reuse the same potential function at all higher levels. thus, even though they exclude the best path, the influence of the best path is still preserved, since the emission scores of labels on the best path are used in the next level recognition. moreover, these best path labels are treated as the target labels at the current level. however, if they are not on the best path of the next level, they will be treated as non-target labels at the next level, hence these adversarial optimization goals eventually hurt performance. in this paper, we use a different potential function at each level to solve this issue. we propose to achieve this by introducing an encoder that pro-duces a set of hidden states at each time step. at each level, we select some hidden states for entity recognition, then, remove these hidden states which have interaction with the best path labels before moving to the next level. in this way, the emission scores of these best path labels are completely different, so we can explicitly exclude the influence of the best path. furthermore, we also propose three different selection strategies for fully leveraging information among hidden states. besides, @xcite proposed to recognize entities from outermost to inner. we empirically demonstrate that extracting the innermost entities first results in better performance. this may due to the fact that some long entities do not contain any inner entity, so using outermostfirst encoding mixes these entities with other short entities at the same levels, therefore leading encoder representations to be dislocated. in this paper, we convert entities to the iobes encoding scheme @xcite , and solve nested ner through applying crf level by level. our contributions are considered as fourfold, (a) we design a novel nested ner algorithm to explicitly exclude the influence of the best path through using a different potential function at each level, (b) we propose three different selection strategies for fully utilizing information among hidden states, (c) we empirically demonstrate that recognizing entities from innermost to outer results in better performance, (d) and we provide extensive experimental results to demonstrate the effectiveness and efficiency of our proposed method on the ace2004, ace2005, and genia datasets. ## proposed method named entities recognition task aims to recognize entities in a given sequence {x t } n @xmath0. for nested ner some shorter entities may be nested within longer entities, while for flat ner there is no such case. existing algorithms solve flat ner by applying a sequential labeling method, which assigns each token a label y @xmath1. to solve nested ner, we follow the previous layered method and extend this sequential labeling method with a multi-level encoding scheme. in this encoding scheme, entities are divided into several levels according to their depths, we apply the sequential labeling method level by level to recognize all entities. ## related work existing ner algorithms commonly employ various neural networks to leverage more morphological and contextual information to improve performance. for example, to handle the out-ofvocabulary issue through introducing morphological features, huang et al. (2015) layered model as a layered model, @xcite dynamically update span-level representations for next layer recognition according to recognized inner entities. @xcite proposed a merge and label method to enhance this idea further. recently, shibuya and hovy (2020) designed a novel algorithm to efficiently learn and decode the second-best path on the span of detected entities. luo and zhao (2020) build two different graphs, one is the original token sequence, and the other is the tokens in recognized entities, to model the interaction among them. @xcite proposed to learn the l-gram representations at layer l through applying a decoder component to reduce a sentence layer by layer and to directly classify these l-gram spans. @xcite proposed an anchor-region network to recognize nested entities through detecting anchor words and entity boundaries first, and then classify each detected span. exhaustive models simply enumerate all possible spans and utilize a maximum entropy tagger @xcite and neural networks @xcite @xcite for classification. @xcite additionally aims to consider the relationship among entities and proposed a novel method to jointly learn both entities and relations. @xcite proposed a hyper-graph structure, in which edges are connected to multiple nodes to represents nested entities. muis and @xcite and @xcite resolved spurious structures and ambiguous issue of hyper-graph structure. and @xcite proposed another kind of hyper-graph structure. ## conclusion in this paper, we proposed a simple and effective method for nested named entity recognition by explicitly excluding the influence of the best path through selecting and removing chunks at each level to build different potential functions. we also proposed three different selection strategies to leverage information from all remaining chunks. besides, we found the innermost-first encoding scheme works better than the conventional outermost-first encoding scheme. extensive experimental results demonstrate the effectiveness and efficiency of our method. however, one of the demerits of our method is the number of chunks, i.e., the maximal depth of entity nesting, must be chosen in advance as a hyper-parameter. we will extend it to arbitrary depths as future work.