Simple Yet Powerful: An Overlooked Architecture for Nested Named Entity Recognition (2022)

Abstract: Named Entity Recognition (NER) is an important task in Natural Language Processing that aims to identify text spans belonging to predefined categories. Traditional NER systems ignore nested entities, which are entities contained in other entity mentions. Although several methods have been proposed to address this case, most of them rely on complex task-specific structures and ignore potentially useful baselines for the task. We argue that this creates an overly optimistic impression of their performance. This paper revisits the Multiple LSTM-CRF (MLC) model, a simple, overlooked, yet powerful approach based on training independent sequence labeling models for each entity type. Extensive experiments with three nested NER corpora show that, regardless of the simplicity of this model, its performance is better or at least as well as more sophisticated methods. Furthermore, we show that the MLC architecture achieves state-of-the-art results in the Chilean Waiting List corpus by including pre-trained language models. In addition, we implemented an open-source library that computes task-specific metrics for nested NER. The results suggest that metrics used in previous work do not measure well the ability of a model to detect nested entities, while our metrics provide new evidence on how existing approaches handle the task.

## introduction named entity recognition (ner) is a widely studied task in natural language processing (nlp) that seeks to identify text spans expressing references to predefined categories such as person names, locations, and organizations @xcite . ner, or in general, the task of recognizing entity mentions 1 , has drawn the attention of the community due to its relevance in several nlp applications. nested named entity recognition is 1 mentions are defined as references to entities that could be named, nominal or pronominal @xcite . figure 1 : an example of a multi-label entity in the chilean waiting list corpus, followed by a nesting of different types. the annotation was translated from its original language. a particular case of ner where entities are nested within each other @xcite . traditional ner models simplify the nested entities by keeping the outermost entity and removing the inner ones. this simplified problem is better known as flat ner and is commonly regarded as a sequence labeling problem since each token can be associated with at most one label. however, removing part of these entities could be a problem in model performance due to losing relevant information and inner dependencies. several methods have been proposed to address the nesting problem. traditional approaches have focused on creating representations of nested entities through structures such as hypergraphs @xcite @xcite . however, they usually suffer from heavy feature engineering, structural ambiguity, or complex models. another category is region-based, which divides the problem into two sequential stages. first, the detection of entity boundaries, and then the assignment of entity types to these regions @xcite @xcite . one of the main drawbacks of this method is its high time complexity. there are also approaches that attempt to transform the nested ner task into a sequence labeling problem @xcite @xcite . although these studies have shown competitive performance, most of them have three critical issues discussed below. first, with the incorporation of large pre-trained language models, the standard lstm-crf @xcite sequence labeling architecture received substantial improvements for flat ner tasks @xcite . however, little research has been conducted on adapting this architecture to nested ner using a single entity approach, i.e., training independent flat ner models for each entity type. this paper revisits this architecture, naming it multiple lstm-crf (mlc). we show that this model yields very positive results despite the apparent simplicity, outperforming several recent approaches explicitly designed for nested entities. second, we note that most of the literature ignores the case in which the same text span is tagged with more than one entity type, as shown in figure 1 . this case is very common in the chilean waiting list corpus @xcite , and it was first noticed by @xcite but was not analyzed further. one of the main advantages of our architecture is that it addresses this problem. third, we argue that the way the community is evaluating this task does not adequately measure the effectiveness of a model at identifying nested entities. specifically, the current metric calculates the micro f1-score over all entities, the same metric used in flat ner. consequently, a model that performs well over flat entities but not nested ones may also obtain good results. to alleviate this problem, we first identify the different types of nesting by formalizing the task, and then we propose new task-specific metrics for these cases. in summary, the main contributions of our work are the following: ## related work in recent years there has been a growing interest from the research community in nested ner. several studies have been conducted to address nested entities, which can be mainly divided into three categories: region-based: these approaches divide the problem into two stages: identifying entity boundaries and then categorizing these regions. @xcite designed a model that enumerates all possible spans within a limited length and then used boundary and average internal token representation to predict entity types. another regionbased model was proposed by @xcite , which uses a sequence labeling layer to detect entity boundaries, and then classifies selected regions into their categorical types. @xcite used ideas from a biaffine model, scoring all possible start-end tokens in a sentence to predict nested entities. although these methods have proven to be effective, they often suffer from high time complexity and fail to identify entities tagged with more than one type. structure-based: there have also been attempts to capture the structure of nested entities. finkel and @xcite represented each input sentence as a constituency tree of nested entities and used a crf-based approach to predict entity types. @xcite proposed a mention hypergraph representation to extract entity mentions. next, muis and lu (2017) improved on previous work by modeling nested ner with mention separators and handcrafted features. similarly, katiyar and cardie (2018) designed a directed hypergraph using lstm features to learn the nesting structure. @xcite recursively introduce the embedding of tokens and regions into flat ner layers simulating the shape of a pyramid. however, these approaches usually suffer from spurious structures and structural ambiguities, as explained in @xcite . sequence labeling-based: some studies report that sequence labeling methods can also perform well on this task. early work mainly exploited the potential of conditional random fields (crf). @xcite proposed three crf-based methods to reduce the nested ner as several bio tagging problems. their best approach, called cascaded crf, uses one model per entity type by using the output of the previous flat ner model as a feature for the current one. @xcite took advantage of inner entity information to encourage outer entity recognition. they dynamically stacked lstm-crf layers predicting entities inside-to-outside until no entities were extracted. @xcite formulated the nested ner task as a sequenceto-sequence problem using an lstm to decode entity types. finally, using a recursive crf-based method, shibuya and hovy (2020) recognized entities iteratively from outermost ones to inner ones. the mlc approach falls into this category by using a sequence labeling approach capable of handling both nested entities and entities tagged with more than one label. ## experiments in this section, we present the datasets, baselines, and settings used in our experiments. ## conclusions and future work this paper presented an effective but overlooked neural model for nested ner based on sequence labeling architectures. specifically, we revisited the multiple lstm-crf (mlc) approach, which uses a single flat ner model per entity type. we argue that this approach has not been analyzed in-depth since large pre-trained language models have not been incorporated. our experimental results show that by adding a character-level language model to the mlc architecture, it achieves state-of-the-art in the chilean waiting list corpus. one of the main advantages of using this approach is that it can handle entities tagged with more than one entity type, barely addressed in previous works. in addition, to alleviate some gaps found in current evaluation metrics, we implemented an opensource library that computes task-specific metrics for nested ner. the results according to these metrics are low, especially when it comes to recognizing complete nestings, i.e., inner and outer entities simultaneously. this finding shows that most models are better at identifying flat entities or part of nested entities, which is not the primary goal of the task. we hope that our study will help raise awareness in the research community that overlooking intuitive models and using only standard metrics when evaluating a new complex solution can be misleading and create an overly optimistic impression of the new solution's performance. future directions include incorporating the hierarchical information of nested entities to improve the performance of our model. in addition, we plan to analyze two underexplored issues in the ner task: crossing and discontinuous entities. the first corresponds to cases where entities are not fully nested in other entities, but there is an overlap, and the second is when entities do not necessarily have consecutive tokens in the sentence.