A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition (2021)

Abstract: Research on overlapped and discontinuous named entity recognition (NER) has received increasing attention. The majority of previous work focuses on either overlapped or discontinuous entities. In this paper, we propose a novel span-based model that can recognize both overlapped and discontinuous entities jointly. The model includes two major steps. First, entity fragments are recognized by traversing over all possible text spans, thus, overlapped entities can be recognized. Second, we perform relation classification to judge whether a given pair of entity fragments to be overlapping or succession. In this way, we can recognize not only discontinuous entities, and meanwhile doubly check the overlapped entities. As a whole, our model can be regarded as a relation extraction paradigm essentially. Experimental results on multiple benchmark datasets (i.e., CLEF, GENIA and ACE05) show that our model is highly competitive for overlapped and discontinuous NER.

## introduction named entity recognition (ner) @xcite is one fundamental task for natural language processing (nlp), due to its wide application in information extraction and data mining @xcite . traditionally, ner is presented as a sequence labeling problem and widely solved by conditional random field (crf) based models @xcite . however, this framework is difficult to handle overlapped and discontinuous entities @xcite , which we illustrate using two examples as shown in figure 1 . the two entities "pennsylvania" and "pennsylvania radio station" are nested with each other, 1 and the sec-ond example shows a discontinuous entity "mitral leaflets thickened" involving three fragments. there have been several studies to investigate overlapped or discontinuous entities @xcite @xcite @xcite @xcite @xcite . the majority of them focus on overlapped ner, with only several exceptions to the best of our knowledge. muis and @xcite present a hypergraph model that is capable of handling both overlapped and discontinuous entities. wang and @xcite extend the hypergraph model with long short-term memories (lstms) @xcite . @xcite proposed a transition-based neural model for discontinuous ner. by using these models, ner could be conducted universally without any assumption to exclude overlapped or discontinuous entities, which could be more practical in real applications. the hypergraph @xcite and transition-based models @xcite are flexible to be adapted for different tasks, achieving great successes for overlapped or discontinuous ner. however, these models need to manually define graph nodes, edges and transition actions. moreover, these models build graphs or generate transitions along the words in the sentences gradually, which may lead to error propagation @xcite . in contrast, the spanbased scheme might be a good alternative, which is much simpler including only span-level classification. thus, it needs less manual intervention and meanwhile span-level classification can be fully parallelized without error propagation. recently, @xcite utilized the span-based model for information extraction effectively. in this work, we propose a novel span-based joint model to recognize overlapped and discon- at issue is the liability of a [[pennsylvania] 1 radio station] 2 under the federal wiretap statute. example 1 the [mitral] 1 valve [leaflets] 1 are mildly [thickened] 1 . sequence labeling model our proposed model the/o mitral/b valve/o leaflets/b are/o mildly/o thickened/b ./o at/o issue/o is/o the/o liability/o of/o a/o pennsylvania/b radio/i station/i under/o the/o federal/o wiretap/o statute/o ./o pennsylvania pennsylvania radio station entity fragment relation graph mitral leaflets thickened entity fragment relation graph sequence labeling model our proposed model example 2 tinuous entities simultaneously in an end-to-end way. the model utilizes bert @xcite to produce deep contextualized word representations, and then enumerates all candidate text spans @xcite , classifying whether they are entity fragments. following, fragment relations are predicted by another classifier to determine whether two specific fragments involve a certain relation. we define two relations for our goal: overlapping or succession, which are used for overlapped and discontinuous entities, respectively. in essence, the joint model can be regarded as one kind of relation extraction models, which is adapted for our goal. to enhance our model, we utilize the syntax information as well by using a dependency-guided graph convolutional network @xcite @xcite . we evaluate our proposed model on several benchmark datasets which includes both overlapped and discontinuous entities (e.g., clef @xcite ). the results show that our model outperforms the hypergraph (muis and @xcite and transition-based models @xcite . besides, we conduct experiments on two benchmark datasets including only overlapped entities (i.e., genia @xcite and ace05 ). experimental results show that our model can also obtain comparable performances with the state-of-the-art models @xcite @xcite . in addition, we observe that our approaches for model enhancement are effective in the benchmark datasets. our code is available at https://github.com/foxlf823/sodner . ## related work in the nlp domain, ner is usually considered as a sequence labeling problem @xcite @xcite . with well-designed features, crf-based models have achieved the leading performance @xcite @xcite . recently, neural network models have been exploited for feature representations @xcite . moreover, contextualized word representations such as elmo @xcite , flair @xcite and bert @xcite have also achieved great success. as for ner, the end-to-end bi-directional lstm crf models @xcite @xcite is one representative architecture. these models are only capable of recognizing regular named entities. for overlapped ner, the earliest model to our knowledge is proposed by @xcite , where they convert overlapped ner as a parsing task. @xcite propose a hypergraph model to recognize overlapped entities and lead to a number of extensions (muis and @xcite @xcite . moreover, recurrent neural networks (rnns) are also used for overlapped ner @xcite . other approaches include multi-grained detection @xcite , boundary detection @xcite , anchorregion network @xcite and machine reading comprehension @xcite . the stateof-the-art models for overlapped ner include the sequence-to-sequence (seq2seq) model @xcite , where the decoder predicts multiple input word rep graph convolutional network ℎ " ℎ # ℎ @xmath0 % &'( # &'( @xmath1,@xmath2,# … ) @xmath3 2 3 … … … mitral leaflets thickened ## method the key idea of our model includes two mechanisms. first, our model enumerates all possible text spans in a sentence and then exploits a multiclassification strategy to determine whether one span is an entity fragment as well as the entity type. based on this mechanism, overlapped entities could be recognized. second, our model performs pairwise relation classifications over all entity fragments to recognize their relationships. we define three kinds of relation types: with the succession relation, we can recognize discontinuous entities. ## conclusion in this work, we proposed an efficient and effective model to recognize both overlapped and discontinuous entities simultaneously, which can be applied to any ner dataset theoretically, since no extra assumption is required to limit the type of named entities. first, we enumerate all spans in a given sentence to determine whether they are valid entity fragments, and then relation classifications are performed to check the relationships between all fragment pairs. the results show that our model is highly competitive to the state-of-the-art models for overlapped or discontinuous ner. we have conducted detailed studies to help comprehensive understanding of our model.