Modularized Interaction Network for Named Entity Recognition (2021)

Abstract: Although the existing Named Entity Recognition (NER) models have achieved promising performance, they suffer from certain drawbacks. The sequence labeling-based NER models do not perform well in recognizing long entities as they focus only on word-level information, while the segment-based NER models which focus on processing segment instead of single word are unable to capture the word-level dependencies within the segment. Moreover, as boundary detection and type prediction may cooperate with each other for the NER task, it is also important for the two sub-tasks to mutually reinforce each other by sharing their information. In this paper, we propose a novel Modularized Interaction Network (MIN) model which utilizes both segment-level information and word-level dependencies, and incorporates an interaction mechanism to support information sharing between boundary detection and type prediction to enhance the performance for the NER task. We have conducted extensive experiments based on three NER benchmark datasets. The performance results have shown that the proposed MIN model has outperformed the current state-of-the-art models.

## introduction named entity recognition (ner) is one of the fundamental tasks in natural language processing (nlp) that intends to find and classify the type of a named entity in text such as person (per), location (loc) or organization (org). it has been widely used for many downstream applications such as relation extraction @xcite , entity linking @xcite , question generation @xcite and coreference resolution @xcite . currently, there are two types of methods for the ner task. the first one is sequence labeling-based methods @xcite @xcite , in which each word in a sentence is assigned a special label (e.g., b-per or i-per). such methods can capture the dependencies between adjacent word-level labels and maximize the probability of predicted labels over the whole sentence. it has achieved the state-of-the-art performance in various datasets over the years. however, ner is a segment-level recognition task. as such, the sequence labeling-based models which focus only on word-level information do not perform well especially in recognizing long entities @xcite . recently, segment-based methods @xcite @xcite have gained popularity for the ner task. they process segment (i.e., a span of words) instead of single word as the basic unit and assign a special label (e.g., per, org or loc) to each segment. as these methods adopt segment-level processing, they are capable of recognizing long entities. however, the word-level dependencies within a segment are usually ignored. ner aims at detecting the entity boundaries and the type of a named entity in text. as such, the ner task generally contains two separate and independent sub-tasks on boundary detection and type prediction. however, from our experiments, we observe that the boundary detection and type prediction sub-tasks are actually correlated. in other words, the two sub-tasks can interact and mutually reinforce each other by sharing their information. consider the following example sentence: "emmy rossum was from new york university". if we know "university" is an entity boundary, it will be more accurate to predict the corresponding entity type to be "org". similarly, if we know an entity has an "org" type, it will be more accurate to predict that "university" is the end boundary of the entity "new york university" instead of "york" (which is the end boundary for the entity "new york"). however, sequence labeling-based models consider the boundary and type as labels, and thus such information cannot be shared between the subtasks to improve the accuracy. on the other hand, segment-based models first detect the segments and then classify them into the corresponding types. these methods generally cannot use entity type information in the process of segment detection and may have errors when passing such information from segment detection to segment classification. in this paper, we propose a modularized interaction network (min) model which consists of the ner module, boundary module, type module and interaction mechanism for the ner task. to tackle the issue on recognizing long entities in sequence labeling-based models and the issue of utilizing word-level dependencies within a segment in segment-based models, we incorporate a pointer network @xcite into the boundary module as the decoder to capture segment-level information on each word. then, these segmentlevel information and the corresponding word-level information on each word are concatenated as the input to the sequence labeling-based models. to enable interaction information, we propose to separate the ner task into the boundary detection and type prediction sub-tasks to enhance the performance of the two sub-tasks by sharing the information from each sub-task. specifically, we use two different encoders to extract their distinct contextual representations from the two sub-tasks and propose an interaction mechanism to mutually reinforce each other. finally, these information are fused into the ner module to enhance the performance. in addition, the ner module, boundary module and type module share the same word representations and we apply multitask training when training the proposed min model. in summary, the main contributions of this paper include: in this section, we review the related work on the current approaches for named entity recognition (ner). these approaches can be categorized into sequence labeling-based ner and segment-based ner. ## proposed model this section presents our proposed modularized interaction network (min) for ner. the overall model architecture is shown in figure 1 (a), which consists of the ner module, boundary module, type module and interaction mechanism. ## conclusion in this paper, we have proposed a novel modularized interaction network (min) model for the ner task. the proposed min model utilizes both segment-level information and word-level dependencies, and incorporates an interaction mechanism to support information sharing between boundary detection and type prediction to enhance the performance for the ner task. we have conducted extensive experiments on three ner benchmark datasets. the experimental results have shown that our proposed min model has achieved the state-ofthe-art performance.