Bipartite Flat-Graph Network for Nested Named Entity Recognition (2020)

Abstract: In this paper, we propose a novel bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER), which contains two subgraph modules: a flat NER module for outermost entities and a graph module for all the entities located in inner layers. Bidirectional LSTM (BiLSTM) and graph convolutional network (GCN) are adopted to jointly learn flat entities and their inner dependencies. Different from previous models, which only consider the unidirectional delivery of information from innermost layers to outer ones (or outside-to-inside), our model effectively captures the bidirectional interaction between them. We first use the entities recognized by the flat NER module to construct an entity graph, which is fed to the next graph module. The richer representation learned from graph module carries the dependencies of inner entities and can be exploited to improve outermost entity predictions. Experimental results on three standard nested NER datasets demonstrate that our BiFlaG outperforms previous state-of-the-art models.

## introduction named entity recognition (ner) aims to identify words or phrases that contain the names of predefined categories like location, organization or medical codes. nested ner further deals with entities that can be nested with each other, such as the united states and third president of the united states shown in figure 1 , such phenomenon is quite common in natural language processing (nlp). ner is commonly regarded as a sequence labeling task @xcite ; figure 1 : an example of nested named entity mentions. solid lines connect the starting and ending indices of inner nested entities. @xcite . these approaches only work for non-nested entities (or flat entities), but neglect nested entities. there have been efforts to deal with the nested structure. @xcite introduced a layered sequence labeling model to first recognize innermost entities, and then feed them into the next layer to extract outer entities. however, this model suffers from obvious error propagation. the wrong entities extracted by the previous layer will affect the performance of the next layer. also, such layered model suffers from the sparsity of entities at high levels. for instance, in the well-known ace2005 training dataset, there are only two entities in the sixth level. sohrab and miwa 2018 proposed a region-based method that enumerates all possible regions and classifies their entity types. however, this model may ignore explicit boundary information. @xcite combined the layered sequence labeling model and region-based method to locate the entity boundary first, and then utilized the region classification model to predict entities. this model, however, cares less interaction among entities located in outer and inner layers. in this paper, we propose a bipartite flat-graph network (biflag) for nested ner, which models a nested structure containing arbitrary many layers into two parts: outermost entities and inner entities in all remaining layers. for example, as shown in figure 1 , the outermost entity thomas jefferson, third president of the united states is considered as a flat (non-nested) entity, while third president of the united states (in the second layer) and the united states (in the third layer) are taken as inner entities. the outermost entities with the maximum coverage are usually identified in the flat ner module, which commonly adopts a sequence labeling model. all the inner entities are extracted through the graph module, which iteratively propagates information between the start and end nodes of a span using graph convolutional network (gcn) @xcite . the benefits of our model are twofold: (1) different from layered models such as @xcite , which suffers from the constraints of one-way propagation of information from lower to higher layers, our model fully captures the interaction between outermost and inner layers in a bidirectional way. entities extracted from the flat module are used to construct entity graph for the graph module. then, new representations learned from graph module are fed back to the flat module to improve outermost entity predictions. also, merging all the entities located in inner layers into a graph module can effectively alleviate the sparsity of entities in high levels. (2) compared with region-based models @xcite , our model makes full use of the sequence information of outermost entities, which take a large proportion in the corpus. the main contributions of this paper can be summarized as follows: ## model our biflag includes two subgraph modules, a flat ner module and a graph module to learn outermost and inner entities, respectively. figure 2 illustrates the overview of our model. for the flat module, we adopt bilstm-crf to extract flat (outermost) entities, and use them to construct the entity graph g 1 as in figure 2 . for the graph module, we use gcn which iteratively propagates information between the start and end nodes of potential entities to learn inner entities. finally, the learned representation from the graph module is further fed back to the flat module for better outermost predictions. ## case study in this example, entities my, my town, that and krispy kreme are nested in the entity the location in my town that was recently abandoned by krispy kreme. our biflag model successfully inference speed (t/s) 0 1000 2000 3000 4000 5000 6000 7000 6708 4751 2851 3563 biflag (ours) zheng et al., 2019 sohrab and miwa, 2018 ju et al., 2018 figure 3: the inference speed of our biflag and compared models on genia test set. t/s indicates token per second. extracts all these entities with exact boundaries and entity categorical labels. without graph construction, nested entities my town, that and krispy kreme are not identified. without interaction between the two modules, the outermost entity the location in my town that was recently abandoned by krispy kreme is mislabeled as loc (location), which is actually a fac (facility) type, inner nested entities my, my town and krispy kreme are not propagated back to the flat module, which maybe helpful to correct the extracting of the outermost entity. ## related work recently, with the development of deep neural network in a wide range of nlp tasks @xcite @xcite @xcite @xcite @xcite zhao, setence interesting aside: starbucks is taking over the location in my town that was recently abandoned by krispy kreme. gold label org: {starbucks, krispy kreme}; fac: {the location in my town that was recently abandoned by krispy kreme; that}; gpe: {my town}; per: {my} no graph org: {starbucks}; loc: {the location in my town that was recently abandoned by krispy kreme}; per: {my} no interaction org: {starbucks, krispy kreme}; loc: {the location in my town that was recently abandoned by krispy kreme}; gpe: {my town}; per: {my} biflag org: {starbucks, krispy kreme }; fac: {the location in my town that was recently abandoned by krispy kreme; that}; gpe: {my town}; per: {my} stacking sequence labeling models to extract entities from inner to outer (or outside-to-inside) can also handle such nested structures. @xcite propose several different modeling techniques (layering and cascading) to combine multiple crfs for nested ner. however, their approach cannot handle nested entities of the same entity type. @xcite dynamically stack flat ner layers, and recognize entities from innermost layer to outer ones. their approach can deal with nested entities of the same type, but suffers from error propagation among layers. region-based approaches are also commonly used for nested ner by extracting the subsequences in sentences and classifying their types. @xcite introduce a neural ex-haustive model that considers all possible spans and classify their types. this work is further improved by @xcite , which first apply a single-layer sequence labeling model to identify the boundaries of potential entities using context information, and then classify these boundary-aware regions into their entity type or non-entity. @xcite propose a sequence-to-nuggets approach named as anchor-region networks (arns) to detect nested entity mentions. they first use an anchor detector to detect the anchor words of entity mentions and then apply a region recognizer to identity the mention boundaries centering at each anchor word. @xcite decompose nested ner into two stages. tokens are merged into entities through real-valued decisions, and then the entity embeddings are used to label the entities identified. ## conclusion this paper proposes a new bipartite flat-graph (bi-flag) model for nested ner which consists of two interacting subgraph modules. applying the divideand-conquer policy, the flat module is in charge of outermost entities, while the graph module focuses on inner entities. our biflag model also facilitates a full bidirectional interaction between the two modules, which let the nested ne structures jointly learned at most degree. as a general model, our biflag model can also handle non-nested structures by simply removing the graph module. in terms of the same strict setting, empirical results show that our model generally outperforms previous state-of-the-art models.