Extract-Select: A Span Selection Framework for Nested Named Entity Recognition with Generative Adversarial Training (2022)

Abstract: Nested named entity recognition (NER) is a task in which named entities may overlap with each other. Span-based approaches regard nested NER as a two-stage span enumeration and classification task, thus having the innate ability to handle this task. However, they face the problems of error propagation, ignorance of span boundary, difficulty in long entity recognition and requirement on large-scale annotated data. In this paper, we propose Extract-Select, a span selection framework for nested NER, to tackle these problems. Firstly, we introduce a span selection framework in which nested entities with different input categories would be separately extracted by the extractor, thus naturally avoiding error propagation in two-stage span-based approaches. In the inference phase, the trained extractor selects final results specific to the given entity category. Secondly, we propose a hybrid selection strategy in the extractor, which not only makes full use of span boundary but also improves the ability of long entity recognition. Thirdly, we design a discriminator to evaluate the extraction result, and train both extractor and discriminator with generative adversarial training (GAT). The use of GAT greatly alleviates the stress on the dataset size. Experimental results on four benchmark datasets demonstrate that Extract-Select outperforms competitive nested NER models, obtaining state-of-the-art results. The proposed model also performs well when less labeled data are given, proving the effectiveness of GAT.

## introduction named entity recognition (ner) aims at detecting the spans and semantic categories of entities from the text. previous studies usually treat ner as a sequential labeling problem @xcite . these studies restrict each token belonging to at most one entity mention, and hence it is unable to handle nested ner @xcite , where one token may belong to multiple mentions. for example in figure 1 , a loc (i.e., location) entity "western canadian" is nested in another gpe (i.e., geo-political entity) entity "the western canadian province of british columbia". some studies seek to reconcile sequential labeling with nested ner @xcite . however, sequential labeling is naturally unsuitable for assigning multiple labels to a single token. considering that, some studies turn to adopt the two-stage framework, including transition-based approaches @xcite , hypergraph-based approaches @xcite @xcite and span-based approaches @xcite @xcite . among them, span-based approaches handle nested ner by extracting possible spans and classifying their categories. although these approaches have the innate ability to cope with this task, they have the following problems: (1) span-based approaches follow the two-stage framework, which inevitably has the problem of error propagation; (2) these approaches usually rely on span content for classification. however, span boundary information is not fully utilized, which is important for precise entity span extraction; (3) it is difficult for spanbased approaches to recognize long entities because the span length in the span extraction phase is limited; (4) these approaches usually rely heavily on large size of training data for obtaining competitive results. in this paper, we propose a novel nested ner approach, named extract-select, which is able to cope with the above shortcomings. specifically, the proposed extract-select works as follows. firstly, an extractor is proposed to extract all possible entity spans specific to a particular typed entity marker, with a novel hybrid selection strategy. then, a discriminator is introduced to evaluate and score entity span candidates predicted by the extractor. the extractor and the discriminator are iteratively trained with generative adversarial training (gat). in the inference phase, the iteratively trained extractor selects final entity spans of the given entity marker from the contexts. extract-select solves the above shortcomings from the following three aspects: to evaluate our extract-select, we conduct experiments on four standard nested ner benchmarks, including ace04, ace05, kbp17 and genia datasets. experimental results show that our model can effectively detect nested entities and achieve state-of-the-art results on the above four datasets. the ablation study on entity marker, gat and hybrid selection strategy reveals that these components are indispensable and all of them contribute to our model. moreover, our model only requires half amount of labeled data to achieve the same performance as baselines, indicting the effectiveness of our approach with less training data. transition-based approaches model nested structure through state transition and construct nested entities through actions. @xcite introduce a scalable transition-based model. @xcite propose an anchor-region architecture which models the head-driven phrase structures. however, these approaches rely heavily on hand-crafted features. hypergraph-based approaches construct hypergraphs by the structure of nested ner and decode results on hypergraphs. muis and lu (2017) introduce a mention hypergraph for nested ner. @xcite propose a hypergraph representation, which is free from structural ambiguity. @xcite propose to capture bidirectional information interactions between hypergraph layers. however, these hypergraphs should be well designed to prevent ambiguous structure. span-based approaches extract entity spans and then classify their categories. @xcite select the most confident entity spans for classification. @xcite propose to merge entities and tokens into entities, and then assign labels. @xcite regards this task as an object detection task, locating and then labeling spans. nevertheless, these two-stage approaches have the problem of ignorance of span boundary, difficulty in long entity recognition and error propagation. @xcite formalizes ner as a machine reading comprehension (mrc) task, which uses bert as backbone and extracts spans of given queries. however, this work relies heavily on training data size. besides, it extracts entity spans based on boundary information but ignores content information. in contrast to their work, we use gat to iteratively train extractor to get better results, and adopt hybrid selection strategy to make full use of both boundary and content information. generative adversarial training (gat) gives a way to learn deep representations without extensively labeled data. it is proposed by @xcite and is characterized by training a generator and a discriminator in competition with each other. gat has been applied in different nlp subtasks, including dialogue generation @xcite and relation extraction @xcite . in these studies, gat proves to be effective in reducing the usage of training data. motivated by these work, we propose to apply gat in ner task to reduce the demand on labeled data. ## problem definition the input of the span selection framework is a sequence @xmath0, x 2 , ..., x |x| }, where |x| denotes the length of the sequence. the possible entity span x s,@xmath1, x s+1 , ..., x e-1 , x e } is a continuous sub-string of x satisfying @xmath2. let y denote the predefined list of all entity categories and y * be the entity marker specific to type @xmath3.g., "loc" has an entity marker "location"). the aim of span selection framework is to find all entities in x for each category y. ## conclusion this paper proposes extract-select, a span selection framework to solve nested ner. it contains an extractor which aims to extract entities specific to a particular entity category with a hybrid selection strategy, and a discriminator scoring the extractor. the extractor and discriminator are trained with gat to reduce the demand on labeled data. comprehensive experiments performed on four widely used nested ner datasets demonstrate the superiority of extract-select. in future, we will (1) attempt to overcome the deficiency issue and (2) investigate in discontinuous and joint ner.