Nested Named Entity Recognition with Span-level Graphs (2022)

Abstract: Span-based methods with the neural networks backbone have great potential for the nested named entity recognition (NER) problem. However, they face problems such as degenerating when positive instances and negative instances largely overlap. Besides, the generalization ability matters a lot in nested NER, as a large proportion of entities in the test set hardly appear in the training set. In this work, we try to improve the span representation by utilizing retrieval-based span-level graphs, connecting spans and entities in the training data based on n -gram features. Specifically, we build the entity-entity graph and span-entity graph globally based on n -gram similarity to integrate the information of similar neighbor entities into the span representation. To evaluate our method, we conduct experiments on three common nested NER datasets, ACE2004, ACE2005, and GENIA datasets. Experimental results show that our method achieves general improvements on all three benchmarks (+ 0.30 \sim 0.85 micro-F1), and obtains special superiority on low frequency entities (+ 0.56 \sim 2.08 recall).

## introduction named entity recognition is one of the major subtasks of information extraction for extracting categorized named entities from unstructured text. recently, neural-based ner architectures have shown remarkable performance with minimal feature engineering, such as cnn-crf @xcite , bilstm+crf @xcite , lstm-cnn-crf @xcite and lattice lstm @xcite . despite their great success, nested ner raises new challenges due to the deeply overlapping or nested entities @xcite . in nested ner, a token may be included in multiple entities (wang * corresponding author and @xcite instead of a single one in the conventional setting, making the problem more difficult to solve. previous exploration on nested ner can be mainly divided into three categories, using various architectures or different formulation for adaptation to the nested scenario. hypergraph-based methods use explicit hypergraphs to represent the possible nested structure or investigate the graph-shaped lexical/syntactic features @xcite @xcite . layered-based methods construct the nested structure through an action sequence @xcite @xcite or layered-models @xcite . span-based methods directly enumerate spans in a sentence, and perform categorical prediction on each span @xcite @xcite . span-based methods adopt the most simple and straightforward formulation as span classification, thus widely used and applied in joint relation extraction recently. despite the simplicity of span-based models, they can hardly fully utilize the rich semantics in spans. previous investigation has shown that span-based models are usually confused when the positive and negative instances are largely overlapped @xcite , as shown in fig. 1 . the minor differences between long entities and their similar spans can easily fool the span-based models. besides, most entities during inference statistically never appear in the training set in nested ner. for example, in ace2004, ace2005, and genia, there are 53.06%, 41.64% and 51.42% entity mentions from the validation set appear fewer than three times in the training set. learning powerful span representations for the prediction of those "unfamiliar" entities is difficult for conventional span-based models. in this work, we try to improve the span representation in span-based methods by utilizing retrievalbased span-level graphs. we seek helpful information in the training set beyond the current sentence. the intuitive assumption is that the entity spans similar to the candidate spans contain related information for discrimination on the candidate spans. we use n-gram similarity to measure the distance between spans. specifically, we treat each entity in the training set and each raw span as nodes, connecting those with high n-gram similarity. the constructed span-level heterogeneous graph records the lexical correlations among entities and various raw spans. we enhance the span representation by including the retrieved local subgraph for feature extraction of a specific span. we perform message passing with gcns @xcite on the retrieved subgraph for neighbor entities representation. the representation of neighbor entities provides rich correlations beyond the current sentence, thus improving the performance on confusing long spans and low-frequency spans. our main contributions are listed as follows: ## related work our work is closely related to nested ner and graphs used in ner. we introduce them accordingly as below. ## our approach in this part, we introduce the details of our approach. we first formulate the target problem, nested ner, as follows. nested ner as span classification following eberts and ulges (2020), we formulate the nested ner problem as the span classification task. the span classification task treats multiple adjacent tokens as a span and predicts the corresponding label. specifically, for a sentence @xmath0, . . . , x n } of n tokens, we extract all spans (with @xmath1, where s ij indicates the span from x i to x j . we predict the corresponding label of s ij as one of the pre-defined entity types or na (not an entity). the formulation as span classification instead of sequence labeling is more suitable for ner in nested scenarios but brings two challenges. the insensitivity to boundaries make the long candidate spans hard to identify. the low-frequency spans, taking the majority in data (fig. 2 ), also increase the difficulty on capturing intra-span representations. we use span-level graphs connecting entities and raw spans to tackle the problems. the overview of our model architecture is shown in fig. 3 . we construct the span-level graph with n-gram similarity to determine the adjacency (sec. 3.1). we initialize the representation of spans and entities with the encoder described in sec. 3.2. the structured correlations among entities and spans are modeled with gcn (sec. 3.3). we incorporate in training the entity categorical prediction to utilize the label of entity mentions on the graph (sec. 3.4). ## experiments in this section, we evaluate our method on three common nested ner datasets, including ace2004, ace2005, and genia. ## conclusions in this work, we enhance the span-based method for nested ner by including retrieval-based spanlevel graphs. our method builds the entity-entity graph and the span-entity graph globally based on n-gram feature similarity. we use gcn to encode such structured correlations to obtain better span representation. we include multi-task learning to encode the label information of similar entities in the graph. the experimental results on three commonly used nested ner datasets, i.e. ace2004, ace2005 and genia, show that our method can improve the f1 score generally and improve recall for entities with low frequency in the training set. graph hyper-parameters we explore two graph hyper-parameters edge threshold τ and gcn layers k in ace2004/ace2005. in fig. 6 (a), we set τ ∈ {0.0, , 0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4} and τ around 1.0 achieves the best performance. large edge threshold influences the performance slightly while small τ degenerates the performance a lot by introducing much noise into the graph. in fig. 6 (b), we set gcn layers @xmath2, 2, 3, 4} and find that 2-layer gcn with 2-hop span-level graph performs better. when @xmath3, our method degrades to using only a span-entity graph and the