HIT : Nested Named Entity Recognition via Head-Tail Pair and Token Interaction (2020)

Abstract: Named Entity Recognition (NER) is a fundamental task in natural language processing. In order to identify entities with nested structure, many sophisticated methods have been recently developed based on either the traditional sequence labeling approaches or directed hypergraph structures. Despite being successful, these methods often fall short in striking a good balance between the expression power for nested structure and the model complexity. To address this issue, we present a novel nested NER model named HIT. Our proposed HIT model leverages two key properties pertaining to the (nested) named entity, including (1) explicit boundary tokens and (2) tight internal connection between tokens within the boundary. Specifically, we design (1) Head-Tail Detector based on the multi-head self-attention mechanism and bi-affine classifier to detect boundary tokens, and (2) Token Interaction Tagger based on traditional sequence labeling approaches to characterize the internal token connection within the boundary. Experiments on three public NER datasets demonstrate that the proposed HIT achieves state-of-the-art performance.

## introduction named entity recognition (ner) is a fundamental task in natural language processing due to the fact that the named entities often convey the key information of the text @xcite . it is common in many practical scenarios that the named entities have a nested structure @xcite . that is, an entity could contain other entities or be a part of other entities. as shown in figure 1 , the entity "the western canadian province of british columbia" in the first example contains two inner entities, i.e., "western canadian" and "british columbia". traditional methods often treat the ner task as a sequence labeling problem @xcite and thus are primarily designed to recognize flat entities in the input sentences (as shown in the second example in figure 1 ). due to the nature of the nested entity, a token might belong to different entities. it is difficult to represent such nested structures using a single label accurately. therefore, the performance of traditional ner methods will dramatically suffer when recognizing nested entities @xcite . in recent years, more sophisticated methods have been developed for the nested ner task, which are grouped into two categories, including sequencebased method and hypergraph-based method. the sequence-based methods @xcite @xcite often utilize the traditional sequence labeling approaches to learn the nested structure. for example, @xcite leverage the hierarchical long short term memory (lstm) networks to capture the nested named entities from the inner entity to the outer entity. however, such methods might still suffer from error propagation due to the fundamental limitation of sequence labeling approaches in representing the nested structure. in response, hypergraph-based methods @xcite introduce the hypergraph structure for learning the nested named entity. these methods replace the undirected graph structure, commonly used in the flat ner task, by the directed hypergraph structure. the advantage lies in that hyperedges can naturally express the nested structure. one issue of their method @xcite is the spurious structure of hypergraphs. @xcite further propose the neural segmental hypergraphs to address this issue. however, if the input sentence is too long or there exist many entity categories, their hypergraph structure becomes too complicated, which in turn makes the optimization of such models very difficult, if not impossible. this paper further explores the precise expression of the nested structure with appropriate model complexity to overcome these shortcomings effectively. we observe two key properties pertaining to the named entity, including (1) explicit boundary tokens and (2) tight internal connection between tokens within the boundary. for example, in figure 1 , "premier" and "columbia" (in the first example) are explicit boundary tokens, and the tokens within the boundary are closely connected with each other. on the other hand, although the candidate region "premier visited province of british columbia" (in the second example) shared the same boundary tokens "premier" and "columbia", the tokens within the boundary suggest this region should not be an entity. this indicates that different internal tokens greatly influence whether the region determined by the boundary tokens is a valid entity. in other words, in the ner task, one region should be identified as a named entity if it meets these two properties. more importantly, these properties are sensitive to the entities with the nested structure. armed with these observations, we propose a novel neural model named hit for recognizing the named entities with the nested structure. our proposed model effectively identifies nested named entities by modeling both the boundary tokens (referred to as "head-tail pair" in this paper) and connection relationship between tokens within the boundary (referred to as "token interaction" in this paper). to be specific, we design a head-tail detector based on the multi-head self-attention mechanism @xcite and the bi-affine classifier @xcite to detect explicit boundary tokens. the main advantage of the multihead self-attention mechanism is that it can directly learn the connection between tokens without having to consider token ordering information. particularly, we adopt focal loss @xcite to address the class imbalance problem in the training process. this is because the head-tail detector aims to detect all candidates of head-tail pairs, only a few of which correspond to valid entities. in addition, we design a token interaction tagger based on traditional sequence labeling approaches @xcite to characterize the internal connection between tokens within the boundary through context. another advantage of the token interaction tagger is that the captured internal connection features contain abundant lexical and semantic information, which can be used to predict the category of entities. by integrating the head-tail detector and token interaction tagger, we apply the region classifier to predict the entity categories. extensive experiments on three public ner datasets, including genia (english) @xcite , germeval 2014 (german) @xcite , and jnlpba (english) @xcite , reveal that our proposed hit achieves state-of-theart performance. ## model in this section, we present the hit model in detail. figure 2 depicts the overall architecture of our model. the hit contains three main components, including the head-tail detector, token interaction tagger, and region classifier. for each given sentence @xmath0, w 2 , ..., w m }, where m is the length of the sentence, hit firstly maps the sentence x to a token representation sequence @xmath1, w 2 , ..., w m }. the representation sequence x is then fed into the head-tail detector to predict whether each pair-wise tokens is the head-tail of an entity. in the meanwhile, token interaction tagger is used to capture internal connections between adjacent tokens based on context, which indicates if the token before or after the current token belongs to an entity. finally, the region classifier is employed to integrate the head-tail detector and token interaction tagger to complete the entity recognition. in the following subsections, we will describe each part of our proposed hit in detail. ## experiments in this section, we first introduce the datasets, the baseline methods, and implementation details. we then present the experimental results used for evaluations, followed by analyzing two key properties and the ablation study of our hit model. ## related work many methods have been proposed for nested ner. early works on dealing with nested entities rely on hand-craft features or rule-based postprocessing @xcite . they use the supervised method that combines the hidden markov model with rule-based postprocessing to extract both the inner and outer entities. moreover, finkel and @xcite propose a chart-based parsing method for handling nested entities. they construct a discriminative constituency tree to represent each sentence, and each entity is represented as one of the subtrees. however, their method has a cubic time complexity. traditionally, the conventional ner is considered as a sequence labeling task. some studies reveal that sequence labeling-based methods can also perform well on the nested ner. muis and @xcite introduce a novel notion of mention separators that can effectively detect the nested entity mention. their method labels gaps between words to yield better performance, which relies on handcrafted features. @xcite propose dynamically stacking flat ner layers, while the number of stacked layers depends on the level of entity nesting. it can recognize entities sequentially from inner to outer. however, their method inevitably suffers from the error propagation since the outer entity detection overly depends on whether the inner entity is correctly recognized or not. zheng et al. (2019) propose a boundary-aware neural model that leverages entity boundaries to predict entity categorical labels. their method modifies the bieo (i.e., beginning, internal, end and other) hypothesis for detecting the boundary of nested entity. more recently, @xcite present a novel hypergraph-based method with linear time complexity to tackle the problem of nested entity mention detection. one issue in their approach is the spurious structures of the hypergraph. @xcite improve the method of @xcite by modeling arbitrary combinations of mentions with a segmental hypergraph. however, such an architecture leads to a higher time complexity during both training and decoding. @xcite propose a hypergraph-based representation based on the bilou tagging scheme. they treat the hypergraph construction procedure as a multi-label assignment process. ## conclusions in this paper, we propose a novel neural model hit for recognizing nested named entity. it leverages the head-tail pair and token interaction to express the entities with the nested structure. specifically, the head-tail detector can detect the head-tail pair of named entities. furthermore, the token interaction tagger captures the internal token connection within the boundary. experiments on three public datasets show that our model achieves significant improvements over the state-of-the-art models. for future work, we will apply hit to other languages, and further explore potential cases of overlapping entities in nested ner task.