D i F i N et: Boundary-Aware Semantic Differentiation and Filtration Network for Nested Named Entity Recognition (2024)

Abstract: Nested Named Entity Recognition (Nested NER) entails identifying and classifying entity spans within the text, including the detection of named entities that are embedded within external entities. Prior approaches primarily employ span-based techniques, utilizing the power of exhaustive searches to address the challenge of overlapping entities. Nonetheless, these methods often grapple with the absence of explicit guidance for boundary detection, resulting insensitivity in discerning minor variations within nested spans. To this end, we propose a Boundary-aware Semantic \underline{Di} fferentiation and \underline{Fi} ltration \underline{Net} work (DiFiNet) tailored for nested NER. Specifically, DiFiNet leverages a biaffine attention mechanism to generate a span representation matrix. This matrix undergoes further refinement through a self-adaptive semantic differentiation module, specifically engineered to discern semantic variances across spans. Furthermore, DiFiNet integrates a boundary filtration module, designed to mitigate the impact of non-entity noise by leveraging semantic relations among spans. Extensive experiments on three benchmark datasets demonstrate our model yields a new state-of-the-art performance.

## introduction named entity recognition (ner) involves the utilization of computer-assisted techniques to identify and extract entities and corresponding semantic types @xcite , including person (per), location (loc), geo-political entity (gpe), and others. ner plays a crucial role in facilitating various downstream tasks such as relation @xcite , event extraction @xcite , and sentiment analysis @xcite . conventional approaches have primarily focused on identifying non-nested entities @xcite @xcite , a trend largely attributed to the constraints of corpus annotations that emphasize flat entity structures. however, the complex nature of natural language frequently features nested named entities, with studies revealing that approximately 30% of sentences in ace04 and ace05 datasets contain such structures @xcite . the prevalence of nested structures underscores the need for efficient models adept at handling such linguistic complexities. in response to this challenge, recent years have witnessed a burgeoning interest in nested ner @xcite @xcite . among the emerging strategies, span-based models stand out as prominent approaches and have set new benchmarks in the field @xcite @xcite . these models excel by leveraging exhaustive search techniques to systematically identify all possible spans, thereby capturing the full spectrum of nested structures. despite the success of span-based methods, they often struggle to fully utilize the rich semantics within spans due to the absence of explicit guidance for boundary detection. previous research indicates that span-based models usually encounter confusion when dealing with nested entities characterized by a high degree of token overlap @xcite @xcite . to illustrate, consider the sentence taken from the ace05 dataset in figure 1 , entities like "the alabama-florida line", "florida", and "alabama", as well as non-entity spans such as "alabama-florida line" or "the alabama-florida" share a significant number of tokens, blurring the semantic distinction between entity and non-entity spans. besides, those low-quality candidate spans, particularly long entities, incur significant computational costs @xcite due to the extensive array of potential spans evaluated during training, inevitably limited by practical constraints. to address this issue, we propose a boundaryaware semantic differentiation and filtration network (difinet) explicitly incorporating semantic differences between nested spans as input features. by leveraging gradient back-propagation, difinet learns appropriate internal representations to augment the distinction among nested entities within the span semantic representation space, enhancing its ability to discern boundaries and accurately classify nested named entities. specifically, difinet integrates bert and a biaffine attention mechanism to construct a matrix of span semantic representations, followed by a self-adaptive semantic differentiation module to transform span representations into semantic differences across spans. additionally, to alleviate the influence of low-quality candidate spans within the matrix, difinet integrates a boundary filtration module. this module serves to model the interaction among spans, effectively reducing noise, with a specific focus on distinguishing semantically similar entity and non-entity spans. our main contributions are summarized as follows: ## related work nested named entity recognition is a task in natural language processing (nlp) that involves identifying and classifying named entities within text data, where entities can have complex and overlapping structures. one approach to tackle this task is the hypergraph method, originally proposed by @xcite . this method maps the nested entity structures to sub-graphs in a hyper-graph and performs classification on them. several extensions have been developed based on this method (muis and @xcite . another approach is the hierarchical method introduced by @xcite , which divides entities into different levels, where each deeper level represents a higher level of entity specificity. following this paradigm, wang et al. (2020) designed a pyramid sequence labeling framework using convolutional neural networks to extract entities from bottom to top. shibuya and hovy (2020) explored suboptimal path decoding to progressively extract entities hierarchically, and wang et al. (2021) further improved it by excluding the influence of the optimal path. however, both hyper-graph and hierarchical methods suffer from high complexity when dealing with complex nested entities. in contrast, seq2seq methods offer a simpler end-to-end approach, typically utilizing lstm-crf @xcite or bart @xcite to predict the label of each position. @xcite improved seq2seq methods by adopting intra-entity and inter-entity de-confounding data augmentation techniques. @xcite designed a dual-slot multi-prompt template with a position slot for locating and a type slot for typing, respectively. nevertheless, when faced with highly complex nesting structures, these methods may encounter long-distance dependency problems, resulting in cascading errors. to address the aforementioned challenges in nested ner, @xcite proposed a span-based method that treats the nested ner task as span prediction problems. this approach involves predicting potential entity spans for each token, followed by filtering and merging these spans to obtain the final nested entities. building upon sohrab's work, several works have made advancements to the span-based method by incorporating graph structure @xcite , valuable span patterns @xcite and attention mechanism @xcite @xcite to achieve state-of-the-art performance. for example, @xcite proposed a biaffine attention mechanism to enhance the interaction between the start and end tokens and assigned scores to each span. when constructing the span-based contrastive loss function, @xcite utilizes concatenation to generate span representations. @xcite redefined ner by modeling it as a boundary-denoising diffusion process. this approach generates named entities by refining and clarifying noisy spans. however, span-based models typically utilize pooling @xcite @xcite , concatenation @xcite @xcite or integration (zhu and @xcite @xcite techniques to generate span representations from token representations. however, this approach often leads to generating semantically similar representations for highly overlapping spans. as a result, effectively capturing the subtle semantic nuances within individual spans becomes challenging. to mitigate the boundary insensitivity issue, we propose to explicitly incorporate span semantic difference features into nested ner task. this allows the model to learn more robust span representations by capturing the nuanced semantic variations between entity and non-entity spans. ## our approach in this section, we introduce the details of our framework as shown in figure 2 . we first formulate the task definition of nested ner as follow, nested ner as boundary detection in the context of nested ner, the task involves analyzing an input sentence denoted as @xmath0, x 2 , . . . , x n } to identify and classify potential entities according to a predefined set of entity types @xmath1, t 2 , . . . , t k }. typically, an entity can be represented by a triplet (s i , e i , t i ), where s i and e i denote the starting and ending position of the entity, respectively, and t @xmath2. this structured representation allows for the precise localization of entity boundaries within the sentence. in a sentence with n tokens, there are a total of n(n + 1)/2 valid spans. ## case study ## conclusion this paper proposes a boundary-aware semantic differentiation and filtration network (difinet) to effectively address the issue of boundary insensitivity in nested named entity recognition tasks. difinet introduces the self-adaptive semantic differentiation module to capture semantic difference information between spans and incorporates the boundary filtration module to reduce noise from non-entity spans and enhance the differences of boundary semantics between spans. experimental results demonstrate that difinet achieves superior performance compared to existing approaches on three benchmark datasets. ablation experiments and case studies further validate the effectiveness of the proposed model. looking ahead, we aim to extend utilization of boundary information in tasks such as event extraction and relation extraction.