A Boundary Offset Prediction Network for Named Entity Recognition (2023)

Abstract: Named entity recognition (NER) is a fundamental task in natural language processing that aims to identify and classify named entities in text. However, span-based methods for NER typically assign entity types to text spans, resulting in an imbalanced sample space and neglecting the connections between non-entity and entity spans. To address these issues, we propose a novel approach for NER, named the Boundary Offset Prediction Network (BOPN), which predicts the boundary offsets between candidate spans and their nearest entity spans. By leveraging the guiding semantics of boundary offsets, BOPN establishes connections between non-entity and entity spans, enabling non-entity spans to function as additional positive samples for entity detection. Furthermore, our method integrates entity type and span representations to generate type-aware boundary offsets instead of using entity types as detection targets. We conduct experiments on eight widely-used NER datasets, and the results demonstrate that our proposed BOPN outperforms previous state-of-the-art methods.

## introduction named entity recognition (ner) is a fundamental task in natural language processing (nlp) that involves identifying and categorizing named entities in text, such as people, locations and organizations. it has drawn much attention from the community due to its relevance in various nlp applications, such as entity linking @xcite and relation extraction @xcite . various paradigms have been proposed for ner, including the sequence labeling @xcite , hypergraph-based @xcite @xcite , sequence-to-sequence @xcite and span-based methods (sohrab figure 1 : a sentence from genia dataset @xcite , containing 8 words and 3 entities. the candidate spans cover the upper triangular region with a total of 36 samples of each matrix. there are 2 and 1 positive samples for "protein" and "cell type" entity types, respectively. @xcite @xcite . among these approaches, the span-based method has become the most popular due to its simplicity and effectiveness. it is straightforward that typically embeds all possible text spans and predicts their entity types, making it suitable for various ner subtasks @xcite . despite significant progress made by span-based methods in ner, there remain two critical issues that require attention. firstly, these methods often suffer from highly imbalanced sample spaces, as exemplified in figure 1 . such imbalance can negatively impact the trainability and performance of deep neural networks @xcite . although some methods @xcite mitigate this issue by restricting the maximum span length, such an approach can also constrain the model's predictive power. secondly, current span-based methods primarily focus on learning the distinction between non-entities and entities, disregarding their relationships. while a model can identify whether "hmg box" is an entity, it may fail to recognize the connection be- in this paper, we intend to model text spans by utilizing boundary offset information as supervision, rather than predict their probability of belonging to entities. as shown in figure 2 , there could be two advantages for deep models when boundary offsets are learnable: i) the natural quantitative relationships between offset values enable the model to capture boundary differences and connections simultaneously. ii) non-entity spans can have specific semantics that guide the positioning of entity spans, leading to an improved sample space with fewer negative samples. based on this observation, we propose the boundary offset prediction network (bopn) for ner. bopn focuses on predicting boundary offsets between candidate spans and their nearest entities, providing a new perspective on modeling text spans. specifically, our method follows the pipeline of first learning span representations and then classifying them for offset prediction. bert @xcite and bilstm @xcite are used to embed texts, followed by a conditional layer @xcite for building span representations. meanwhile, we also treat entity types as inputs rather than classification targets, which are fused with span representations to generate typeaware boundary offsets in parallel. finally, we incorporate multiple 3d convolution layers to capture the natural quantitative relationships between the offset values. we evaluate our method on eight widely-used ner datasets, including five english ner datasets and three chinese ner datasets. the experimental results demonstrate that our approach outperforms the existing state-of-the-art methods. furthermore, a detailed examination reveals a significant im-provement in recall scores when aggregating results across offset labels, which is particularly beneficial for recall-sensitive applications. ## problem definition named entity recognition (ner) aims to identify of all entities within an input sentence @xmath0, based on a pre-defined set of entity types @xmath1. typically, an entity is specified by token boundaries and a entity types. our proposed method focuses on predicting the boundary offset between each candidate text span and its nearest entity. hence, we formulate each text span as a quadruple: {x i , x j , f s , y m }, where i and j denote the start and end boundary indices of the span, f s represents the start or end boundary offset from its nearest entity of type y m . note that an entity span is a special case with f @xmath2. annotation guidelines to facilitate understanding, we present the essential boundary offset labels as follow: the annotation procedure for boundary offsets involves three steps. initially, a 3-dimensional matrix @xmath3, where m denotes the number of entity types and n represents the length of the sentence. next, we annotate the center spans with the offset label "0" based on the golden entities present in x. entities of different types are assigned to their respective sub-matrices. finally, for non-entity spans, we compute the start and end boundary offset values with respect to all center spans. their annotation is determined by the absolute minimum offset value. if the absolute minimum offset value is less than s, we annotate the corresponding *s or *e; otherwise, we label the span as "out-of-range". u bert lstm lstm lstm @xmath4... ... ... ... (a) span encoder entity type representations ffn u × × × × span representations prediction (b) boundary offset prediction biaffine classifier move 3d convolution layers dilation [1,2,3] linear + + + + + + ffn cln layer (0, 0, 0) (0, 0, 1) (0, 2, 4) (1, 6, 7) coordinate offset ... ... ... -1e 0 -1s 1e ... ... ... ... ## methods figure 3 provides an overview of our method, which encompasses two primary components: a span encoder (section 3.1) and a boundary offset predictor (section 3.2). the span encoder is responsible for encoding entity types and sentences, utilizing word representations to construct span representations. subsequently, the entity type and span representations are inputted into the boundary offset predictor, facilitating type-aware offset classification. ## related work in recent years, various paradigms for named entity recognition (ner) have been proposed, among which span-based methods have become one of the most mainstream approaches, treating ner as a text span classification problem. with the development of pre-trained language models, some works @xcite @xcite obtain span representations by connecting boundary representations or aggregating token representations and feeding them into a linear classifier for type prediction. alternatively, @xcite utilizes a biaffine classifier to fuse start and end boundary representations directly for span classification. to further enhance span representation, several other methods @xcite propose fusing representations of token, boundary, and related entity spans. meanwhile, some methods try to improve spanbased methods by adding boundary supervision. specifically, @xcite and @xcite additionally detect entity boundaries with multi-task learning, while @xcite perform boundary regression after span prediction. @xcite design two word-word relations for span classification. compared with previous methods, our proposed method utilizes continuous boundary offset values to model text spans, which can capture both the boundary differences and connections between non-entity and entity spans. in addition to span-based methods, there are three widely-used ner methods. the traditional sequence labeling methods @xcite assign each token a tag with a pre-designed tagging scheme (e.g., bio). to address nested entities, some works @xcite @xcite add struggles or design special tagging schemes. hypergraphbased methods @xcite @xcite represent the input sentence as a hypergraph for detecting nested entities, which must be carefully designed to avoid spurious structures. sequence-to-sequence methods reformulate ner as a sequence generation problem. for example, @xcite first apply the seq2seq model for ner, inputting the sentence and outputting start positions, entity lengths, and types. @xcite use the seq2seq model and enhanced bilou scheme to address nested ner. @xcite treats ner as an entity span sequence generation problem with pointer network based on bart @xcite . ## conclusion in this paper, we introduce a novel approach for named entity recognition (ner) called the boundary offset prediction network (bopn). bopn predicts the boundary offsets between candidate spans and their nearest entities, leveraging entity types as inputs. by incorporating entity types, bopn enables parallel prediction of type-aware boundary offsets, enhancing the model's ability to capture fine-grained entity boundaries. to capture the interactions between boundary offsets, we employ multiple 3d convolution layers, which refine the offset predictions and capture the inherent quantitative relationships between adjacent text spans. the experimental results demonstrate that our proposed method achieves state-of-the-art performance on eight widely-used datasets, including five english ner datasets and three chinese ner datasets. moreover, further analysis reveals a significant improvement in recall scores by utilizing boundary offset as supervision, showcasing the utility of our approach for recall-sensitive applications in ner.