A Boundary Offset Prediction Network for Named Entity Recognition
Authors: Minghao Tang
Yongquan He
Yongxiu Xu
Hongbo Xu
Wenyuan Zhang
Yang Lin
Venue: EMNLP 2023 Findings (2023)
Keywords: named entity recognition
span-based methods
boundary connections
boundary offset prediction network
type-related boundary offsets
TL;DR: This paper presents the Boundary Offset Prediction Network (BOPN), a novel approach for NER that predicts the boundary offsets between candidate spans and their nearest entity spans.

Abstract: Named entity recognition (NER) is a fundamental task in natural language processing that aims to identify and classify named entities in text. However, span-based methods for NER typically assign entity types to text spans, resulting in an imbalanced sample space and neglecting the connections between non-entity and entity spans. To address these issues, we propose a novel approach for NER, named the Boundary Offset Prediction Network (BOPN), which predicts the boundary offsets between candidate spans and their nearest entity spans. By leveraging the guiding semantics of boundary offsets, BOPN establishes connections between non-entity and entity spans, enabling non-entity spans to function as additional positive samples for entity detection. Furthermore, our method integrates entity type and span representations to generate type-aware boundary offsets instead of using entity types as detection targets. We conduct experiments on eight widely-used NER datasets, and the results demonstrate that our proposed BOPN outperforms previous state-of-the-art methods.

A Boundary Offset Prediction Network for Named Entity Recognition
Minghao Tang'? 12 Yongquan He Yongxiu Xu 122 Hongbo Xu', Wenyuan Zhang' and Yang Lin
Institute of Information Engineering, CAS, China
Schooo of Cyber Security, UCAS, China *Meituan, China
(tangminghao, xuyongxiu, hbxxujgei
heyongquanêmei tuan. com
Abstract
proteii
proteta
oolliype
Named entity recognition (NER) is fundamen- task natural language processing that aims identify and classify named entities in text. However, span-based methods for NER typi- cally assign entity types text spans, resulting in an imbalanced sample space and neglecting the connections between non-entity and entity spans. To address these issues, we propose novel approach for NER, named the Bound- ary Offset Prediction Network (BOPN), which predicts the boundary offsets between candi- date spans and their nearest entity spans. By leveraging the guiding semantics of boundary offsets, BOPN establishes connections between non-entity and entity spans, enabling non-entity spans function additional positive sam- ples for entity detection. Furthermore, our method integrates entity type and span represen- tations to generate type-aware boundary offsets instead of using entity types as detection tar- gets. We conduct experiments on eight widely- used NER datasets, and the results demonstrate that our proposed BOPN outperforms previous oftte-of-theare methods.
HMG box contalining transcription factors tymphocyte differentiation
Span
Entity Type
HMG
totventity
HNG lus
jesteii
nubsentity
Sitors
tor-entity
bralway
tymphootte
Afferssatin
Figure 1: A sentence from GENIA dataset (Ohta et al., 2002), containing 8 words and entities. The candidate spans cover the upper triangular region with total of 36 samples of cach matrix. There are 2 and positive samples for "protein" and "cell type" entity types, re- spectively.
and Miwa, 2018; Shen et al., 2021: Chen et al., 2021). Among these approaches, the span-based method has become the most popular due to its simplicity and effectiveness. It is straightforward that typically embeds all possible text spans and predicts their entity types, making suitable for various NER subtasks (Li ct al., 2021a, 2022).
1 Introduction
Named entity recognition (NER) a fundamental task in natural language processing (NLP) that in- volves identifying and categorizing named entities in text, such as people, locations and organizations. It has drawn much attention from the community due to its relevance in various NLP applications, such as entity linking (Le and Titov, 2018; Hou et al., 2020) and relation extraction (Miwa and Bansal, 2016; Li ct al., 2021b).
Despite significant progress made by span-based methods in NER, there remain two critical issues that require attention. Firstly, these methods often suffer from highly imbalanced sample spaces, as exemplified in Figure Such imbalance can neg- atively impact the trainability and performance of deep neural networks (Johnson and Khoshgoftaar, 2019). Although some methods (Shen et al., 2021; Wan et al., 2022) mitigate this issue by restricting the maximum span length, such an approach can also constrain the model's predictive power. Sec- ondly, current span-based methods primarily focus on learning the distinction between non-entities and entities, disregarding their relationships. While a model can identify whether "HMG box" an entity, it may fail to recognize the connection be-
Various paradigms have been proposed for NER, including the sequence labeling (Huang et al., 2015; Ju et al., 2018), 8), hypergraph-baseee based (Lu and Roth, 2015; Katiyar and Cardie, 2018; Wang and Lu, 2018), sequence- souuence-to-sqqueee to-sequence (Gillick et al., 2016; Yan et al., 2021) and span-baser methods (Sohrab
Yongxiu Xu is the corresponding author

---

provement in recall scores when aggregating results across offict labels, which particularly beneficial for recall-sensitive applications.
Problem Definition
Named Entity Recognition (NER) aims to iden- tify of all entities within an input sentence X {In 1 (= based on a pre-defined set of entity types Typically, an entity is specified by token boundaries and entity types.
End
Ebd
Text spans annotated with boundary offset. "IS" or "IE" represents a span has offset from its nearest entity at the start end boundary, and so on.
Figure 2:
Our proposed method focuses on predicting the boundary offset between each candidate text span and its nearest entity. Hence, formulate each text span quadruple: {#t, Ttj Js: Vm.) where and j denote the start and end boundary indices of the span, fs represents the start or end boundary offset from its nearest entity of type ym- Note that an entity span is a special case with fa = 0.
tween "HMG" and "HMG box." To enhance the model's ability to recognize entities, is crucial to explicitly capture both boundary differences and connections between non-entities and entities.
In this paper, we intend to model text spans by utilizing boundary offset information supervi- sion, rather than predict their probability of belong- ing to entities. As shown in Figure 2, there could be two advantages for deep models when boundary offsets are learnable: i) The natural quantitative re- lationships between offset values enable the model to capture boundary differences and connections simultaneously. ii) Non-entity spans can have spe- cific semantics that guide the positioning of entity spans, leading improved sample space with fewer negative samples.
A Annotation Guidelines To facilitate understand- ing, we present the essential boundary offset labels as follow:
Center Span: refers to an entity span with an offset label of "0".
*S or *E: denotes the annotation of the start or end boundary offsets for non-entity spans. "x" represents offset value in the range of [-S, S.S... 1, I,... ,JS, where denotes the maximum offset value.
Based on this observation, we propose the Boundary Offset Prediction Network (BOPN) for NER. BOPN focuses on predicting boundary off- sets between candidate spans and their nearest enti- ties, providing new perspective modeling text spans. Specifically, our method follows the pipeline of first learning span representations and then clas- sifying them for offset prediction. BERT (Devlin et al., 2019) and BiLSTM (Lample et al., 2016) are used to embed texts, followed by Conditional Layer (Liu et al., 2021) for building span represen- tations. Meanwhile, we also treat entity types as inputs rather than classification targets, which fused with span representations generate type- aware boundary offsets in parallel. Finally, we incorporate multiple 3D convolution layers to cap- ture the natural quantitative relationships between the offset values.
Out-of-Range: refers the annotation of non-entity span with an absolute boundary offset value from its nearest entity exceeding the maximum offset value S.
The annotation procedure for boundary offsets involves three steps. Initially, a trix O E MxNN is constructed according the input sentence X, where M denotes the num- ber of entity types and N represents the length of the sentence. Next, we annotate the center spans with the offset label "0" based on the golden en- tities present in X. Entities of different types are assigned to their respective bub-matrices. Finally, for non-entity spans, we compute the start and end boundary offset values with respect to all center spans. Their annotation is determined by the ab- solute minimum offset value. If the absolute min- imum offset value is less than S, we annotate the corresponding *S or *E; otherwise, we label the span as "Qut-ef-Range".
a 3-dimensional ma-
We evaluate our method cight widely-used NER datasets, including five English NER datasets and three Chinese NER datasets. The experimental results demonstrate that our approach outperforms the existing sfate-of-the methods. Furthermore, a detailed examination reveals a significant im-

---

(a) Span
(b) Boundary
Biaffine Classifier
3D convolation Layers
<Protein>
Type>
Fatily type
inpresentaations
Prediction
Dilation [1,23]
HMG
Span
Represestations
containing
transcription
CLN Layer
[0,0,1]
<Protela>
(0,0,0)
-1E
FFN
Catrutaring
lymphoeyte
1241
(0,2,4)
difierenciation
CCel Syper tymphngge
(1.6,7)
1E
FFN
Coordinate
Figure 3: An overview architecture of our method, which mainly consists of two components: a Span Encoder and a Boundary Offset Predictor.
3 Methods
Span Representation Given the token represen- tations H h2,,h2,.. ,,y, the span repre- sentation Vij can be considered as a fusion of the boundary representations (h₂, hy), Following Li et al. (2022), we adopt the Conditional Layer Nor- malization (CLN) (Liu et al., 2021) mechanism build a high-quality span representation:
Figure 3 provides overview of our method, which encompasses two primary components: Span Encoder (Section 3.1) and a Boundary Offset Predictor (Section 3.2). The Span Encoder is re- sponsible for encoding entity types and sentences, utilizing word representations to construct span representations. Subsequently, the entity type and span representations are inputted into the bound- ary offset predictor, facilitating type-aware offset classification.
Vij CCN(hs;,h)) hy) = % Norm(h:) + Aj
(1)
where Norm(-) is the instance normalization func- tion (Ulyanov et al., 2016), 7, and X, are the condi- tion parameters that are obtained by two differ- ent feedforward networks: % FFN(h;) and À (h,();).
3.1
Span Encoder
Drawing inspiration from the prompt-based meth- ods (Qin and Eisner, 2021; Han et al., 2022), consider entity types as task-oriented inputs, indi- cating the specific types of entities that the model needs to predict within a given sentence.
While valid candidate spans are restricted to the upper triangular region of the adjacent text span ma- trix, a region embedding = [Cup. Clow] E 2R are adapted to distinguish the positions of text spans. The final representation of each span is ob- tained as: big juij eapl if < j; dzj [rij. Stow] if i > j.
To achieve this, we create a set of additional type tokens, denoted as P {Pm))M J.M where Pmn represents a learnable special token corresponding to entity type yn- Next, we concatenate the soft tokens P with the sentence to form a single se- quence, and employ BERT (Devlin et al., 2019) to encode them simultaneously. The output of BERT is then passed through BiLSTM (Lample al., 2016) generate final embedding features H {h1, fu..... hM+N] (M+N)xx where d is the hidden size. Finally, we split to obtain entity type representations H' E ddd and token representations X E R respectively.
3.2 Boundary Offset Predictor
As previously mentioned, we utilize the entity types as inputs to guide the model in generating type- aware boundary offsets, rather than categorizing each text span into particular entity type.
The biaffine classifier (Yu et al., 2020) is em- ployed fuse entity type representations and span representations. Specifically, given an entity type representation hm E HY and span representation

---

ogj E V, a scoring vector Cmij E R can be com-
Inference with Boundary offsets During the in- ference process, decoding entities based on pre- dicted boundary offsets is a straightforward pro- cedure. The output of our method is a matrix of size M x N x N, where each cell represents a potential entity and contains information about its boundaries and type. For example, a cell with coor- dinates (m, i, i. and the prediction "-1E" indicates an entity of type ym with start boundary æ; and an end boundary 2j+1- Conversely, if the predicted value is "out-of-range, implies that the cell does not correspond to any entity.
puted as:
Ny - (hy() 6/3 - (
(2)
Cmij = (Iim)T + (him @ vig)) +
(3)
where L is the number of offset labels'; U E R Rixddxd, W E R4x24,, and R' are learn- able parameters, do the biaffine hidden size.
3D Convolution Layer Furthermore, we utilize multiple 3-dimensional convolution (3DConv) lay- ers to capture the inherent quantitative relationships between the boundary offsets of adjacent text spans. As depicted in Figure 3(b), the 3D convolution ker- nels traverse the complete score matrix C in three directions, thereby aggregating offset predictions for adjacent text spans across all entity types. The computation in a single convolution layer can be expressed as:
However, blindly accepting all predicted bound- ary offsets may result in sub sub-optimal outcomes it disregards the quantitative relationship between boundary offsets. Therefore, we introduce two heuristic rules to identify unreasonable predictions: i) Predicted boundary offsets that do not align with their nearest center span. ii) Predicted boundary offsets that do not adhere to a sequential order with neighboring spans.
Q (3DConv()))
(4)
where Q E IR. is the GELU activation function (Hendrycks and Gimpel, 2016). We assign different dilation rates to each convolution layer, and then concatenate their outputs followed by a linear to calculate final prediction scores:
4
Experimental Settings
4.1 Datasets
To evaluate our method, we conducted experiments five English NER datasets, including CoNLL 2003 (Sang and De Meulder, 2003), OntoNotes 52, ACE 20043, ACE 2005 and GENIA (Ohta et al., 2002); and three Chinese NER datasets, including MSRA (Levow, 2006), Resume NER (Zhang and Yang, 2018) and Weibo NER (Peng and Dredze, 2015). Note that ACE 2004, ACE 2005 and GENIA are nested NER datasets, others are flat datasets. For OntoNotes 5, we take the same train/dev/test as used in CoNLL 2012 shared task (Pradhan al., 2012). For ACE 2004 and ACE 2005, we use the same data split as Lu and Roth (2015). For GE- NIA, we follow Katiyar and Cardie (2018) to split train/test as 9:1. For other datasets, we employ the same settings in previous works (Ma et al., 2020; Yan et al., 2021; Zhu and Li, 2022).
Q Linear(Qu Q2 Q3):
(5)
To obtain the probability distribution of span (i,j) over the offset labels, amis E Q is fed into a softmax layer:
mmij (oftmax(@mij)).
3.3 Training and Inference
Learning Objective In our method, the learning objective is to accurately assign a boundary offset to each text span, which can be treated as a multi- class classification problem and optimized using cross-entropy loss:
N Σ m
M
1
L
Σ
log (0mi)))))
4.2 Implementation Details
M
We use BioBERT-vI.1 (Lee et al., 2020) as the contextual embedding in GENIA. For other En- glish corpora, we BERT-large-cased (Devlin et al.. 2019) as the contextual embedding. For Chinese
where Omij E D represents the ground-truth, which is an one-hot vector encoded from the anno- tated adjacent text span matrix O.
Giienn maximum offsct S, L = 4S +-2 when considering both start and end boundary offsct; 1. = 25 + 2 when only considering start or end boundary offset.
2 hhups://catlog Idc wwtp:::........d.dddieremme wweenneeduLLDC2005500
c https://catalog Idc upenn. www.uu.ewee..D..//.. LCC200000 "htps://catalog.ldde www..udennedduu. cdc.2//....2200606

---

CoNLL 2003
OntoNotes
Models
R
Fi
P
Fi
Sequence Labeling Methods
BiLSTM-CRE (Miwa and Bansal, 2016)
91.03
86.04
86.53
86.28
BERT-Tagger (Devlin et al., 2019)
92.80
90.01
88.35
89.16
Span-based Methods
Biaffine YYu al., 2020)*
92.46
92.67
92.55
89.94
89.81
89.88
W2NER (Li ct al., 2022)
92.71
93.44
93.07
90.03
90.97
90.50
Boundary Smooth (Zhu and Li, 2022)*1
92.89
93.20
93.04
90.42
90.81
90.61
DiffusionNER (Shen et al., 2023a)
92.99
92.56
92.78
90.31
91.02
90.66
Others
Seq2Seq (Straková et al., 2019)
92.98
BartNER (Yan et al., 2021>
92.57
93.53
93.05
89.65
90.87
90.26
PIQN (Shen al., 2022)
93.29
92.46
92.87
91.43
90.73
90.96
PrompiNER (Shen al., 2023b)
92.48
92.33
92.41
BОPN (Ours)
93.22
93.15
93.19
90.93
91.40
91.16
Table 1: Results on English flat NER datasets CoNLL 2003 and OntoNotes means our re- re-implementation via their code. denotes fair comparison that their BERT encoder is consistent with our model.
MSRA
Resume NER
Weibo NER
Models
P
R
F
P
F
Р
R
Fi
Sequence Labeling Methods
Lattice (Zhang and Yang. 2018)
93.57
92.79
93.18
94.81
94.11
94.46
53.04
62.25
58.79
Flat (Li et al., 2020)
96.09
95.86
68.55
SoftLexicon (Ma al., 2020)
95.75
95.10
95.42
96.08
96.13
96.11
70.94
67.02
70.50
MECT (Wu et al., 2021)
96.24
95,98
70.43
Span-based Methods
W2NER (L et al., 2022)
96.12
96.08
96.10
96.96
96.35
96.65
70.84
73.87
72.32
Boundary Smooth (Zhu and Li, 2022)
96.37
96.15
96.26
96.63
96.69
96.66
70.16
75.36
72.66
DiffusionNER (Shen et al., 2023a)
95.71
94.11
94.91
BOPN (Ours)
96.44
96.34
96.39
96.73
96.83
96.78
71.79
73.90
72.92
Table 2: Results on Chinese flat NER datasets MSRA, Resume and Weibo.
corpora, we use the BERT pre-trained with whole word masking (Cui al., 2021).
Results and Analysis
The BiLSTM has one layer and 256 hidden size with dropout rate of 0.5. The size of region em- bedding de is 20, The maximum offset value S is selected in {1, 2.). For all datasets, we train our models by using AdamW Optimizer (Loshchilov and Hutter, 2017) with a linear warmup-decay learning rate schedule. See Appendix for more details. Our source code can be obtained from https///gthhb..cm//mann////99/
5.1 Main Results
The performance of our proposed method and the basclines on English flat NER datasets presented in Table 1. The experimental results demonstrate that our approach surpasses the previous state-of- the-art (SOTA) methods by +0.12% on the CoNLL 2003 dataset and +0.20% on the OntoNotes 5 dataset, achieving superior performance with Fi scores of 93.19% and 91.16%, respectively. For Chinese flat NER datasets, we provide the results in Table 2. Similarly, our proposed method achieves SOTA performance terms of F1 scores, surpass- ing the previous best method by +0.1 -0,13, +0.12%, and +0.26% in Fi scores on the MSRA, Resume NER, and Weibo NER datasets, respectively. The performance results on English nested NER datasets are presented Table 3. Remarkably,
4.3 Evaluation
We use strict evaluation metrics where predicted entity is considered correct only when both the boundaries (after adding boundary offset) and type are accurately matched. The precision, recall and Fi scores are employed. We run our model for five times and report averaged values.

---

ACE 2004
ACE 2005
GENIA
Models
R
F1
P
R
F1
P
P1
Sequence Labeling Methods
Layered (Ju et al., 2018)
74.2
70.3
72.2
78.5
71.3
74.7
Pyramid (Wang al., 2020)
86.08
86.48
86.28
83.95
85.39
84.66
79.45
78.94
79.19
Span-based Methods
Biaffine (Yu al., 2020)
87.3
86.0
86.7
85.2
85.6
85.4
78.2
78.2
78.2
Locate und Label (Shen et ul., 2021)
87.44
87.38
87.41
86.09
87.27
86.67
80.19
80.89
80.54
W2NER (I.i ct al., 2022)
87.33
87.71
87.52
85.03
88.62
86.79
83.10
79.76
81.39
Triaffine (Yuan et al., 2022)
87.13
87.68
87.60
86.70
86.94
86.82
80.42
82.06
81.23
Boundary Smooth (Zhu and Li. 2022)
88,43
87.53
87.98
86.25
88.07
87.15
DiffusionNER (Shen al., 2023a)
88.11
88.66
88.39
86.15
87.72
86.93
82.10
80.97
81.53
Others
Seq2Seq (Straková et al., 2019)
84,33
83.42
78.20
BarNNER (Yan et al., 2021)
87.27
86.41
86.84
83.16
86.38
84.74
78.57
79.30
78.93
PIQN (Shen al., 2022)
88.48
87.81
88.14
86.27
88.60
87.42
83.24
80.35
81.77
PrompINER (Shen et ul., 2023b)
87.58
88.76
88.16
86.07
88.38
87.21
BOPN (Ours)
89.13
89.40
89.26
89.56
91.23
90.39
82.14 14
82.16
82.14
Table 3: Results on English nested NER datasets ACE 2004, ACE 2004 and GENIA.
CoNLI 2003
Resume NER
ACE 2004
5.2 Ablation Studies
In order to assess the impact of cach component in our method, we conduct ablation studies on the CoNLL 2003, ACE 2005, and Resume NER datasets. The results of these studies are presented in Table 4.
BOPN (Ours)
93.19
96.78
89.26
w/o Type Inp.
92.87
96.41
88.83
w/O Region Emb.
92.71
96.22
88.71
w/o BO
92.74
96.26
88.62
w/o 3DConv
92.87
96.40
89.11
MBO (S 1)
93.11
96.75
89.14
Maximum Boundary Offset We investigate the
MBO (S
93.15
96.78
89.26
impact of training the model with different maxi- mum offset values S through our ablation studies.
MBOO (S 3)
93.19
96.71
89.22
3DConv (l=
93.08
96.69
89.18
The hyperparameter S determines the annotation scope of non-entity spans with boundary offset. Specifically, the extreme scenario of setting S corresponds to a condition "w/o BO" (without Boundary Offset). The results indicate a significant decline in performance when employing "w/o BO," confirming the usefulness of utilizing boundary off- sets as supervision. However, we also observe that the optimal S value varies across different datasets. This could be attributed to the fact that larger S value provides more boundary knowledge but also increases the label search space. Consequently, hy- perparameter tuning for becomes necessary to achieve the best performance in practice.
-3DConv (I=
93.19
96.75
89.26
-3DConv (l=3)
93.05
96.78
89.25
Table 4: Ablation Studies. MBO means the maximum
boundary offset value.
our proposed BOPN achieves substantial improve- ments in performance these datasets, with Fi scores increasing by +0.87% +2.97%, and +0.37% on ACE 2004, ACE 2005, and GENIA, respec- tively. These results align with our expectations, as the boundary features of nested entities are more intricate compared to flat entities. We attribute this improvement to two key factors: 1) Our method predicts the boundary information of various en- tity types in parallel, effectively avoiding nested boundary conflicts between different types of enti- ties. 2) By predicting boundary offsets, our method expands the predictive range for each text span, al- lowing for more granular and precise identification entity boundaries.
In addition, we analyze the learning curves of our model with different maximum offset values. Figure demonstrates that larger S can accelerate the training process of the model. We think the reason may be that larger S not only leads an increase of positive samples but also results in a decrease of negative samples, thereby ultimately enhancing the trainability of the model.

---

100
MBQ 5-2
80
+0.69
+0.55
+2.12
15.15
60
12.50
F1
wio BO
MBO S=1
MBO S
MBO S-3
122129) 826) 3,905)
41387)
5/1
7120)
0117)
(+120)
Enilyy Langhn Support
10
Epochs
Figure 5: A comparison Fl-scores on entities of different lengths in GENIA dataset. Entity supports are in the parentheses.
Figure 4: The learning curves on ACE 2004 dataset.
Label
P
F1
Support
representation and classification, they cannot be evaluated independently. Nonetheless, our ablation studies demonstrate the effectiveness of learning boundary offset information and the usefulness of each composition in our model.
-2S
81,51
82.02
81.76
5029
-IS
81.62
82.97
82.29
5292
IS
79.55
81.47
80.50
3281
2S
76.27
79.55
77.88
1438
-2E
78.64
77.19
77.90
1464
-IE
79.79
80.58
80.18
3254
IE
82.26
82.20
82.23
5393
5.3
Detailed Analysis
2
82.37
80.75
81.57
5113
Performance on Different Offset Labels We in- 5495
0
81.92
81.95
81.93
vestigate the performance of cach boundary offset label, and the results are presented in Table 5. No- tably, the offset label "0" has complete entity sup- port and achieves an Fi score of 82.04%. Further- more, we observed positive correlation between the quantity entity support and the performance of boundary offset labels.
ALL
79.21
84.22
81.64
5495
w/ rules
81.85
82.56
82.20
5495
Table 5: Performance of each boundary offset label on GENIA, where the maximum offset value is The reported results is out of five experiments.
When a text span is not predicted as "out-of- range", its assigned label can be utilized to deter- mine the position of its nearest entity. By aggre- gating all predictions of offset labels, we observe sharp decrease in precision score, along with significant increase in recall score, when compared to only considering the center span (with an offset label of "O"). This finding suggests that different offset labels provide distinct information that as- sists the model in recognizing additional entities. Nevertheless, this approach can introduce noisy predictions due to the model's inadequate perfor- mance certain labels. Despite this limitation, it may have practical applicability in recall-sensitive applications.
3D Convolution Layer "w/o 3DConv" indicates the 3D convolution layers are removed. As seen, the results show decline in performance across all datasets, indicating the importance of 3D convo- lution layers in capturing the interactions between boundary offsets adjacent text spans.
"w/o Type Inputs" refers setting where the entity types encoded with the sentence are replaced, in which the randomly initialized en- tity type embeddings are fed into the biaffine clas- sifier. The results obtained in this setting show slight decline in performance.
Type Inputs
Region Embedding The results demonstrate a slight drop in performance across all datasets with- out region embeddings. This suggests that integrat- ing sample distribution features can be reasonable approach for enhancing text span representations.
As discussed in Section 3.3, devise two heuristic rules to remove improbable predictions. Our findings reveal that this approach enhances the precision score, with only a minor reduction in the recall score, leading to an overall improvement the F, score.
As the CLN layer and biaffine classifier serve as fundamental components our approach for span

---

90
linear classifier for type prediction. Alternatively, Yu et al. (2020) utilizes a biaffine classifier to fuse start and end boundary representations directly for span classification. To further enhance span repre- sentation, several other methods(Wan) Wan et al., 2022; Yuan et al., 2022) propose fusing representations of token, boundary, and related entity spans.
80
FT Soores
60
50
40
Meanwhile, some methods try to improve span- based methods by adding boundary supervision. Specifically, Zheng et al. (2019) and Tan et al. (2020) additionally detect entity boundaries with multi-task learning, while Shen et al. (2021) per- form boundary regression after span prediction. Li et al. (2022) design two word-word relations for span classification. Compared with previous methods, our proposed method utilizes continuous boundary offset values to model text spans, which can capture both the boundary differences and con- nections between non-entity and entity spans.
30
S-2
20
100%
75%
50%
25%
12.5%
6.75%
Percentage of training samples
Figure 6: Effect of varying percentage of training sam- ples on GENIA. We train all models for 50 epochs and report their best performance.
Performance on Entities with Varying Lengths We explore the model performance on entities of different lengths in GENIA. As shown in Figure 5, we compare the F1 scores of models which are training with different S. The model achieves higher F1 scores across all columns when S 2, with more pronounced performance improvement for longer entities. The results highlight the use- fulness of learning boundary offsets between non- entity and entity spans, which helps the model learn boundary features more effectively.
In addition to span-based methods, there are three widely-used NER methods. The traditional sequence labeling methods (Huang et al., 2015; Lample et al., 2016) assign each token a tag with a pre-designed tagging scheme (c.g., BIO). To ad- dress nested entities, some works (Ju et al., 2018; Wang et al., 2020; Rojas et al., 2022) add struggles or design special tagging schemes. Hypergraph- based methods (Lu and Roth, 2015; Katiyar and Cardie, 2018; Wang and Lu, 2018) represent the input sentence as hypergraph for detecting nested entities, which must be carefully designed avoid spurious structures. Sequence-to--sequene meth- ods reformulate NER as a sequence generation problem. For example, Gillick et al. (2016) first ap- ply the Seq2Seq model for NER, inputting the sen- tence and outputting start positions, entity lengths, and types. Straková et al. (2019) use the Seq2Seq model and enhanced BILOU scheme address nested NER. Yan et al. (2021) treats NER as an en- tity span sequence generation problem with pointer network based on BART (Lewis et al., 2019).
Size of Training Data As the boundary offset labels contain more informative knowledge, we hypothesize that our proposed BOPN would per- form better with limited training data. As shown in Figure 6, our model achieves impressive results, exhibiting only a 5.46% decrease in performance when trained with a mere 12.5% of the available training data. In contrast, when boundary informa- tion is not utilized during training, the model's 's per- formance declines rapidly as the amount of training data decreases, thus creating significant obstacles to effective training.
6 Related Work
In recent years, various paradigms for named en- tity recognition (NER) have been proposed, among which span-based methods have become one of the most mainstream approaches, treating NER as a text span classification problem. With the de- velopment of pre-trained language models, some works (Sohrab and Miwa, 2018; Luan et al., 2019; Wadden et al., 2019) obtain span representations by connecting boundary representations or aggre- gating token representations and feeding them into
Conclusion
In this paper, we introduce novel approach for named entity recognition (NER) called the Bound- ary Offsct Prediction Network (BOPN), BOPN pre- dicts the boundary offsets between candidate spans and their nearest entities, leveraging entity types as inputs. By incorporating entity types, BOPN enables parallel prediction of type-aware boundary offsets, enhancing the model's ability to capture

---

fine-grained entity boundaries. To capture the in- teractions between boundary offsets, we employ multiple 3D convolution layers, which refine the offset predictions and capture the inherent quanti- tative relationships between adjacent text spans.
References
Pei Chen, Haibo Ding, Jun Araki, and Ruihong Huang. 2021. Explicitly capturing relations between en- tity mentions via graph neural networks for domain- specific named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Volume 2: Short Pupers), pages 735-742.
The experimental results demonstrate that our proposed method achieves state-of-the oftt---rrr perfor- mance on eight widely-used datasets, including five English NER datasets and three Chinese NER datasets. Moreover, further analysis reveals a sig- nificant improvement in recall scores by utilizing boundary offset as supervision, showcasing the util- ity of our approach for recall-sensitiv applications in NER.
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, and Ziqing Yang. 2021. Pre-training with whole word masking for chinese bert. IEEF/ACM Transac- tions on Audio, Speech, and Language Processing. 29:3504-355 3514.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies, Volume (Long and Short Papers), pages 4171- 4186.
Limitations
The proposed BOPN approach has certain limita- tions that should be acknowledged. Firstly, while BOPN treats boundary offsets as classification tar- gets, it does not explicitly model the order rela- tionship between offset values. Although the 3D convolution layers are employed to implicitly cap- ture interactions between boundary offsets, they do not provide strong constraint on the ordering of offset labels.
Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2016. Multilingual language process- ing from bytes. In Proceedings of the 2016 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, pages 1296-1306.
Additionally, the method uses boundary offsets to convert some non-entity spans into positive sam- ples, which leads to higher recall scores but poten- tially lower precision scores. To optimize predic- tion results, hcuristic rules are applied to filter out unreasonable samples. However, these rules are based on observations and may not be comprehen- sive enough to handle all cases effectively.
Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. 2022. Ptr: Prompt tuning with rules for text classification. Al Open, 3:182-192.
Dan Hendrycks and Kevin Gimpel. 2016. Gaus- sian enror linear units (gelus). arXiv preprint 16X6v:1606.08413
Feng Hou, Ruili Wang. Jun He, and Yi Zhou. 2020. Im- proving entity linking through semantic reinforced entity embeddings. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 6843-6848.
Therefore, there is still a need to explore more effective ways to integrate and optimize the offset predictions in order to address these limitations and enhance the overall performance of the BOPN approach.
Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirec- tional Isum-erf models for sequence tagging. arXiv preprint arXiv: I 1508.0199)
Ethics Statement
Justin M Johnson and Taghi M Khoshgoftuar. 2019, Sur- vey on deep learning with class imbalance. Journal of Big Data, 6-1):1-54.
To address ethical concerns, we provide the two detailed description: 1) All experiments were con- ducted on existing datasets derived from public scientific papers. 2) Our work does not contain any personally identifiable information and does not harm anyone.
Meizhi Ju. Makoto Miwa, and Sophia Ananiadou. 2018. A neural layered model for nested named entity recog- nition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long Papers), pages 1446-1459.
Acknowledgements
Arzoo Katiyar and Claire Cardie. 2018. Nested named entity recognition revisited. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, volume 1.
This work was supported by Strategic Priority Re- search Program of Chinese Academy of Sciences (NO. XDC02040400).

---

Guillaume Lample, Miguel Ballesteros. Sandeep Sub- ramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Compula- tional L Linguistics: Human Language Technologies, pages 260-270.
emotions. In Findings of the Association for Com- putational Linguisties: ACL-IJCNLP 2021, pages 4332-4339.
Iyy Loshchilov and Frank Hutter. 2017. Decou- pled weight decay regularization. arXiv preprint arXiv:1711.0500..
Wei Lu and Dan Roth. 2015. Joint mention extrac- tion and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 857- 867.
Phong Le and Ivan Titov. 2018. Improving entity link- ing by modeling latent relations between mentions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Popers), pages 1595-
Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf. and Hannaneh Hajishirzi. 2019. A general framework for information extraction using dynamic span graphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisticx: Human Language Tech- nologies, Volume (Long and Short Papers), pages 3036 3046, Minneapolis. Minnesota. Association for Computational Linguistics.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang- 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):12344 1240.
Gina-Anne Levow. 2006. The third international chi- nese language processing bakeoff:: Word segmenta- tion and named entity recognition. In Proceedings of the Fifth SIGHAN workshop on Chinese language processing, pages 108-117.
Ruotian Ma, Minlong Peng, Qi Zhang, Zhongyu Wei, and Xuan-Jing Huang. 2020. Simplify the usage of lexicon in chinese ner. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5951-5960
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, AbdeIrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: De- noising sequence-to-sequeee pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.1344 13461.
Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using Istms on sequences and tree structures. In Proceedings of the 54th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1 1105-1116.
Fei Li, ZhiChao Lin, Meishan Zhang, and Donghong Ji. 2021a. A span-basee model for joint overlapped and discontinuous named entity recognition. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4814 1814-4828, Online. Association for Computational Linguisties.
Tomoko Ohta, Yuka Tateisi, Jin-Dong Kim, Hideki Mima, and Junichi Tsujii 2002. The genia corpus: An annotated research abstract corpus in molecular biology domain. In Proceedings of the human lan- guage technology conference, pages 73-77. Citeseer.
Nanyun Peng and Mark Dredze. 2015. Named en- tity recognition for chinese social media with jointly trained embeddings. In Proceedings of the 2015 con- ference on empirical methods in natural language processing, pages 548-554.
Jingye Li, Hao Fei, Jiang Liu, Shengqiong Wu, Meishan Zhang, Chong Teng, Donghong Ji, and Fei Li. 2022. Unified named entity recognition as word-word rela- tion classification. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 36, pages 10965-10973.
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll- 2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Joint conference on EMNLP and CoNLL-shared task, pages 1-40.
Jingye Li, Kang Xu, Fei Li, Hao Fei, Yafeng Ren, and Donghong Ji. 202 Mrn: A locally and globally mention-based reasoning network for document- level relation extraction. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1359-1370)
Guanghui Qin and Jason Eisner. 2021. Learning how to ask: Querying Ims with mixtures of soft prompts. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 520--5212.
Xiaonan Li, Hang Yan, Xipeng Qiu, and Xuan-Jing Huang. 2020. Flat: Chinese ner using flat-lattice transformer. In Proceedings of the 58th Annual Meet- ing of the Association for Computationall Linguistics, pages 6836-6842.
Matías Rojas, Felipe Marruuzaa and Jocelyn Dun- stan. 2022. Simple yet powerful: An overlooked architecture for nested named entity recognition. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2108-2117.
Ruibo Liu, Jason Wei, Chenyan Jia, and Soroush Vosoughi. 2021. Modulating language models with

---

Erik Tjong Kim Sang and Fien De Meulder. 2003, In- troduction to the conll- 2003 shared task: Language- independent named entity recognition. In Proceed- ings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142-147.
Juncheng Wan, Dongyu Ru, Weinan Zhang, and Yong Yu. 2022. Nested named entity recognition with span- level graphs. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 892-903, Dublin, Ireland. Association for Computational Linguistics.
Yongliang Shen, Xinyin Ma, Zeqi Tan, Shuai Zhang, Wen Wang, and Weiming Lu. 2021. Locate and la- bel: A two-stage identifier for nested named entity recognition. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 2782-2794.
Bailin Wang and Wei Lu. 2018. Neural segmental hy- pergraphs for overlapping mention recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 204- 214.
Jue Wang. Lidan Shou, Ke Chen, and Gang Chen. 2020. Pyramid: A layered model for nested named entity recognition. Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 5918-5928.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023a. Diffusion- ner: Boundary diffusion for named entity recognition. arXiv preprint arXiv:2305. 13298.
Shuang Wu, Xiaoning Song, and Zhenhua Feng. 2021. Mect: Multi-metadate embedding based cross- transformer for chinese named entity recognition. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume 1: Longg Papers), pages 1529- 1539.
Yongliang Shen, Zeqi Tan, Shuhui Wu, Wenqi Zhang, Rongsheng Zhang, Yadong Xi, Weiming Lu, and Yueting Zhuang. 2023b. Promptner: Prompt locat- ing and typing for named entity recognition. arXiv preprint arXiv:2305.0 17011
Yongliang Shen, Xiaobin Wang, Zeqi Tan, Guangwei Xu, Pengjun Xie, Fei Huang, Weiming Lu, and Yuet- ing Zhuang. 2022. Parallel instance query network for named entity recognition. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Pupers), pages 947-961.
Hang Yan. Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, and Xipeng Qiu. 2021. A unified generative framework for various ner subtasks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 5808 5828.5822.
Mohammad Golam Sohrab and Makoto Miwa. 2018. Deep exhaustive model for nested named entity recogmition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Process- ing, pages 2843-2843
Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named entity recognition as dependency parsing. Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 6470- 6476.
Jana Straková, Milan Straka, and Jan Hajic. 2019. Neu- ral architectures for nested ner through linearization. Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 5326- 5331.
Zheng Yuan, Chuanqi Tan, Songfang Huang, and Fei Huang. 2022. Fusing heterogeneous factors with triaffine mechanism for nested named entity recog- nition. In Findings of the Association for Computa- tional Linguistics: ACL 2022, pages 3174-3186.
Chuanqi Tan, Wei Qiu, Mosha Chen, Rui Wang, and Fei Huang. 2020. Boundary enhanced neural span classification for nested named entity recognition. In Proceedings of the AAAI conference artificial intelligence, volume 34, pages 9016-9023.
Yue Zhang and Jie Yang. 2018. Chinese ner using lattice Istm. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Vol- ume I: Long Papers), pages 1554-1564.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lem- pitsky. 2016 Instance normalization: The miss- ing ingredient for fast stylization. arXiv preprint arXiv:1607.08022
Changmeng Zheng, Yi Cai, Jingyun Xu, Ho-fung Le- ung. and Guandong Xu. 2019, A boundary- aware neural model for nested named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIL pages 357-366, Hong Kong, China. Association for Computational Linguis- tics.
David Wadden, Ulme Wennberg, Yi Luan, and Han- naneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- pages 5784-
guage Processing (EMNLP-IJCNN 5789, Hong Kong, China. Association for Computa- tional Linguistics.
Enwei Zhu and Jinpeng Li. 2022. Boundary smoothing for named entity recognition. In Proceedings of the

---

60th Annual Meeting of the Association for Compu- tational Linguistics (Volume Long Papers), pages 7096-7108.
Softoexicon (Ma et al., 2020) proposes a Chi- nese NER method in which lexicon informa- tion is introduced by simply adjusting the char- acter representation layer.
A Appendix
MECT (Wu al., 2021) uses embedding in a two-stream transformer to in- tegrate Chinese character features with the radical-level embedding.
A.1 Datasets
We evaluate our method on eight datasets, in- cluding CONLL 2003, OntoNotes 5, ACE 2004, ACE 2005, and GENTA for English NER datasets; MSRA, Resume NER and Weibo NER for Chi- nese NER datasets. Table 6 presents the detailed statistics of datasets.
Biaffine (Yu et al., 2020) classifies text spans by a biaffine classifier between boundary rep- resentations.
Locate and Label (Shen et al., 2021) pro- posed a two-stage identifier of locating entities with boundary regression first and classifying them later.
A.2 Implementation Details
We use BioBERT-vI.1 (Lee et al., 2020) as the con- textual embedding in GENIA. For other English corpora, we BERT-large-cased (Devlin et al., 2019) the contextual embedding. For Chinese corpora, we use the BERT pre- tre-trained with whole word masking (Cui et al., 2021). Our model is imple- mented with PyTorch and trained with a NVIDIA RTX3090 GPU. We use grid search to find the best hyperparameters which are tuned on the devel- opment set. The range of hyperparameters we used for eight datasets are listed in Table 7.
W2NER (Li et al., 2022) models NER word-word relation classification, including the next-neighboring- word and the tail-head- word relations.
Triaffine (Yuan et al., 2022) proposed tri- affine mechanism to fuse information of inside tokens, boundaries, labels for NER.
Boundary Smooth (Zhu and Li, 2022) pro- posed boundary smoothing regularization technique for span-based neural NER models.
A.3 Baselines
We compare BOPN with the following baselines:
BiLSTM-CRE (Miwa and Bansal, 2016) is model for sequence labeling tasks that com- bines BiLSTM with CRF layers.
DiffusionNER (Shen al., 2023a) formu- lates NER as a noundary-denoisin diffusion process, which samples noisy spans from a Gaussian distribution.
BERT-Tagger (Devlin et al., 2019) that uti- lizes the pre-trained language model BERT as feature extractor and incorporates tag classifier for fine-tuning.
Seq2Seq (Straková et al., 2019) converts the labels nested entities into a sequence and then uses a seq2seq model to decode entities.
Lattice (Zhang and Yang, 2018) proposed lattice-structured LSTT model for Chinese NER.
BartNER (Yan et al., 2021) formulates NER as an entity span sequence generation prob- lem based on the pre-training Seq2Seq model BART (Lewis et al., 2019).
Layered (Ju al., 2018) dynamically stacks flat NER layers solve nested NER task.
PIQN (Shen et al., 2022) sets up global and learnable instance queries to extract entities from sentence in parallel manner.
Flat (Li al., 2020) proposes flat-lattice transformer for Chinese NER, which converts the lattice structure into a flat structure con- sisting of spans.
PromptNER (Shen et al., 2023b) unifies en- tity locating and entity typing in prompt learn- ing for NER, which predicts all entities by filling position slots and type slots.
Pyramid (Wang et al., 2020) designs pyra- mid layer and inverse pyramid layer to decode nested entities.

---

CoNLL 2003
OntoNotes 5
ACE 2004
ACE 2005
GENIA
MSRA
Resume
Weibo
Types
4
18
7
7
5
3
8
4
#Train.S
17291
59924
6200)
7194
16692
46471
3819
1350
#Dev.S
8528
745
969
463
270
#Test.S
3453
8262
812
1047
1854
4376
477
270
Avg.Len.S
14.38
18.11
22.61
18.97
25.41
45.54
31.17
54.57
#Train.E
29441
128738
22204
9389
50509
74703
13438
1855
#Dev.E
20354
2514
1112
1497
379
#Test.E 5648
12586
3035
1118
5506
6181
1630
409
Avg.Len.E 1.45
1.83
2.50
2.28
1.97
3.24
5.88
2.60
Table 6: Dataset Statistics. "#" denotes the amount. "S." and "E." denote sentence and entity mentions, respectively.
Parameter
Value
Epoch
[50, 80]
Batch size
[8, 16]
Learning rate (BERT)
[5e-6, 3e-5]
Learning rate (Other)
le-3
LSTM hidden size
256
LSTM dropout
0.5
Region embedding size de
20
Biaffine hidden size do
150
Biaffine dropout
0.2
Maximum offset value S
[1,3]
Adam epsilon
Ie-8
Warm factor
0.1
Table
settings.


Reviews:
[{"review_id": "yxyJlWx8HK", "reviewer": "Reviewer_GY7H", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your valuable comments. We truly appreciate your comments. Here are our responses:\n\nQ: The improvement of the NER performance is relatively marginal (compared to the existing models).\n\nA: We acknowledge that the existing NER methods have made significant advancements in recent years, reaching higher performance levels and achieving substantial gains has become increasingly challenging. Different from previous methods, our approach mainly focuses on  optimizing imbalanced sample spaces by considering the entity-relative position information. Specifically, we employ multiple 3D convolution layers to explicitly capture the the inherent quantitative relationships between the boundary offsets of adjacent text spans. However, it is important to recognize that every incremental performance improvement requires great effort. For instance, the previous state-of-the-art model (W2NER 2022) achieved a 93.07% F1 score on CoNLL 2003, only surpasses the baseline model (Bert-Tagger) by 0.27% F1 score.While our method outperforms Bert-Tagger by 0.39% F1 on CoNLL 2003.\n\n\nWe also beg to provide clear clarification that we conducted experiments on eight widely-used NER datasets, and our method consistently demonstrated superior performance across all of them. Specifically, we achieved boosted F1 scores of +0.87%, +2.97%, and +0.37% on ACE 2004, ACE 2005, and GENIA, respectively. Moreover, our method significantly enhances recall performance by effectively aggregating all boundary offset results. This innovative approach offers practitioners new insights, particularly in industries where recall holds greater importance than F1 performance. \n\n\nWe genuinely hope that our responses address your queries and contribute to a positive outcome. Your time, effort, and consideration are highly valued and deeply appreciated.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper proposes a new span-based NER model with state-of-art performance\n* Extensive experiments are conducted on multiple popular NER datasets\n* The paper is well-written", "reasons_to_reject": "* The improvement of the NER performance is relatively marginal", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a new span-based NER model that solves the task by predicting boundary offsets of named entities. The main idea is that it uses an offset to the nearest entity span as the target labels of each candidate span. Furthermore, it assigns input embeddings to entity types and treats them as inputs. These techniques enhance the expressive power of the model, improving the NER performance.\n\nThe paper is well-written and easy to read. The model's effectiveness is tested using extensive experiments on multiple popular NER datasets.\n\nOne weakness is that, although the proposed model is relatively complex, the performance improvement compared to the existing models is marginal. For example, without using the 3D convolution layer, the model does not outperform the previous models on CoNLL 2003 and Resume NER.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CmGzKtUTW7", "reviewer": "Reviewer_no4M", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your invaluable and encouraging feedback. We deeply appreciate your thoughtful comments, and we are grateful for the opportunity to incorporate your suggestions. Your insights will undoubtedly help us improve the readability of our paper and provide a more comprehensive discussion about the major error patterns in the final version of our work. Once again, thank you for your invaluable contribution.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A. The paper is clear in its motivation, problem formulation, overall description of the method and results.\n\nB. The paper improves the NER performance by modeling it a bit differently than ongoing span-based models for NER and leveraging effectively the relative positioning information between entity and non-entity spans. The potential to boost recall can be particularly attractive for practitioners. \n\nC. The authors perform extensive experiments on 8 datasets from 2 languages and with varying complexity (number of entity types, presence of nested entities, etc.), not only convincing the reader of the overall results, but also clearly demonstrating the effects of various architectural choices, effects of text span sizes and training parameter choices.", "reasons_to_reject": "A. The paper readability can be improved with a round of proof-reading, in that certain typos can be fixed.\n\nB. While the Limitation section introspects the shortcomings of the approach, the paper can be improved by discussing the major error patterns that still remain despite outperforming SOTA methods.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper introduces a new approach for the NER task based on the idea that the relationships between entity and non-entity text spans matter, and accordingly, the boundary offsets of candidate spans relative to the nearest entity span helps improve the prediction. The authors design a boundary offset prediction network (BOPN) that, unlike conventional span-based NER modeling, model the problem as a multi-class classification task where the output space involves predicting the offset labels, corresponding to whether a span refers an entity span, is away from the nearest entity span by some \"allowable\" offset, or is outside the maximum offset value. Another flexibility of the model is that it is designed to be type-aware by including the entity type as an input (instead of output), with a trainable token. The approach is aimed to tackle two challenges with span-based deep networks performing NER: 1) addressing the imbalance problem between entity and non-entity spans by leveraging their relative positioning, and 2) not ignoring the relationship between entity and non-entity span, but instead using the non-entity spans to provide better positive signals. As such, the approach not only fares better than state-of-the-art methods, but is also shown to improve recall.\n\nThe paper starts with a good brief introduction to the problem, drawing the reader gradually to the two key challenges that motivate the new design. The intuition behind the design, the architecture, and all modeling aspects (annotation guidelines, representation, prediction network, training, inference) have been explained well, with diagrams, examples and appropriate level of equations and citations that convey all the important details. The results of the approach are clear from the extensive experiments performed on 8 datasets (5 English, 3 Chinese). \n\nWhile the results outperform existing methods, they are far from perfect. Therefore, the paper could be improved by including a discussion of when the technique falls short. Please note, this is different from the limitations of the approach, in that it is about supporting results with error analysis that help readers understand patterns that cannot be addressed or continue to remain challenging. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- L124: Omit 'that'\n- Fig 3: 'The' -> 'An'\n- L193: 'an region embedding' -> 'a region embedding'\n- L351: 'Thel' -> 'The'\n- L355: Omit redundant 'datasets'"}, "extra_text": {}}, {"review_id": "lqGKfJ0Al5", "reviewer": "Reviewer_TfVb", "summary": "", "questions": "What are the pros and cons compared to recent generative IE models?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. Here are our replies:\n\nQ1：About the comparison with the paper [1] (\"Automated Concatenation of Embeddings for Structured Prediction\") on CoNLL 2003 dataset.\n\n\nA1: We did not chosen paper[1] as our baseline model, mainly for the following two reasons: \n\n\n1) The research motivations differ significantly. While our method focuses on innovating the NER model architecture, the primary objective of the paper [1] is to find better word representations rather than improving model architectures. \n\n\n2) Comparing our method with the paper [1] would introduce unfairness  due to differences in experimental settings. The paper [1] proposes an algorithm that automatically combines 11 auxiliary resources to generate word embeddings. In contrast, our approach emphasizes innovation on model architecture and solely uses BERT-large for word embeddings. In Table 1 of the paper [1], the combination of nine resources (ELMo, Flair, base BERT, GloVe word embeddings, fastText word embeddings, non-contextual character embeddings, multilingual Flair (M-Flair), M-BERT, and XLM-R embeddings) achieves 93.0 F1 on CoNLL 2003. However, our method achieves a slightly higher F1 of 93.19 using only BERT-large.  In Table 2 of the paper [1], they further add XLNet and RoBERTa in addition to the nine resources, resulting in an F1 score of 94.6. It is known that XLNet and RoBERTa can  produce more powerful word embeddings. We would like to share  experimental results that replaces BERT-large with XLNet and RoBERTa in our method, which achieves F1 scores of 94.5 and 94.2 F1 on CoNLL 2003, respectively.\n\n\nWe genuinely hope that our responses address your queries and contribute to a positive outcome. It is crucial to ensure fair conditions for comparisons. We appreciate your understanding and continued support.\n\n\nQ2: About the comparison with the paper [2] (\"Dice Loss for Data-imbalanced NLP Tasks\") on Ontonotes dataset.\n\n\nA2: Similarly, we refrain from comparing our approach with paper [2] due to the different motivations. Paper [2] proposes the dice-based loss function to enhance model performance, while our method aims to innovate the NER model architecture. Paper [2] utilizes the MRC-NER model as the backbone. In a fair environment with the same loss function and experimental settings, our method demonstrates comparable performance on Ontonotes dataset, achieving 91.16 (ours) vs. 91.11 (MRC-NER).\n\n\nQ3：About the pros and cons compared to recent generative IE models.\n\n\nA3: We have referenced two influential papers, [3] (A unified generative framework for various NER subtasks, 2021) and [4] (GPT-NER: Named entity recognition via large language models, 2023). The pros and cons of our approach are as follows:\n\nPros:\n\n\nPersonalize: Generic generative IE models commonly aim to handle IE tasks with a unified structure, in which providing personalize service would be challenging. Our method leverages entity-relative positional information to model entity spans, which can capturing both the differences and connections between entity and non-entity spans. Moreover, by incorporating predictions of all boundary offsets, our approach provides more comprehensive entity predictions, significantly improving recall performance. This is a capability not possessed by generic generative IE models.\n\n\nInference Speed: Our method utilizes BERT as the context embedding layer, which has a time complexity of O(n^2). In contrast, generative models typically are decode-only models with a time complexity of O(n^3), even with optimizations like FlashAttention and Multi-query Attention. Inference speed is crucial for NER as it constitutes a fundamental step in information extraction.\n\n\nPerformance: In the fully supervised setting, our method achieves higher performance compared to paper [3] and [4] under relatively fair conditions.\n\n\nAccessibility: Considering hardware resources (GPU) and training time costs, our method may be more easily accessible. GPT-3.5 requires the use of an API, while other open-source generative models (e.g., T5, llama) require more GPU resources.\n\nCons:\n\n\nFew-shot Capability: Paper [4] demonstrates exceptional performance in few-shot and zero-shot scenarios, an area where our method may be comparatively weaker.\n\n\nLonger Sentences: Paper [4] can process up to 4k tokens in a single instance.\n\nWe genuinely appreciate your feedback and suggestions, and we will incorporate them into our work to improve its quality. We highly appreciate your time, effort, and consideration.\n\n\n[1] Automated Concatenation of Embeddings for Structured Prediction (2021)\n\n\n[2] Dice Loss for Data-imbalanced NLP Tasks (2020)\n\n\n[3] A unified generative framework for various NER subtasks (2021)\n\n\n[4] GPT-NER: Named entity recognition via large language models (2023)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Extensive experiments on eight datasets. \n2. Good technical presentation.\n", "reasons_to_reject": "1.The authors claim that BOPN achieves state-of-the-art performance. However, it is still far from the real SOTA. e.g., \n 2021 Automated Concatenation of Embeddings for Structured Prediction. CoNLL 03 F1: 94.6\n2019 Dice Loss for Data-imbalanced NLP Tasks. Ontonotes F1 92.07\n\n2. Some recent generative models are not included. \n", "questions_to_authors": "What are the pros and cons compared to recent generative IE models?", "paper_topic_and_main_contributions": "\nBoundary Offset Prediction Network (BOPN) is proposed to predict the boundary offsets between candidate spans and their nearest entity spans. Experiments are conducted on eight widely-used NER datasets. \n", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: While the reviews have a consensus on positive soundness, there is a discrepancy in the excitement evaluation.

Regarding soundness, the paper conducts extensive experiments on 8 datasets from 2 languages, supporting the effectiveness of the proposed method.  The paper would be more insightful if it provided an error analysis.  All the reviewers see the paper is well-written.

As for excitement, the proposed span-based NER model is novel.  The result of improving recall has a practical impact.  I would lean toward the slightly positive side in excitement, mainly because of how the proposed method addresses two problems with existing span-based NER models.
Decision: Accept-Findings