A Query-Parallel Machine Reading Comprehension Framework for Low-resource NER
Authors: Yuhao Zhang
Yongliang Wang
Venue: EMNLP 2023 Findings (2023)
Keywords: NER
low-resource
in-domain transfer
cross-domain transfer

Abstract: Named entity recognition (NER) is a fundamental task in natural language processing. Recently, NER has been formulated as a machine reading comprehension (MRC) task, in which manually-crafted queries are used to extract entities of different types. However, current MRC-based NER techniques are limited to extracting a single type of entities at a time and are largely geared towards resource-rich settings. This renders them inefficient during the inference phase, while also leaving their potential untapped for utilization in low-resource settings. We suggest a query-parallel MRC-based approach to address these issues, which is capable of extracting multiple entity types concurrently and is applicable to both resource-rich and resource-limited settings. Specifically, we propose a query-parallel encoder which uses a query-segmented attention mechanism to isolate the semantics of queries and model the query-context interaction with a unidirectional flow. This allows for easier generalization to new entity types or transfer to new domains. After obtaining the query and context representations through the encoder, they are fed into a query-conditioned biaffine predictor to extract multiple entities at once. The model is trained with parameter-efficient tuning technique, making it more data-efficient. We conduct extensive experiments and demonstrate that our model performs competitively against strong baseline methods in resource-rich settings, and achieves state-of-the-art results in low-resource settings, including training-from-scratch, in-domain transfer and cross-domain transfer tasks.

A Query-Parallel Machine Reading Comprehension Framework for
Low-resource NER
Yuhao Zhang
Yongliang Wang Alibaba Group
Alibaba Group
zyh277500etaoba..cmm
Abstract
(3RC
LOC
Named entity recognition (NER) is funda- mental task in natural language processing. Re- cently, NER has been formulated as a machine reading comprehension (MRC) task, in which manually-crafte queries are used to extract entities of different types. However, current MRC- base NER techniques are limited to tracting a single type of entities at a time and are largely geared towards resource-rich set- tings. This renders them inefficient during the inference phase, while also leaving their po- tential untapped for utilization in low-resource settings. We suggest query- pereylla MRC- based approach to address these issues, which is capable of extracting multiple entity types concurrently and is applicable to both resource- rich and limited settings. Specifically, propose query--prallee encoder which uses a segeesee attention mechanism to iso- late the semantics of queries and model the query-context interaction with a unidirectional flow. This allows for easier generalization to new entity types transfer new domains. After obtaining the query and context repre- sentations through the encoder, they are fed into query conditiond biaffine predictor to extract multiple entities at once. The model is trained with parameter-efficiet tuning tech- nique, making it more data-efficient. We con- duct extensive experiments and demonstrate that our model performs competitively against strong baseline methods in resource-rich set- tings, and achieves state-of-the-art results in low-resource settings, including training- from- scratch, in-domain transfer and cross-domain transfer tasks.
1O THe
Moki
Modil
Quar
Coane
Quey
Qacy
FER
Qacry
Teearo iic Sirieg
Recae que Rost Seriigs
Lowe Kesoure
BERT-
(b) QPMRC-NER
Figure 1: Comparison between machine reading com- prehension based NER frameworks. (a) The BERT- MRC method extracts one type of entities at time and is primarily employed in standard resource-rich su- pervised settings.
(b) The proposed QPMRC-NER is capable of recognizing multiple entity types simultane- ously and is effective in both resource-rich and resource- limited settings.
and Bansal, 2016; Zhong and Chen, 2021) and in- formation retrieval (Cheng et al., 2021).
Traditionally, NER has been formalized as a quence labeling task, assigning single tag class to each token in a sentence (Chiu and Nichols, 2016; Ma and Hovy, 2016; Xia al., 2019; Devlin al., 2019; Lin et al., 2020). Recently, several new NER paradigms have been proposed, which conceptual- ize NER as span classification (Luan et al., 2019; Yu et al., 2020; Li et al., 2021: Shen et al., 2021; Fu et al., 2021a), sequence generation (Straková et al.. 2019; Yan et al., 2021; Tan et al., 2021; Paolini al., 2021; Lu al., 2022) and constituency pars- ing (Fu et al., 2021b; Lou al., 2022) tasks. spite of achieving promising performance, these ap- proaches require large amounts of well-annotated in-domain data (Lison et al., 2020; Ma et al., 2022). The data annotation usually involves carefully de- fined guideline and annotators with domain exper- tise, which could be quite time-consuming and cost- prohibitive. As result, the development of NER systems a costly endeavor in real-world scenarios where usually only a small amount of labeled data is available for new domains (I (Huang et al., 2021 I). Recently, some works reformulate NER as a ma-
1 Introduction
Named entity recognition (NER) is fundamen- tal task in natural language processing, aiming detecting the text spans that refer entities. It has been widely used in various downstream tasks, such as entity linking (Martins et al., 2019; Tedeschi et al., 2021), relation extraction (Miwa

---

chine reading comprehension (MRC) task (Li et al., 2020; Mengge et al., 2020). As illustrated in Fig- ure ((a), the input sentence is regarded as the con- text, a collection of queries that have been manu- ally crafted to represent various entity types can be
QPMRC-NER across a diverse range of NER tasks. Our model exhibits competitive performance in resource-rich settings and achieves state-of-the-art results in low-resource settings, including training- from-scratch, in-domain transfer, and cross-domain transfer tasks.
viewed as questions, then extracting entities from the context could be solved by employing ques- tion answer techniques. Prior MRC-based NER approaches limited extracting one type of en- tities a time and have largely been applied the standard supervised setting. In this context, they have exhibited superior performance in comparison to conventional sequence labeling techniques and have demonstrated comparable performance with reduced training data. Nonetheless, the extraction of singular entity type at time is not particu- larly efficient, and their potential for deployment in low-resource settings remains largely unexplored.
Method
This section presents the task formulation in Sec- tion 2.1, then introduces the proposed method QPMRC-NER, illustrated in Figure QPMRC-
NER consists of two components: the query- parallel encoder and the query-conditioned biaffine predictor, explained in Section 2.2 and Scction 2.3 respectively.
2.1 Task Formulation
Given an input sentence S = [wi, W/2 .... Wnn) with sequence length n. the goal is to extract all entities L = {< I Inni Here, Ietart and Iend indicate the start and end positions of the i-th entity span, 7:00 denotes the type of the i-th entity which belongs to a finite set of entity types E. Our method utilizes a query Q for each entity type to extract entities from the in- put sequence. Thus, the task can be formulated as extracting all entities L from S based queries {Q1, Q2...., Q g]}-
To alleviate these issues, this paper presents a query-parallel machine reading comprehension framework for named entity recognition (QPMRC- NER). is suitable for both resource-rich and resource-limited settings, and is characterized by enhanced efficiency, as illustrated in Figure 1(b). It consists of two key components: the query-parallel encoder and the query-conditioned biaffine predic- tor. The query-pur paraalll encoder takes the combina- tion of the context sentence and the queries with shared continuous prompt as input, and utilizes a query-segmented attention mechanism to sepa- rate the queries from one another and model the query-context interaction with unidirectional flow, thus facilitating easier generalization to new entity types and domain transfer. After obtaining the con- textualized representations from the query-parally encoder, we feed them to the query condiiiond biaffine predictor extract entities of multiple types simultaneously. The model is trained with parameter-efficient techniques for data-efficiency (Li and Liang, 2021; Pan et al., 2022). We con- duct extensive experiments, and in most cases, our model outperforms the present SOTA methods.
_ TTlag > >== from it.
2.2 Query-Parally Encoder
In order to extract different types of entities simul- taneously with the machine reading comprchension framework, we concatenate the input sentence and queries and feed them into transformer-hhe based encoder, from which we obtain the context word representations and entity type representations that are used for entity prediction.
Query Generation and Model Inputs Conven- tional MRC-based NER methods rely manually- crafted queries to represent each entity type, which often requires domain expertise and laboriously tuning of query words, rendering non-reusable for new entity types.
The main contributions of this paper are sum- marized as follows: (1) We introduce novel query-parallel machine reading comprehension framework QPMRC-NER that is capable of han- dling low-resource NER tasks effectively and effi- ciently, (2) Our MRC-based architecture facilitates simultaneous extraction of entities belonging to multiple categorics, resulting in faster inference speed. (3) We conduct extensive evaluations of
Rather than relying manually tuning, we con- struct queries using shared prompt prefix that is composed of a set of Iearnable vectors. This ap- proach renders the prompt more suitable for NER task and facilitates generalization when dealing with new entity types. So the query for the i-th entity type is Q₂ = {po. Pl. Pm, Ei,1, ..., Ei }. where Po: Pr, Pm represent the shared learnable

---

Condi-oondd
Eniiyy Probabiltiee for Different Spans and Enity Types
Bloffine Predictor
0.216.76
0.11
0.41 0.31
0:48
reprosentation
pos represettation
pos representatioon
reprosentaion
pos reprexenttion
pos roprescotaiion
Query-Parallel Encoder
Irput Sereence Represemaion
Queyy Representiin
BERT-basen Encoder
Token
J67.8
to'
W2
W/g
Wy 18671
P0 Pera
61,1 18e1!
Pa... Pm
62,1 222 Isö
C|E,1 188r!
Positton
4
4
12
l3 ...
bn let11
In
11 hm+1
1o 1o... km
1m+1 lm+2 Im+3.
to Km
bm+1 hm+2
Type
Ma
M6
M6--- Ub
Ub
Input Sentence
R1
Query 42
Query 4(8)
Figure 2: The overall architecture the QPMRC-NER
vectors and ei,1: Ci,n represent tokens of the i-th entity type name.
In order to address this issue, we propose the query query-segmented attention, which is employed substitute for the self-attention component in the transformer architecture. The guery-segmentee tention segregates queries from one another and models the query-context interaction with a unidi- rectional flow as depicted in Figure 3. This mecha- nism enables context tokens to only attend to other context tokens, while query tokens can attend to tokens within the same query and to tokens in the context, allowing them recognize the most per- tinent parts of the context according to the entity type.
The model inputs are illustrated in Figure We use the concatenation of context sentence and queries as the token inputs and assign the two parts with different segment ids, namely the and up. We use the absolute position of tokens within each query context sentence as the position ids, which guarantees the continuous prompt share the same position ids across queries. This precludes the oc- currence of position confliction, the queries isolated from each other by virtue of the query- segmented attention mechanism proposed below. Query-Context Interaction via Query- Segmented Attention The attention mechanism plays a critical role in modeling interactions between tokens in the transformer-based backbone. With self-attention, tokens in different queries could interact with each other. While modeling the dependencies between multiple entity types may be beneficial in some occasions, adds difficulty for generalizing to new entity types in low resource setting.
Thus, the query-segmented attention can be for- mulated as:
Q= W H
(1)
Mij {
0, allow to attend
(2)
-00,
prevent from attending
QQK Vdle
A = softmax
M (H-
(3)
where parameters W W.KK W. C Rda xda project the previous layer's output H E Rddxdh to queries, keys and values respectively. The mask matrix M {0, -in ddx controls whether two tokens attend to each other by setting its values to 0 or -n. At Xd, represents the output of the query-segmented attention.
Query
Queryz
Query3
Context
After the transformer encoding, the outputs the last-layer are used to acquire the query and
Figure 3: An illustration of Query--eeeaee Serre-Segeented Attention

---

context representations. For each query Qi the representation of its first token in the prompt is employed as the query representation ho: For each context word w/ its representation hj is obtained by aggregating the corresponding word-piece token representations through mean pooling.
Then we predict whether span starting from po- sition and ending position k belongs to certain entity type i by:
r3, -h Umhy
+ (1 + h2 @ 4n + bm
(6)
Parameter-Effiiint Tuning The query-parallel encoder is initialized with the weights of pre- trained BERT model (Devlin et al., 2019). Rather than fine-tuning the BERT encoder directly, we tuning strategy that enables effective tuning of a pretrained language model by updating only small number of extra parameters while keeping most of the pretrained parameters frozen, thereby making the tuning pro- cess more data-efficient (Li and Liang, 2021; Pan et al., 2022). Specifically, UNIPELT (Mao et al., 2022) as the parameter- efficient tuning method, which integrates several existing parameter-efficient tuning techniques as sub-modules and controls them via gating mecha- nism. Further details are in Appendix C.
1/2. sigmoid(r).)
(7)
where 42- E embedding from a learnable look-up table, Um e Rd4x1x Wm E ([2d, 1 and bm R are learnable parameters. Different from traditional bi- affine predictor, the probabilities of an entity span belong to each entity type are predicted indepen- dently without inter-type competition, thus allow- ing for the sharing of predictor parameters across different entity types, which beneficial for entity type expanding. To avoid conflict predictions flat NER tasks during inference, we only keep the span with the highest prediction probability for any overlapping entity spans.
Rd represents the (k - j)th width
adopt a parameter-
approach employs
Besides the span-level prediction, we suggest token-level auxiliary task to enhance the quality of the context and query representations. This task predicts whether a token is associated with par- ticular entity type through a feed-forward neural network as follows:
2.3 Query-Conditioned Biaffine Predictor
After obtaining the context and query representa- tions from the query perralll encoder, we feed them to an entity predictor to generate the entity predic- tions. The biaffine classifier (Yu et al., 2020) has been demonstrated be an effective approach for entity prediction in prior studies, which jointly pre- dicts the start and end positions of entity span as well as the entity types. However, the prior biaffine approaches are employed in the span prediction paradigm without leveraging the the semantics of entity type names, which limits its ability to gener- alize to new entity types.
h4t = gelu((h* @ h5) + by)
(8)
V = sigmoid(h) b))
(9)
For both the span and token level tasks, we use the binary cross entropy function as the loss metrie:
=
Σ
Σ
1] v
In this work, we propose query- conditiond biaffine predictor for entity prediction, which inte- grates the biaffine predictor into the query- peralllalllll machine reading comprehension framework and could utilizing the semantic meaning of entity type names. For each query Q; we first obtain two query-conditioned representations ht and h cor- responding to the start and end positions of entity spans through
OSISASn 3EE
+ 11 1033 = log(1 -5,5,
(10)
G= Σ Σ
ΣΣ
10 1] o3
O5jin iCE
+ + 0] log(1
v³)
(11)
neural networks:
where C. and La are span level and token level losses, 03 and Ø) the corresponding labels.
h velu(("* @ hh)5,, ba)
(4)
Then the final loss L training set D is:
A g + h3) + bc)
(5)
C C -=Ec. XCi
(12)
where gelu an activation function (Hendrycks and Gimpel, 2016), W., We 2d xd2, be. be E dd are learnable parameters.
where is a coefficient.

---

3 Experiments
English CoNLLO3
Method
Pre.
Rec.
FI
3.1 Setups
Tagger
91.93
91.54
91.73
Evaluation Tasks To verify the effectiveness of the proposed method, we conduct extensive exper- iments in both resource-rich and resource-limae -imited NER settings:
TemplateNER
90.51
93.34
91.90
PIQN
93.29
92.46
92.87
BERT-MRCT
92.47
93.27
92.87
LighNNER
92.39
93.48
92.93
BS-NER
93.61
93.68
93.65
OPMRC-NER
93.29
92.78
93.03
Standard supervised NER setting: This task evaluates the efficacy of model when ade- quate labeled data available for training.
Chinese MSRA
Method
Pre.
Rec.
F1
BARTNER
93.21
91.97
92.58
Training-from-screen scrathh low lowrresource NER set- ting: This task assess models' effectiveness under conditions of limited training data.
PIQN
93.61
93.35
93.48
BERT-MRCT
96.18
95.12
95.75
BS-NER
96.37
96.15
96.26
QPMRC-NER
96.49
95.12
95.80
In-domain low-resource NER setting: In this task, models are trained on domain where some entity types have sufficient number of labeled data while others have only limited amount of labeled data, reflecting a common scenario where new entity types emerge in existing text domain. This could demonstrate the model's capacity for in-domain transfer.
GENIA
Method
Pre.
Rec.
F1
BERT-MRCT
81.25
76.36
78.72
BARTNER
78.87
79.60
79.23
Pyramid
80.31
78.33
79.31
PIQN
83.24
80.35
81.77
OPMRC-NER
78.87
80.87
79.86
Table I: Results for standard supervised NER. means the result produced by (Yan et al., 2021)
Cross-domain low-resource NER setting: This task involves training models on resource-rich source domain, followed by fine- tuning on a target domain with different entity types and a limited amount of labeled data, demonstrating the model's capacity for cross- domain transfer.
methods as well as state-of-the-art approaches, in- cluding the classic BiLSTM-CRF (Ma and Hovy, 2016) and BERT-Tagger (Devlin et al., 2019), sin- gle query machine reading comprehension method BERT-MRC (Li al., 2020), parameter-efficien tuning method LightNER (Chen al.. 2022), template-based low-resource NER method Tem- plateNER (Cui et al., 2021), cross-domain special- ized method DoSEA (Tang et al., 2022), and sev- eral recently proposed supervised methods includ- ing Pyramid (Wang et al., 2020), BARTNER (Yan et al., 2021), PIQN (Shen et al., 2022), and Bound- ary Smoothing NER (BS-NER) (Zhu and Li, 2022). See Appendix for detailed introduction of those models.
Datasets For standard supervised NER ting, we conduct experiments on three datasets: CoNLLO3 (Tjong Kim Sang and De Meulder, 2003), MSRA (Levow, 2006) and GENTA (Ohta et al., 2002), which are English flat NER dataset, Chinese flat NER dataset and English nested NER dataset respectively. For training-from-sctt frommscratcc- low- resource setting,
the MIT Restaurant Re- view (Liu al., 2013), MIT Movie Review (Liu al., 2013), and Airline Travel Information Sys- tems (ATIS) (Hakkani-Tür et al., 2016) datasets. For the in-domain low-resource setting, we use the CoNLLO3 (Tjong Kim Sang and De Meulder, 2003). For cross-domain low-resource setting, we use the CrossNER dataset (Liu et al., 2021) which contains five diverse domains, including politics, natural science, music, literature and artificial intel- ligence.
Implementation Details We use pretrained BERT (Devlin et al., 2019) to initialize our en- coder. For a fair comparison, we use BERT- large on CoNLLO3, MIT Restaurant Review, MIT Movie Review and ATTS, BERT-base on Cross- NER, BioBERT-large (Chiu et al., 2016) on GE- BERT-BERTRWEEEE (Cui al., 2020)
NIA and on Chinese MSRA. We utilize the AdamW Opti- mizer (Loshchilov and Hutter, 2019) with a cosine annealed warm restart schedule (Loshchilov and Hutter, 2017) to train our model. In order to as-
Baselines We conduct a comprehensive compar- ison of QPMRC-NER with several representative

---

MIT Movie
MIT Restaurant
ATIS
Method
10
20
50
100
200
500
10
20
50
100
200
500
10
20
50
BERT-Tagger
25.2
42.2
49.6
50.7
59.3
74.4
21.8
39.4
52.7
53.5
57.4
61.3
44.1
76.7
90.7
TemplateNER
37.3
48.5
52.2
56.3
62.0
74.9
46.0
57.1
58.7
60.1
62.8
65.0
71.7
79.4
92.6
BERT-MRC
18.7
48.3
55.5
62.5
80.2
82.1
12.3
37.1
53.5
63.9
65.5
70.4
35.3
63.2
90.2
LighNNER
41.7
57.8
73.1
78.0
80.6
84.8
48.5
58.0
62.0
70.8
75.5
80.2
76.3
85.3
92.8
QPMRC-NER
61.8
76.3
80.2
82.7
84.5
86.2
51.1
58.9
70.3
73.2
76.4
79.0
91.9
94.4
94.8
Table 2: Results for training-from-scene fromsscratc low-resource NER
sess the model performance, we utilize span-based evaluation metrics, where entity is only consid- ered accurate if both the entity boundary and entity type correct. The Fi-score is used the metric for evaluation. See Appendix A for more detailed hyper- pperr-par settings.
Method
PER
ORG
LOC*
MISC
Avg
BERT-Tagger
75.71
77.59
60.72
60.39
69.62
TemplateNER
84.49
72.61
71.98
73.37
75.59
BERT-MRC
91.14
72.71
66.24
65.04
75.77
QPMRC-NER
95.65
85.52
84.10
74.17
86.68
Table 3: Results for in-domain low-resource NER. The
3.2 Standard Supervised NER
Avg represents micro-average Fl-score of all entity types and represents entity types with a limited amount of labeled data.
We first evaluate our method under the standard supervised NER settings on the CoNLLO3, MSRA and GENIA datasets. For the CoNLLO3, we follow (Lample et al., 2016; Yu et al., 2020; Yan et al., 2021) to train the model with the combined data from the training and development sets. A compar- ison with the state-of-the-ar methods are listed in Table I. It is evident that QPMRC-NER, which has been designed for low-resource NER, surpasses several recently proposed strong baseline models in rich-resource settings and performs comparable with the state-of-the-ar methods, suggesting that QPMRC-NER also suitable for NER tasks with abundant training data.
our method surpasses it significantly especially in cases where K is small. For instance, the F1 score of our model in 10-shot setting is either higher comparable to the F1 score of BERT-MRC in 50-shot setting across all three datasets, suggesting that our method is more suitable for low-resource NER setting, particularly in the machine reading comprehension NER paradigm.
3.4 In-domain Low-resource NER
We conduct experiments for in-domain low- resource settings on the CoNLL03 dataset, where only a limited amount labeled data is available for some entity types. Following (Cui et al., 2021), we downsample 4,001 training instances, including 3,763 "ORG". 2,496 "PER", 50 "LOC", and 50 "MISC".
3.3 Seaining-from-xcratte Low-resource NER
We analyze the efficacy of our method under the low-resource NER settings where only K samples of cach entity type are avail- able for training. Following (Cui et al., 2021), we employ the MIT Restaurant Review, MIT Movie Review and ATTS datasets for model evaluation by randomly sampling a fixed number of training instances per entity type (K=10, 20, 50, 100, 200, 500 for MIT Movie and MIT restaurant, and K=10, 20, 50 for ATIS).
The evaluation results are presented in Table
3. We observe significant performance boosts on resource-rich entity type "PER" and "ORG". as well as resource-linme -imited entity type "LOC". The performance gain on the entity type "MISC" is rel- atively smaller, which may due to the fact that the entity type name "miscellaneous entity" is more semantic ambiguous and difficult to be represented by the query compared to other entity types. Our method achieves 110.91% Fl-score improvement on average compared to baseline methods, which
The experimental results shown in Table demonstrates that method achieves significant performance boosts over baseline approaches and can more effectively utilize low-resource data. Sep- cifically, when compared to the traditional machine reading comprehension based method BERT-MRC,

---

Method
Pol.
Sci.
Mus.
Lite.
AI.
Avg
BILSTM-CRF
53.89
49.12
43.65
41.87
43.18
46.34
BERT- BERT-Tagger
66.56
63.73
66.59
$9.95
50.37
61.44
TemplateNER
65 41
62.93
64.67
64.55
57.64
63.04
LightNER
71.63
65.55
71.62
65.8
57.46
66.41
BERT-MRC
70.23
67.25
70.64
62.53
62.77
DoSEA
75.52
71.69
73.10
68.59
70.99
QPMRC-NER
77.06
76.90
78.34
69.40
66.33
73.61
Table 4: Results for cross-domal low-resource NER. The Avg represents macro-average Fl-score of all domains.
Music (Source: None)
Music (Source: CoNL.O03
Method
Prec.
Rec.
F1
Prec.
Rec.
F1
QPMRC-NER
81.50
70.40
75,55
83.58
73.72
78.34
w/o continuous prompt
82.16
66.52
73.51
82.52
71.18
76.43
w/o gs-attention
78 18.45
61.70
69.08
83.21
73.83
78.24
w/o qs-attention:
81,11
61.70
70.09
80.25
61.59
69.69
w/o query-parallel biaffine
81,36
66.49
73.18
82.14
72.29
76.90
w/o auxiliary task
84.16
75.00
83.60
73.24
78.07
Table 5: Results of ablation studies. The experiment is conducted on the CrossNER music domain dataset with vaining-frome fcrahhc and cross-domain low-resource settings. The "qs-attention," and "qs-attentiony" denote two variation models in query-segmented attention ablation study described in 3.7.
demonstrates it is more suitable for in- in-oomiin trans- fer and handling new emerging entities.
Datasets
Entity Type
BERT-MRC QPMRC-NER
MIT Movie
1.00x
1.32x
ATIS
79
1.00x
2.16x
3.5 Cross-domain Low-resource NER
We explore the cross-domain low-resource NER settings in which models are initially trained in resource-rich source domain and then fine-tuned and evaluated in target domain with limited la- beled data. In order evaluate the models dif- ferent target domains with domain specific entity types, the CrossNER dataset used as specifi- cally designed for this task. It uses the CoNLL03 as the source domain and covers five distinct target domains, namely music, literature, artificial intel- ligence, politics, and natural science. There are 100 training instances for the first three domains and 200 instances for the last two domains. The empirical results Table 4 indicate our approach surpasses existing state-of-the--rt methods all target domains, suggesting that our model more suitable for cross-domain knowledge transfer.
Table 6: Results for inference speedup. Experiments are conducted on NVIDIA Tesla V100 Graphics Card with 32GB graphical memory.
respective speedup of 1.32 and 2.16x. The ob- served acceleration in inference speed is attributed to the query parallel setting employed by QPMRC- NER, which enables single forward pass for all entities, opposed to BERT-MRC's requirement of a separate forward pass for each entity. The de- gree of runtime improvement is contingent upon the number entity types present in the dataset, with datasets featuring a greater number of entity types exhibiting greater potential for leveraging the query parallel setting.
3.7 Ablation Studies
3.6 Inference Speed
We conduct ablation studies to analyze the effects of different components of our model and validate the design decisions. Specifically, four settings are evaluated in the ablation studies: (1) Shared
Table 6 presents the results of our method's infer- ence speedup in comparison to BERT-MRC on the
MIT Movie and ATIS datasets, demonstrating a

---

Prompt: In our model, a query representing an en- tity lype composed by combining the shared con- tinuous prompt with the entity type name. We ab- late the shared continuous prompt to testify whether it is helpful for entity type representation and knowledge transfer across domains. (2) Query- Segmented Attention: The query-segmented at- tention used in the muery-parallel encoder isolates the queries from each other and allows queries at- tend to the context but not vice versa. We verify the effectiveness of guery-segmenter attention through two variations by relaxing the constraints. The first variation permits the context tokens to attend to the queries, thereby enabling the semantics of queries to interact indirectly through the context. The sec- ond one applies vanilla self-attention directly, and the first token of each entity type name is utilized to represent the queries. (3) Query-Conditioned Bi- affine Predictor: To evaluate the effectiveness of the query-conditioned biaffine predictor utilized in our model, we conduct an experimental analysis by removing it and only employing word-level classifi- (4) Auxiliary Task: We access the effectiveness of the word level classification auxiliary task by removing it and only use L. as training object.
attention mechanism outperforms the vanilla self- attention mechanism by a significant margin of 8.65% the cross-domain setting. Additionally, the query-conditioned biaffine predictor, designed for the query parallel NER framework, achieves su- perior performance compared to using word-level classification directly. Also, incorporating word- level classification an auxiliary task can further enhance the model's ' performance.
Related Work
Although NER is usually formalized as sequence labeling task (Chiu and Nichols, 2016; Ma and Hovy, 2016; Xia et al., 2019; Devlin et al., 2019), several new NER paradigms have been proposed recently, conceptualizing NER as span classifica- tion (Luan et al., 2019; Yu et al., 2020; Li et al., 2021; Shen et al., 2021; Fu et al., 2021a), sequence generation (Straková et al., 2019; Yan et al., 2021; Tan et al., 2021: Lu et al., 2022), constituency pars- ing (Fu et al., 2021b; Lou al., 2022) and machine reading comprehension (Li et al., 2020; Mengge et al., 2020) tasks and achieving impressive results. However, these approaches are mainly focus on standard supervised setting, which is not suitable for low-resource scenarios. Another line of re- search sought to address the low-resource NER task using techniques such as prototype-based learning (Fritzler et al., 2019; Yang and Katiyar, 2020; Hen- derson and Vulié, 2021), template-basede learning (Cui et al., 2021) and contrastive learning (Das et al., 2022). But they often fail to fully exploit the potential of pretrained language models (PLMs) and perform inferior to standard sequence labeling NER methods in resource-riee rich settings. To bridge the gap, we propose machine reading compre- hension based method, which is effective in both resource-rich and resource-limitee NER settings.
cation for named entity recognition.
This experiment is conducted in both training- from-scratch and cross-domain low-resource set- tings to assess the models' capacity to utilize small amounts of data and transfer knowledge across do- mains. The CrossNER music domain data is em- ployed for evaluation. The experimental results, as presented in Table 5, demonstrate that the pro- posed model exhibits superior performance in the cross-domain settings as compared to the training- from-scratch setting, suggesting its ability to trans- fer knowledge from the source domain to the target domain, therehy enhaneing its overall performance. Notably, QPMRC-NER outperforms its ablation variations in both settings. The experimental re- sults indicate that shared continuous prompting is beneficial in improving query representation qual- ity. Furthermore, the query-segmented attention mechanism is found to be more efficient than other attention mechanisms in our query-pant ppraalll MRC- based framework. This may be attributed to its ability to leverage the shared query prompt while avoiding modeling entity type interaction with low- resource data, which can have negative effects, par- ticularly in the cross-domain setting. As evidenced by the experimental results, the query-segmented
Conclusion
In this paper, we propose a query-parallel machine reading comprehension framework, which predicts all entities simultaneously and is applicable to both resource-rich and resource-limites settings. Specifically, introduce the query-parally en- coder, which leverages the query-segmentedd atten- tion mechanism to facilitate more straightforward generalization to new entity types. Additionally, we propose the query-conditioned biaffine predictor, which enables parallel prediction of entities. The

---

model is trained with parameter-efficient technique for data-efficiency. Extensive experimentation has shown that our approach attains a faster inference speed, exhibits competitive performance against strong baselines in resource-rich setting, and yields ofatc-of-the-art outcomes in low-resource contexts, including scraach in-domain trans- fer, and cross-domain transfer tasks.
Billy Chiu, Gamal Crichton, Anna Korhonen, and Sampo Pyysalo. 2016. How to train good word em- beddings for biomedical NLP. In Proceedings of the 15th Workshop on Biomedical Natural Language Processing, pages 166-174, Berlin, Germany, Asso- ciation for Computational Linguisties.
Jason P.C. Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional LSTM-CNNS. Trans- actions of the Association for Computational Linguis- tics, 4:357-370.
Limitations
Leyang Cui, Yu Wu, Jian Liu, Sen Yang. and Yue Zhang. 2021. Template-based named entity recognition us- ing BART. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021. pages 1835-1845, Online. Association for Computational Linguisties.
In this work, we propose a query-parallel machine reading comprehension framework for NER task, which extracts multiple entity types simultaneously and achieve promising results for both resource- rich and essource-limites settings. In this approach, each query is semantically isolated, responsible for giving semantic signals the pretrained lan- guage model and extracting entities of its type. The method is effective in low- www-resurre settings when the entity type name is semantically unambiguous, but encounters difficulties otherwise, such as facing the miscellaneous other-class words. Thus, further research is needed to determine how to address these entity types and improve the performance of the model. Moreover, in QPMRC-NER, the input length restriction imposed by pretrained language models limits the number of parallel processed queries. To handle fine-grained entity extraction with a large number of entity types, segmentation of queries into multiple groups and separate encod- ing becomes necessary. Another potential approach is using a lightweight model to filter out irrelevant entity types for each sentence, thereby retaining only a small subset of entity type candidates. This avenue of investigation is left for future research.
Yiming Cui, Wanxiang Che. Ting Liu, Bing Qin, Shijin Wang. and Guoping Hu. 2020. Revisiting pre-trained models for Chinese natural language processing, In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 657-668, Online, As- sociation for Computational Linguisties.
Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca Passonneau, and Rui Zhang. 2022. CONTaiNER: Few-shot named entity recognition via contrastive learning, In Proceedings of the 60th Anmual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 6338-6353, Dublin, Ireland. Association for Computational Linguisties.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186, Minncapolis, Minnesota. Association for Computational Linguistics.
Alexander Fritzler, Varvara Logacheva, and Maksim Kretov, 2019. Few-shot classification in named en- tity recognition task. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, SAC 19, page 993-1000, New York, NY, USA. As- sociation for Computing Machinery.
References
Xiang Chen, Lei Li. Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si. Huajun Chen, and Ningyu Zhang. 2022. LightNER: A lightweight tuning paradigm for low-resource NER via plug- gable prompting. In Proceedings of the 29th Inter- national Conference on Computational Linguistics, pages 2374-2387, Gycongju, Republic of Korea. In- ternational Committee on Computational Linguistics.
Jinlan Fu, Xuanjing Huang. and Pengfei Liu. 2021a. SpanNER: Named entity re-/recognition span pre- diction. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the I 11th International Joint Conference Natu- ral Language Processing (Volume I: Long Papers), pages 7183-7195, Online. Association for Computa- tional I. Linguistics.
Xiang Cheng, Mitchell Bowden, Bhushan Ramesh Bhange, Priyanka Goyal, Thomas Packer, and Faizan Javed. 2021. An end-to-end solution for named en- tity recognition in ecommerce search. Proceedings of the AAAI Conference on Artificial Intelligence, 15(17):
Yao Fu, Chuanqi Tan, Mosha Chen, Songfang Huang. and Fei Huang. 2021b. Nested named entity recog- nition with partielly-observed treecrfs. Proceedings of the AAAI Conference on Artificial Intelligence. 35(14): 1839-12847.

---

Demi Guo, Alexander Rush, and Yoon Kim. 2021. Parameter- efficieen transfer learning with diff prun- ing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing (Volume 1: Long Pupers), pages 4884-4896, Online. Association for Computational Linguistics.
Fei Li, ZhiChao Lin, Meishan Zhang, and Donghong Ji. 2021. A apan-based model for joint overlapped and discontinuous named entity recognition. In Proceed- ings the 59th Annual Meeting of the Association for Computational Linguistics and the 1 lik International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 48 14-4828, Online. Association for Computational Linguistics.
Dilek Hakkani-Tii Tür. Gokhan Tur, Asli Celikyilmaz, Yun- Nung Chen, Jianfeng Gao, Li Deng, and Ye-Y Yi Wang. 2016. Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM. In Proc. Inter- speech 2016, pages 715-719.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguisties and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 4582- 4597, Online. Association for Computational Lin- guistics.
Matthew Henderson and Ivan Vulié. 2021. Von: Data-efficient and few-shot slot labeling. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3375-3389, Online. Association for Computational Linguistics.
Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020. A unified MRC framework for named entity recognition. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5849-5859, On- line. Association for Computational Linguisties.
Dan Hendrycks and Kevin Gimpel. 2016. Bridging non- linearities and stochastic regularizers with gaussian error linear units. CoRR, abs/1606.08415.
Bill Yuchen Lin, Dong-Ho Lee, Ming Shen, Ryan Moreno, Xiao Huang, Prashant Shiralkar, and Xi- ang Ren. 2020. TriggerNER: Learning with entity triggers as explanations for named entity recogni- tion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8503 8511, Online. Association for Computational Linguistics.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficienn transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2790-2799. PMLR.
Pierre Lison, Jeremy Barnes, Aliaksandr Hubin, and Samia Touileb. 2020. Named entity recognition with- out labelled data: A weak supervision approach. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1518- 1533, Online. Association for Computational Linguis- tics.
Edward Hu, Yelong Shen, Phil Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Lu Wang, and Weizhu Chen. 1021. Lora: Low-rank adaptation of large language models.
Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao. and Jiawei Han. 2021. Few- shot named entity recognition: An empirical baseline study. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 10408-10423, Online and Punta Cana, Dominican Republic. Association for Computational Linguisties.
Jingjing Liu, Panupong Pasupat, Scott Cyphers, and Jim Glass. 2013. Asgard: A portable architecture for multilingual dialogue systems. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages
Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pas- cale Fung. 2021. Crossner: Evaluating cross-domain named entity recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15): 13452- 13460.
Guillaume Lample, Miguel Ballesteros, Sandeep Sub- ramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 260-270, San Diego, Califoriia Association for Computational Linguistics.
Ilya Loshchilov and Frank Hutter. 2017. SGDR:
stochastic gradient descent with warm restarts. In 5th International Conference
Learning Representa- tions, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Gina-Anne Levow, 2006. The third international Chi- nese language processing bakeoff: Word segmenta- tion and named entity recognition. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, pages 108-117, Sydney, Australia, Asso- ciation for Computational Linguistics.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA. May 6-9, 2019. OpenRe- view.net.

---

Chao Lou, Songlin Yang, and Kewei Tu. 2022. Nested named entity recognition as latent lexicalized con- stituency parsing. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Pupers), pages 6183-6198, Dublin, Ireland. Association for Computational Lin- guistics.
Makoto Miwa and Mohit Bansal. 2016. End-to and-to-end re- lation extraction using LSTMs on sequences and tree structures, In Proceedings of the 54th Annual Meet- ing of the Association for Computational Linguistics (Volume I: Long Pupers), pages 105-1116, Berlin, Germany, Association for Computational I Linguistics.
Tomoko Ohta, Yuka Tateisi, and Jin-Dong Kim. 2002. The genia corpus: An annotated research abstract cor- pus in molecular biology domain. In Proceedings of the Second International Conference on Human Lan- guage Technology Research, HLT '02, page 82-86. San Francisco, CA, USA. Morgan Kaufmann Pub- lishers Inc.
Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022. Uni- fied structure generation for universal information extraction. Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5755-5775 Dublin, Ireland. Association for Computational Linguistics.
Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hongsheng Li. 2022. St-adapter: Parimeter-efficient image-to-video transfer learning. In Advances in Neural Information Processing Systems, volume 35, pages 06462-26477. Curran Associates, Inc.
Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019. A general framework for information extraction using dynamic span graphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 3036-3046, Minncapolis, Minnesota. Association for Computational Linguistics.
Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Ci- cero Nogueira dos Santos, Bing Xiang, and Stefano Soatto. 2021. Structured prediction as translation between augmented natural languages. In 9th Inter- national Conference on Learning Representations, ICLR 2021.
Jie Ma. Miguel Ballesteros, Srikanth Doss, Rishita Anubhai, Sunil Mallya, Yaser Al-Onaizan, and Dan Roth. 2022. Label semanties for few shot named entity recognition. In Findings of the Association for Computational Linguistics: ACL 2022. pages 1956- 1971, Dublin, Ireland. Association for Computational Linguistics.
Yongliang Shen, Xinyin Ma. Zeqi Tan, Shuai Zhang, Wen Wang. and Weiming Lu. 2021. Locate and la- bel: A two-stage identifier for nested named entity recognition. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the I 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 2782-2794 Online. Association for Computa- tional Linguistics.
Xuezhe Ma and Eduard Hovy. 2016. End-to -nd se- quence labeling via bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the As- sociation for Compututional Linguistics ( (Volume I: Long Pupers), pages 1064-1074, Berlin, Germany. Association for Computational Linguistics.
Yongliang Shen, Xiaobin Wang. Zeqi Tan, Guangwei Xu, Pengjun Xie, Fei Huang, Weiming Lu, and Yuet- ing Zhuang. 2022. Parallel instance query network for named entity recognition. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages 947-961, Dublin, Ireland. Association for Computa- tional Linguistics.
Yuning Mao, Lamhert Mathias, Rui Hou, Amjad Alma- hairi, Hao Ma, Jiawei Han, Scott Yih, and Madian Khabsa. 2022. UniPELT: A unified framework for parameter-efficieet language model tuning. In Pro- ceedings of the 60th Annual Meeting of the Associa- tion for Computational Linguistics (Volume I: Long Papers), pages 6253-6264, 6264, Dublin, Ireland. Associa- tion for Computational Linguistics.
Jana Straková, Milan Straka, and Jan Hajic. 2019. Neu- ral architectures for nested NER through lineariza- tion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages Florence, Italy. Association for Compu- tational Linguistics.
Pedro Henrique Martins, Zita Marinho, and André F. T. Martins. 2019. Joint learning of named entity recog- nition and entity linking. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 190- 196, Florence, Italy, Association for Computational Linguistics.
Zeqi Tan, Yongliang Shen, Shuai Zhang, Weiming Lu, and Yueting Zhuang. 2021. A sequence-to-se net- work for nested named entity recognition. In Pro- ceedings of the 30th International Joint Conference on Artificial Intelligence, IJCAI-21.
Xue Mengge, Bowen Yu, Zhenyu Zhang, Tingwen Liu, Yue Zhang, and Bin Wang. 2020. Coarse-to-Fine Pre- training for Named Entity Recognition. In Proceed- ings of the 2020 Conference on Empiriaal Methods in Natural Language Processing (EMNLP), pages 6345-638 63544, Online. Association for Computational Linguistics.
Minghao Tang, Peng Zhang, Yongquan He, Yongxiu Xu, Chengpeng Chao, and Hongbo Xu. 2022. DoSEA: A -onnafc-peccfic antity-aware framework for cross- domain named entity recogition. In Proceedings of

---

the 29th International Conference on Computational Linguistics, pages 145,, Gyeongju, Republic of Korea, International Commitiee Computational Linguisties.
Enwei Zhu and Jinpeng Li. 2022. Boundary smooth- ing for named entity recognition. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 7096-7108, Dublin, Ireland. Association for Computational Linguistics.
Simone Tedeschi, Simone Conia, Francesco Cecconi, and Roberto Navigli. 2021. Named Entity Recog- nition for Entity Linking: What works and what's next. Findings of the Association for Computa- tional Linguistics: EMNLP 2021. pages
Punta Cana, Dominican Republic. Association for Computational Linguistics.
Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction the CoNLL-2003 shared task: -egguagg-iddeeeddet named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning HLT-NAACI. 2003, pages 142- 147.
Jue Wang, Lidan Shou, Ke Chen, and Gang Chen, 2020. Pyramid: A layered model for nested named entity recognition. In Proceedings the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 5918 5928,, Online. Association for Computa- tional Linguisties.
Congying Xia, Chenwei Zhang, Tao Yang. Yaliang Li, Nan Du, Xian Wu, Wei Fan, Fenglong Ma, and Philip Yu. 2019. Multi- muati-grained named entity recognition. Proceedings of the 57th Annual Meeting of the As- for Computational Linguistics. pages 1430- 1440, Florence, Italy. Association for Computational Linguistics.
sociation for
Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, and Xipeng Qiu. 2021. A unified generative framework for various NER subtasks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1ith International Joint Conference Natural Language Processing (Volume 1: Long Papers), pages 5808-5822, Online. Association for Computational Linguistics.
Yi Yang and Arzoo Katiyar. 2020. Simple and effective few-shot named entity recognition with structured nearest neighbor learning. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 6365-6377 6375, Online. Association for Computational Linguistics.
Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named entity recognition dependency parsing. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 6470- 6476, Online. Association for Computational Lin- guistics.
Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 61, Online. Association for Computational Linguistics.

---

A Hyper-parameters
LightNER (Chen al., 2022) formulates NER as an entity span sequence generation task. It adopts a sequencee-o-ssequenee model with pointer and pluggable prompting mechanism to tackle low- resource NER task.
The detailed syper-parameten used our model are listed in Table 7. For the parameter-efficient tuning module UNIPELT (Mao et al., 2022), we adopt the original hyper-parameter parameter settings.
PIQN (Shen et al., 2022) treats NER as span prediction problem by setting up learnable instance queries extract entities from sentence simulta- neously.
Hyper-parameeter
Value
General
predictor hidden size
300
DoSEA (Tang et al., 2022) treats NER as a ma- chine reading comprehension based framework and is designed for cross-domain NER task. It is able to recognize distinctions that are domain-specift and mitigate the subtype conflicts between domains. Boundary Smoothing NER (BS-NER) (Zhu and Li, 2022) treats NER as a span prediction prob- lem and proposes the boundary smoothing regu- larization technique to boosting the model perfor- mance.
prefix length
10
adapter bottleneck size
48
LoRA rank
8
prompt length
10
learing rate
[8c-5, Ie-4, 1.2c-4]
batch
auxiliary task coefficent
[0.1. 11
warmup ratio
Resource-limited Setting
steps
[10000, 20000)
restart steps
[2000, 3000]
Resource-rich Setting
C Parameter-Efficien Tuning with UNIPELT
epochs
100
restart epoch
The UNIPELT (Mao et al., 2022) utilized in our model as the parameter-efficien transfer learning component as illustrated Figure 4, as is able to adapt to the data or task setup dynamically. It in- corporates three existing parameter-efficient tuning methods including Adapter (Houlsby et al., 2019), Frefix-tuning (Li and Liang, 2021) and LoRA (Guo et al., 2021; Hu et al., 2021) sub-modules and controls them via gating mechanism.
Table 7: Hyper-parameters for model
B Baselines
We use the following methods baselines: BiLSTM-CRF (Ma and Hovy, 2016) treats NER sequence labeling task utilizing BiLSTM and CRF.
BERT-Tagger (Devlin et al., 2019) treats NER as a sequence labeling task utilizing BERT.
BERT-MRC (Li et al., 2020) formulates NER as a machine reading comprehension task. It extracts entities from the context based on manually-crafted queries representing different entity types.
Adaper
Crt
Adé & Noens
Pyramid (Wang et al., 2020) layered neural model for nested entity recognition, which permits each decoding layer to consider the global informa- tion from both the upper and lower layers.
FeolForwsed
Add & Norrs
Inidieational Mulishesl Attenion
BARTNER (Yan et al., 2021) formulates NER an entity span generation task utilizing model with pointer mech-
Profa Tiring
Gato
anism.
TemplateNER (Cui et al., 2021) is template- based method which treats NER a language model ranking problem within a sequence-to- sequence framework and designed for low- resource NER.
Figure Illustration of the UNIPELT-cnhanced trans-
former layer used in the query-parallel encoder

---

Adapter (Houlsby al., 2019) augments the Transformer layer of the pretrained language model with a trainable bottleneck layer. This layer con- sists of pair of down+up projections which reduce and then recover the size of the token hidden states. Mathematically, it could be denoted as:
AA W d(W up down Hp)
(13)
Prefix-tuning (Li and Liang, 2021) enhances the multi-head self-attention in each Transformer layer by adding of number of trainable vectors to the input, allowing the original tokens to attend to virtual tokens if they are real.
LoRA (Guo et al., 2021; Hu al., 2021) intro- duces trainable low-rank matrices and utilizes them to update the Query Q and Value V in multi-head self self-attention in each Transformer layer, which can be formulated as:
Q (Wo aig T W in
down
(14)
V (Wyy ar W W Hi
(15)
Gating Mechanism The UNIPELT adds trainable gate Gm, for each sub-module m, E {Adapter, Prefial uning, LoRA} in the Trans- former layer to achieve fine-grained control over these sub-modules. The gate output would be higher if its corresponding sub-module is more useful for the task.
Specifically, for adapter, we feed its direct input HF a feedforward network get the gating estimation 9A E (0,1), 1). Then the output of adapter with gating mechanism HA would be:
HAG 9AHA+ Hy
(16)
Similarly, for prefix-tuning gating function Gp is applied to the prefix vectors Pк and Pv. As for LoRA, the hyper-parameter a is substituted by the gating function GL-


Reviews:
[{"review_id": "AFzculOKbu", "reviewer": "Reviewer_Nr7p", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for dedicating your time to reviewing our paper and offering us invaluable feedback.\n\nQ: Some recent works need to be discussed and more works need to be compared in experiments. e.g. PromptNER, COPNER, LightNER \n \nA: Thanks very much for bring to our attention some recent works.\n\nLightNER: LightNER is a state-of-the-art method for low-resource named entity recognition. We have included LightNER as one of our baseline methods, as demonstrated in the Baselines section of our paper (Section 3.1). In order to evaluate the efficacy of our method, we compare it against LightNER in both resource-rich and low-resource settings, and our method outperforms LightNER in both scenarios. \n\nPromptNER: PromptNER is an NER method that integrates entity locating and entity typing through prompt learning. We did not incorporate PromptNER as a baseline in our study because it was published in ACL 2023, which is subsequent to our submission to EMNLP 2023. \nThere are notable distinctions between our method and PromptNER. In PromptNER, each entity is predicted using a single prompt, necessitating the estimation of the maximum number of entities per sentence in advance. Moreover, PromptNER employs a complex dynamic template filling mechanism to establish the correspondence between prompts and entities. Conversely, our approach employs a distinct methodology where each prompt is allocated for the prediction of entities associated with a specific type, thus mitigating the complexities mentioned earlier. \nFortunately, we share the same experimental settings as PromptNER for training-from-scratch low-resource NER and in-domain low-resource NER, following a previous work TemplateNER (Cui et al. 2021). The comparative results are shown below.\n\nResults for training-from-scratch low-resource NER:   \n| Method     | MIT Movie 10 | MIT Movie 20 | MIT Movie 50 | MIT Movie 100 | MIT Movie 200 | MIT Movie 500 | MIT Restaurant 10 | MIT Restaurant 20 | MIT Restaurant 50 | MIT Restaurant 100 | MIT Restaurant 200 | MIT Restaurant 500 | ATIS 10 | ATIS 20 | ATIS 50 |\n|--------------|--------------|--------------|--------------|----------------|----------------|----------------|--------------------|--------------------|--------------------|---------------------|---------------------|---------------------|---------|---------|---------|\n| PromptNER  | 55.6         | 68.2         | 76.5         | 80.4           | 82.9           | 84.5           | **56.1**           | **62.6**           | 69.3               | 71.3                | 74.4                | 77.4                | 91.5    | 94.3    | **95.5**    |\n| QPMRC-NER  | **61.8**     | **76.3**     | **80.2**     | **82.7**       | **84.5**       | **86.2**       | 51.1               | 58.9               | **70.3**           | **73.2**            | **76.4**            | **79.0**                | **91.9** | **94.4** | 94.8    |\n\nResults for in-domain low-resource NER:  \n\n| Method     | PER   | ORG   | LOC*  | MISC* | Avg.   |\n|------------|-------|-------|-------|-------|--------|\n| PromptNER  | 76.96 | **88.11** | 82.69 | 62.89 | 79.75  |\n| QPMRC-NER | **95.65** | 85.52 | **84.10** | **74.17** | **86.68** |\n\nAdditionally, our compare our method against PromptNER on the supervised NER setting using the CoNLL03 dataset with BERT-large encoder. The comparative results are presented as follows:  \n| Method |  Precision | Recall | F1 |\n|---------|---------------|---------------|---------------|\n| PromptNER  | 92.48                    | 92.33                 | 92.41             |\n| QPMRC-NER  | **93.29**                | **92.78**             | **93.03**         |\n\nWe can see that in general, our QPMRC-NER method performs either higher or comparable to PromptNER.\n\nCOPNER: COPNER is an approach for few-shot NER that utilizes distance metric learning. It employs class-specific words as metric referents to infer the entity types of test tokens by comparing them with these referents. Our proposed method, QPMRC-NER, distinguishes itself from COPNER in several ways. QPMRC-NER adopts a machine reading comprehension-based approach to accurately predict entity spans directly. In contrast, COPNER relies on a metric learning-based approach that predicts the entity type of each token individually, without explicitly verifying entity boundaries. Consequently, COPNER necessitates the inclusion of the Viterbi decoding approach proposed in Structshot (Yang and Katiyar, 2020) to enhance the accuracy of its predictions.\nIn our study, we adopt experimental setups similar to those employed in TemplateNER (Cui et al., 2021) and LightNER (Chen et al., 2022). These setups differ from those utilized in COPNER. \nGiven that the training-from-scratch low-resource Named Entity Recognition (NER) setting is largely similar, we proceed to conduct experimental comparisons on the MIT-Movie dataset. Through these comparisons, our method exhibits superior performance in comparison to COPNER.\n| Method     | MIT Movie 10 | MIT Movie 20 | MIT Movie 50 | MIT Movie 100 | MIT Movie 200 | MIT Movie 500 | \n|------------|--------------|--------------|--------------|----------------|----------------|----------------|\n| COPNER  | 59.9         | 67.2         | 74.1         | 77.8           | 80.1          | 81.7           | \n| QPMRC-NER  | **61.8**     | **76.3**     | **80.2**     | **82.7**       | **84.5**       | **86.2**       | \n\n\n___\n\nQ: More comparisons are needed to elucidate the difference and advantages with PIQN.  \n\nA: PIQN is a Named Entity Recognition (NER) method that utilizes global and learnable instance queries to extract entities from sentences in a parallel manner. In contrast, our proposed QPMRC-NER differs from PIQN in terms of both algorithm design and application tasks.\nIn PIQN, each entity is predicted by a specific prompt, similar to the approach employed in PromptNER. Consequently, it involves estimating the maximum number of entities per sentence and undergoing a dynamic label assignment process. In contrast, QPMRC-NER adopts a different approach by assigning each prompt to represent entities associated with a specific type, thereby avoiding such issues altogether.  \nMore importantly, it is worth noting that PIQN is primarily designed for supervised NER settings and not be suitable for low-resource NER tasks. We conducted tests on PIQN within various low-resource scenarios and found that its performance did not match that of other low-resource NER methods, such as BERT-MRC and LightNER. Consequently, we did not include PIQN as a baseline method for low-resource NER tasks. Instead, we employed representative and state-of-the-art methods from previous research for comparison purposes.  \nSince QPMRC-NER can also be applied to standard NER settings, we included PIQN as one of the supervised NER baseline methods in our study. As indicated in Table 1, our method achieves superior or comparable performance compared to PIQN in this context.\n___\n\nQ: Include more methods for inference speed comparison  \n\nA: The proposed QPMRC-NER is a machine reading comprehension (MRC) based method that aims to enhance performance in both resource-rich and resource-limited NER tasks, while also improving inference speed compared to conventional MRC-based methods such as BERT-MRC.\nGiven that different paradigms of NER methods like sequence labeling, machine reading comprehension, distant metric learning, and sequence generation can significantly impact inference speed, our primary comparative analysis centers around methods operating within the machine reading comprehension paradigm, specifically focusing on BERT-MRC. Notably, our proposed method demonstrates a significant speed improvement, achieving up to a 2x speedup when compared to BERT-MRC.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The approach proposed by the paper is rather novel to me. This authors proposed entity type-specific query-parallel network on top of MRC-based NER method and PIQN method. It avoids both the multiple rounds of querying in MRC-based methods and the dynamic loss designment during training of PIQN network.\n2. The experimental setup is sufficient and a variety of settings are considered. Although this method did not achieve the best results on some datasets, it also proved to have promising performance in different settings.", "reasons_to_reject": "1. Some recent works need to be discussed and more works need to be compared in experiments. e.g.  PromptNER, COPNER, LightNER\n2. The Inference Speed comparison is inadequate, and I would like to see more methods such as COPNER and PIQN.\n3. More comparisons are needed to elucidate the difference and advantages with PIQN.\n\n+ COPNER: Contrastive Learning with Prompt Guiding for Few-shot Named Entity Recognition (Huang et al., COLING 2022)\n+ LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting (Chen et al., COLING 2022)\n+ PromptNER: Prompt Locating and Typing for Named Entity Recognition (Shen et al., ACL 2023)", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposed a query-parallel approach to enhance the MRC-based NER methods, which can extract multiple entity types concurrently and is applicable to both resource-rich and low-resource settings. The query-parallel encoder used a unidirectional flow to encode the query and context. And a query-conditioned biaffine predictor to predict entities of all types simultaneously. The authors conducted extensive experiments in resource-rich setting and low-resource settings and achieved superior perfornance.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pxexnaSjRh", "reviewer": "Reviewer_Km8b", "summary": "", "questions": "- On lines 106-107, it says, \"The model is trained with parameter-efficient techniques for data-efficiency.\" Parameter-efficient techniques are used to reduce memory requirements, no? I do not understand what \"data-efficiency\" is referring to here.\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for dedicating your time to reviewing our paper and offering us invaluable feedback.\n\nQ: It is a little unclear to me how this would scale to a setting of fine-grained entity extraction (https://ojs.aaai.org/index.php/AAAI/article/view/8122). At the very least, it would be helpful if the authors added a comment to the limitations section about how this would scale to cases where the number of entity types is large (e.g. 10s or even 100s).\n\nA: Thank you for providing valuable suggestions. In the QPMRC-NER approach, the context sentence and queries representing different entity types are concatenated and fed into a BERT-like encoder. Consequently, the simultaneous prediction of entity types is constrained by the maximum input length allowed by the pretrained language model. However, our method is capable of accommodating a larger number of entity types (e.g., the ATIS dataset used in our study consists of 79 entity types) by utilizing shorter continuous prompts. In cases where the entity types are even larger, we can segment them into multiple parts and feed them into the encoder separately. Nevertheless, additional post-processing steps are required to prevent conflicting predictions in the flat NER task. Alternatively, employing a lightweight model to pre-filter irrelevant entity types for each sentence and retaining only a small subset of entity type candidates could be a considerable approach. However, this avenue of investigation is left for future research. We will provide a more comprehensive discussion on the scalability of QPMRC-NER for fine-grained entity extraction in the final version of our paper.\n___\n\nQ: On lines 106-107, it says, \"The model is trained with parameter-efficient techniques for data-efficiency.\" Parameter-efficient techniques are used to reduce memory requirements, no? I do not understand what \"data-efficiency\" is referring to here.\n\nA: In the context of our study, data efficiency pertains to a model's capacity to achieve a satisfactory level of generalization performance with a limited quantity of data. Employing parameter-efficient techniques, as opposed to directly fine-tuning the model, can mitigate memory requirements while simultaneously enhancing the effectiveness of data utilization.\n___\n\nQ: I don't understand what the \"Transfer\" arrow in Figure 1 is trying to show. It almost appears to point from \"Context\" to \"ORG/query\". Could the authors make it more explicit when/how the transfer happens? Maybe updating the figure caption to explain.\n\nA: In Figure 1, the \"Transfer\" arrow is intended to depict the potential advantages that the proposed QPMRC-NER approach can obtain through the transfer of knowledge from a resource-rich domain to a resource-limited domain. This transfer include in-domain and cross-domain transfer scenarios that are discussed in Section 3.4 and 3.5 in the paper. In our final version, we will enhance the clarity of the figure to better convey this concept.\n___\n\nQ: \"Unidirectional attention\" seems like a poor name, as it suggests a mechanism like the causal or masked self-attention of transformer decoders in language modeling. I also don't clearly understand how the modification in this paper is \"unidirectional\". Something like \"query-segmented attention\" would be more immediately understandable (IMO) and avoids the erroneous connection to causal attention.\n\nA: Thank you for the suggestion. We will revise the term \"Unidirectional attention\" to a more immediately understandable name.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The MRC-based framework for NER is a popular approach. Research on this approach will greatly benefit from the query-parallel modifications proposed in this paper.\n- The authors are careful to benchmark on many datasets, from different languages and domains, in resource-rich and resource-poor settings. Their model performs comparably or better in almost all cases, significantly outperforming previous approaches in the low-resource setting.", "reasons_to_reject": "- It is a little unclear to me how this would scale to a setting of fine-grained entity extraction (https://ojs.aaai.org/index.php/AAAI/article/view/8122). At the very least, it would be helpful if the authors added a comment to the limitations section about how this would scale to cases where the number of entity types is large (e.g. 10s or even 100s).", "questions_to_authors": "- On lines 106-107, it says, \"The model is trained with parameter-efficient techniques for data-efficiency.\" Parameter-efficient techniques are used to reduce memory requirements, no? I do not understand what \"data-efficiency\" is referring to here.\n", "paper_topic_and_main_contributions": "This paper introduces QPMRC-NER, a modified machine-reaching comprehension (MRC) based method for NER. The primary contribution is to extend the MRC-NER framework such that multiple queries can be issued in parallel, therefore improving efficiency during inference.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- I don't understand what the \"Transfer\" arrow in Figure 1 is trying to show. It almost appears to point from \"Context\" to \"ORG/query\". Could the authors make it more explicit when/how the transfer happens? Maybe updating the figure caption to explain.\n- \"Unidirectional attention\" seems like a poor name, as it suggests a mechanism like the causal or masked self-attention of transformer decoders in language modelling. I also don't clearly understand how the modification in this paper is \"unidirectional\". Something like \"query-segmented attention\" would be more immediately understandable (IMO) and avoids the erroneous connection to causal attention."}}, {"review_id": "Va2nAnB2E7", "reviewer": "Reviewer_HR2U", "summary": "", "questions": "- Why did you not include DoSEA for all low-resource experiments (but only in Table 4)?\n- Why does query 2 in Fig 2 have two entity names?\n- What is the Type part (u_a and u_b tokens) of the input in Fig 2? You never explained this. Are these used to determine M_{i,j}?\n- Are the chosen BERT encoders the same as those in the baseline models you compare against? (If not, could performance stem from just using a better BERT?)  [This kind of information should be clarified more explicitly in the paper]\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the detailed feedback and invaluable suggestions.\n\nQ: Not all experiments seem to use the better competitor model. (cf. only Table 4 includes DoSEA, where it seems to be the strongest competitor; hence why not include it in other low-resource oriented experiments?)  Why did you not include DoSEA for all low-resource experiments (but only in Table 4)?\n\nA: DoSEA (DoSEA: A Domain-specific Entity-aware Framework for Cross-Domain Named Entity Recogition, Coling 2022) is a NER method designed specifically for cross-domain NER, which can identify domain-specific semantic differences and mitigate the subtype conflicts between domains and achieve sota performance as illustrated in Appendix B. Since its designed and only suitable for cross-domain NER, we only compare QPMRC-NER with it in the cross-domain low-resource NER setting.\nLow-resource NER encompasses various distinct settings, each characterized by its unique attributes. Certain NER methods are exclusively tailored to address the requirements of a specific setting, while others possess a versatility that allows them to be applicable across multiple settings. In our experimental investigation, we adopt a comprehensive approach by incorporating representative methods alongside state-of-the-art approaches as baseline methods. This selection encompasses models specifically devised to handle diverse low-resource settings, such as BERT-MRC and LightNER, as well as task-specific methodologies like DoSEA.\n___\n\nQ: Not specified which performance metrics are reported exactly (e.g., Table 2: is this micro- or macro-average F1, over all entity types? are the latter the same in all 3 datasets?)  \n\nA: The metric chosen to assess the performance of all experiments is the micro-average F1-score. We will provide a more explicit elucidation of this choice in our final version.\n___\n\nQ: Are the chosen BERT encoders the same as those in the baseline models you compare against? (If not, could performance stem from just using a better BERT?)  \n\nA: The BERT encoders we use are the same as those in the baseline models in all experiments , as elucidated in the Implementation Detail section of Section 3.1.\nTo elaborate, we utilize BERT-large for CoNLL03, MIT Restaurant Review, MIT Movie Review, and ATIS datasets. For the CrossNER dataset, we employ BERT-base, while for the GENIA dataset, we utilize BioBERT-large. Additionally, the Chinese MSRA dataset is processed using Chinese-BERT-WWM. We will make it more clearly in our final version.\n___\n\nQ: Why does query 2 in Fig 2 have two entity names?  \n\nA: The e_{i,j} represents the jth token of the entity i. In Figure 2, the query 2 serves as a representation of an entity type that encompasses a name consisting of two tokens. In our final version, we will provide a more explicit clarification to ensure better understanding and clarity.\n___\n\nQ: What is the Type part (u_a and u_b tokens) of the input in Fig 2? You never explained this. Are these used to determine M_{i,j}?  \n\nA: The u_a and u_b represent the segment ids for input sentence and queries respectively, enabling the model to differentiate between different input components. The details regarding the model inputs are expounded upon in the \"Query Generation and Model Input\" section of Section 2.2. We will explain the symbols and make it more clarity in our final version.\n___\n\nQ: It would be worth to also point to the distinct setting of few-shot NER (and clearly state the different model input/output in such works). A quick search immediately led me to this: https://paperswithcode.com/paper/a-comparative-study-of-pre-trained-encoders-1  \n\nA: Thank you for the suggestion. We will incorporate a relevant low-resource NER survey as a point of reference for a better distinction and understanding of the different NER setups encountered in low-resource contexts.\n___\n\nTypos Grammar Style And Presentation Improvements  \n\nWe express our sincere gratitude for the comprehensive suggestions provided to enhance the presentation of our work. We will diligently address any typographical errors and refine the representations accordingly, taking into account the invaluable insights provided. Your suggestions will significantly contribute to improving the overall quality and clarity of our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- New multiple-entity-types-at-once NER system based on BERT and biaffine encoding; model dubbed QPMRC-NER (the authors may want to think about an easier mnemonic name though)\n- Results showing the benefits compared to recent works in terms of better NER performance in low-resource settings, and speed (~ up to 2x)\n", "reasons_to_reject": "- Model & experiment details do not suffice for independent reproduction of the model implementation nor experimental results\n- Not all experiments seem to use the better competitor model (cf. only Table 4 includes DoSEA, where it seems to be the strongest competitor; hence why not include it in other low-resource oriented experiments?)\n- Not specified which performance metrics are reported exactly (e.g., Table 2: is this micro- or macro-average F1, over all entity types? are the latter the same in all 3 datasets?)", "questions_to_authors": "- Why did you not include DoSEA for all low-resource experiments (but only in Table 4)?\n- Why does query 2 in Fig 2 have two entity names?\n- What is the Type part (u_a and u_b tokens) of the input in Fig 2? You never explained this. Are these used to determine M_{i,j}?\n- Are the chosen BERT encoders the same as those in the baseline models you compare against? (If not, could performance stem from just using a better BERT?)  [This kind of information should be clarified more explicitly in the paper]\n", "paper_topic_and_main_contributions": "The paper considers NER, and particularly focuses on low-resource settings. \nThe authors frame it as a machine reading comprehension (MCR) task -- as do several recent works -- and proposes a model that jointly extracts multiple entity types at once. They thus propose feeding multiple queries (one for each entity type) to an encoder (BERT-based), propose an adapted attention mechanism (basically avoiding the multiple queries to attend to each other), use a biaffine classifier to which they also feed the entity type name, and add an auxiliary loss term on individual token classification (besides the span identification).\nExperiments are presented in various settings: (1) large training data set, (2) low-resource training from scratch, (3) low-resource with transfer from a different large domain.\nResults show competitive results for (1), and performance exceeding prior works for (2) and (3).\nContributions are mainly the new QPMRC-NER model with the above characteristics.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "n/a"}, "extra_text": {"Missing_References": "It would be worth to also point to the distinct setting of few-shot NER (and clearly state the different model input/output in such works). A quick search immediately led me to this: https://paperswithcode.com/paper/a-comparative-study-of-pre-trained-encoders-1", "Typos_Grammar_Style_and_Presentation_Improvements": "General comments:\n- please ensure all symbols are properly defined (e.g., d_x, d_h, d_a are what kind of dimensions?)\n- please unambiguously state what performance metric is reported (e.g., Table 1 are micro- or macro-averages over the different entity types?)\n- please indicate meaning of bold/underline values in tables. (Btw, please always indicate the best results; why don't you do it in Table 1?)\n- please do not use vertical lines in tables (e.g., personally, I found this quite helpful to create nice looking tables: https://people.inf.ethz.ch/markusp/teaching/guides/guide-tables.pdf) \n\nDetails:\n- 026: predictor to predict entities simultaneously -> predictor to extract multiple entities at once\n- 029: efficiency -> efficient\n- 030: perform -> performs\n- 032: setting -> settings\n- 074: employing the -> employing\n- 075: The prior -> Prior\n- 081: demonstrated the ability to achieve ... -> demonstrated ...\n- 097: \"continuous prompt\": not clear what this means. How can a prompt be \"continuous\"? I'd expect it to be \"static\", or possibly \"dynamic\" if it's template-based...; it would be highly useful to include a concrete example (e.g., in Fig 2 use an actual prompt rather than just abstract identifiers  p_0 ... p_m)\n- 107: \"parameter-efficient techniques for data efficiency\" -> overly vague; if you don't want to give more details here, just say it is \"data efficient\" and point to a reference / later part of the paper where you detail what you're talking about\n- 116: facilitates the -> facilitates\n- 118: thereby resulting -> resulting\n- 127-134: overly verbose; I suggest:\n    Section 2.1 presents the task formulation and introduces our QPMRC-NER method, illustrated in Fig. 2. QPMRC-NER comprises (i) a query-parallel encoder and (ii) a query-conditioned biaffine predictor, detailed in §2.2 and §2.3 respectively.\n- 142: \"a query Q for each entity type\": it seems this query is static and always the same for all entity types (cf. p_0...p_m remains the same for Q1, Q2... in Fig.2)? Please be clear about this. Also, are p_i actual words/tokens, or arbitrary learnable embeddings?\n- 150: \"context\": is this the actual input sentence to perform NER on? Or does it contain something else/more? Please be clear\n- 153-54: that can be further used -> that are used\n- Fig 2:\n  + please clearly define all symbols\n  + clarify the meaning of the matrices in the top right (I assume they represent y^i_{k,l}, scoring span (k,l) with probability of representing an entity of type i ?\n  + Encoder: why not be more specific, and say \"BERT-based encoder\"?\n  + Not clear whether you train independent Biaffine Predictors for each type, or whether they all share the same parameters\n  + why does Q2 have 2 entity type tokens e_{2,1} and e_{2,2}? seems arbitrary\n- 170: is -> are\n- 184: \"transformer-based\": please also already state which specific model(s) you're using in the experiments\n- 202-208: not clear whether you define separate attention parameters W for each entity type, or if these are shared \n- 209-210: please define the d_* parameters\n- 212, 214: inf -> <infinity symbol>\n- 217: utilized -> used \n- 222: token -> word-piece token\n- 238: For further details, please refer to Appendix C -> Further details are in Appendix C\n- 287: overlapped -> overlapping\n- 291-92: token is  ... type -> token $i'$ is .... type $j$\n- 296: explain what y^{i'}_j represents  (see also 291-92)\n- 331: distinct -> different\n- 366: classic -> the classic\n- Table 1:\n    + please highlight best/runner-up results, as you do in Table 2\n    + caption: reproduced -> produced\n- 368: for detailed -> for a detailed\n- Table 3:\n   + I'd suggest to include a top row with the number of instances per entity type (which now is listed in the paper text instead)\n   + what does the * indicate?\n   + last row = macro-average over previous columns? or is it micro-average over all instances?\n   + Would it make sense to repeat the low-resource sampling of just 50 LOC/MISC instances multiple times, and then report averages over these runs, to have a more robust result (with some confidence interval instead of just a single value)?\n- 427: under -> for\n- 428: setting -> settings\n- Table 4: \n   + are these micro/macro averages of F1 over all instances / entity types?\n   + do all domains have the exact same entity types? in the same proportions? (If not, then maybe differences across them may simply stem from their different entity type distributions, not because of the domain per se)\n- §3.6: please indicate hardware information\n- 483: avoid line break after item number '(1)'  (hint: in LaTeX, use the enumitem package with the 'inline' option)\n- §3.7:\n   + I don't understand how exactly the uni-attention ablations work; please expand your explanation\n   + same issue with the \"query-conditioned biaffine predictor\"; does it mean you now only use loss L_t from eq. (12) and thus essentially do individual token classification rather than span identification?\n- 524: indicating -> indicates\n- 541: However -> Also    [doesn't sound like a contrasting statement, rather something new/extra]\n- 535: \"vanilla self-attention\": is this the same as your \"w/o uni-attention_2\"? Please be consistent in naming your ablation models\n- §4.1 \n   + if it's the only subsection within §4, why name it?\n   + seems to be largely a repetition + slight expansion of what you already stated in §1; please avoid duplication of similar statements\n- 468: \"inferior\": compared to what alternative?\n- 574: could predict -> predicts    [hint: https://learnenglish.britishcouncil.org/grammar/english-grammar-reference/can-could]\n- 594: could extract -> extract\n- 597: semantic -> semantically\n- 600: \"It\": not 100% clear what this refers to\n- 602: could encounter -> seems to encounter  [or just plainly: 'encounters']\n- 608-9: \"query isolation restriction\": not clear what you mean exactly\n\n"}}]

Author rebuttal:

Meta-review: This paper aims to improve MRC-style NER by introducing a query-parallel design, which can extract multiple entity types at the same time and works well for both resource-rich and resource-limited settings. Comprehensive evaluation on multiple datasets under different settings demonstrates the effectiveness of the proposed method.

Strengths:
- This is a solid improvement for MRC-style NER
- Comprehensive evaluation that demonstrates the effectiveness of the method especially under resource-limited settings

Weaknesses:
- As reviewer HR2U pointed out, there are many minor writing issues in the current paper that collectively makes it harder to read and follow
- There are concerns over reproducibility, especially for a complex method like this. There is no indication of code release, either in the paper or in the rebuttal when the reviewer pointed this out as a weakness.
Decision: Accept-Findings