2INER: Instructive and In-Context Learning on Few-Shot Named Entity Recognition
Authors: Jiasheng Zhang
Xikai Liu
Xinyi Lai
Yan Gao
Shusen Wang
Yao Hu
YIQING LIN
Venue: EMNLP 2023 Findings (2023)
Keywords: prompt-based learning
instruction finetuning
in-context learning
NER

Abstract: Prompt-based learning has emerged as a powerful technique in natural language processing (NLP) due to its ability to leverage pre-training knowledge for downstream few-shot tasks. In this paper, we propose 2INER, a novel text-to-text framework for Few-Shot Named Entity Recognition (NER) tasks. Our approach employs instruction finetuning based on InstructionNER to enable the model to effectively comprehend and process task-specific instructions, including both main and auxiliary tasks. We also introduce a new auxiliary task, called Type Extracting, to enhance the model's understanding of entity types in the overall semantic context of a sentence. To facilitate in-context learning, we concatenate examples to the input, enabling the model to learn from additional contextual information. Experimental results on four datasets demonstrate that our approach outperforms existing Few-Shot NER methods and remains competitive with state-of-the-art standard NER algorithms.

2INER: Instructive and In-Context Learning on Few-Shot Named Entity Recognition
Jiasheng Zhang' Xikai Liu
Xinyi Lai
Yan Gao²
Shusen Wang² Yao Hu?
Yiqing LIN'
Shanghai Jiaotong University "Xiaohongshu Inc. CChongqing University {js. yiqing. Lin]@s edu. Malkaiyyduun haxian, xiahou aohongshu.coo com
laixinyi@cqu. edu. cn
Abstract
layer necessitates a consistent label set for both the training and testing data, which poses challenge for knowledge transfer. Therefore, these models need to be trained from scratch to adapt to a new domain with a different label set, highlighting the requirement for large amount of data for these methods.
Prompt-based learning has emerged pow- erful technique in natural language processing (NLP) due to its ability to leverage pre-training knowledge for downstream few-shot tasks. In this paper, we propose 2INER, a novel text- to-text framework for Few-Shot Named Entity Recognition (NER) tasks. Our approach em- ploys instruction finetuning based on Instruc- tionNER (Wang al., 2022) enable the model to effectively comprehend and process sask-specific instructions, including both main and auxiliary tasks. We also introduce new auxiliary task, called Type Extraction, to en- hance the model's understanding of entity types in the overall semantic context of sentence. To facilitate in-context learning, we concate- nate examples to the input, enabling the model to learn from additional contextual information. Experimental results on four datasets demon- strate that approach outperforms existing Few-Shot NER methods and remains compet- itive with state-of-the-ar standard NER algo- rithms.
Due to the high cost of sequence labeling an- notation in real-world scenarios, labeled data for NER is often limited. As result, few-shot NER has gained significant attention due its practical applications. Meanwhile, applying prompt-base learning (Han et al., 2021) on PLMs is an effee- tive way to solve few-shot problems (Brown et al., 2020). PLMs can learn a lot of knowledge re- garding human languages by training on large amount of self-supervised corpus. In order to ex- plore the potential of PLMs, prompt-basee based learn- ing reformulate the downstream tasks to text-to- text framework with additional prompt indicating task descriptions (e.g. instruction fine-tuning (Wei et al., 2021; Chung et al., 2022: Sanh et al., 2021)). Through this approach, the model can effectively leverage the knowledge present in PLMs to en- hance downstream skills without the need for ad- ditional large amounts of downstream data. This enables the model to achieve remarkable perfor- mance in few-shot settings.
1 Introduction
Named Entity Recognition (NER) has been a funda- mental task of Natural Language Processing (NLP)
and there are three types of sub-tasks in NER: flat NER (Tjong Kim Sang and De Meulder, 2003), nested NER (Kim al., 2003) and discontinuous NER (Karimi al., 2015). All three sub-tasks aim to locate named entities, extract the entity spans, and classify each span into pre-defined la- bel categories. In terms of the flat NER which is the main focus of this paper, it can be formu- lated as a sequence labeling paradigm by assign- ing labels to each token in the sentence through token-classification models. The dominant meth- ods include combining Pre-trained Language Mod- els(PLMs) (Devlin et al., 2019) with label-specific classifier (LC) (Strubell et al., 2017; Cui and Zhang, 2019). However, the fixed shape of the output LC
Recently, many prompt-based NER methods have emerged to address the limitations of tradi- tional few-shot NER approaches. TemplateNER (Cui et al., 2021) treats original sentence and pre- dicted template filled by entity spans as source and target sequence, respectively, but all candi- date spans must be enumerated during inference, leading to high computational cost. BARTNER (aan et al., 2021) proposed pointer mechanism to unified all NER sub-tasks into one sequence- to-sequence (seq2seq) framework. BARTNER uti- lizes the raw sentence as input and outputs pointer index and tag index which represent the location

---

of the span and the corresponding label index in the category, respectively. To further adapt BART- NER for few-shot settings, LightNER (Chen et al.. 2022b) proposed a lightweight tuning approach for low-resource settings by adding a unified learnable verbalizer and incorporating learnable parameters into the self-attention layers. Nonetheless, due to the fact that pointer mechanism only outputs the indexes of entities and labels, the model en- counters challenges in effectively leveraging the capabilities of PL.Ss to directly comprehend the se- mantic meaning between entities and labels. Thus instead of using a pointer mechanism, Instruction- NER (Wang et al., 2022) directly generates entity spans and types in the target sequence and applies instruction fine-tuning with two auxiliary tasks to further mining the capabilities of PLMs, which leads to significant few-shot improvements.
(Raffel al., 2020) for natural language gener- ation. In terms of the source sentence, we use instructions to distinguish between tasks by giv- ing a comprehensive task description and include an alternative field to identify the entity type that requires detection. Moreover, we suggest incorpo- rating in-context demonstration examples into the source sentence to enable the model to learn from external knowledge. For the target sentence, we use natural language to represent entity spans and types instead of pointer mechanism. addition to the two auxiliary tasks used in InstructionNER, we propose a new task called type extraction to further explore the potential of PLMs to understand label semantics. Type Extraction task requires the model to identify all the entity types presented in the orig- inal sentence and learn to understand the meaning of entity types at the overall semantic level of the sentence. Our contributions can be summarized as follows:
In terms of the auxiliary tasks InstructionNER, they propose two auxiliary tasks from two perspec- tives: span recognition (Entity Extraction) and en- tity labeling (Entity Typing). However, we argue that NER can be further divided into three parts: 1) understand the the relationship between the label and semantic meaning of the sentence. 2) extract the spans. 3) annotate the given spans. We believe that both span recognition and entity labeling can be benefit from having a deeper understanding of the label semantics. Therefore, we proposed a new auxiliary task, called Type Extraction, to help the model to acquire this ability.
To utilize external knowledge, we apply demonstration-base baased in-context learning examples to the instruction template. The in-context exam- ples enable the model to directly learn which spans correspond to which types from these additional information, leading to better few-shot abilities.
We expand the NER capabilities by dividing them into three components instead two. And we propose novel auxiliary task for instructions fine-tuning, called type extraction, to address the existing gap. It can enable the model to understand the meaning of the entity types through the overall semantic level of the sentence, which will improve span recognition and entity labeling abilities.
Meanwhile, none of the above methods take the additional external knowledge into account. Cur- rent literature related to utilize external knowl- edge in NER involve (Chen et al., 2022a) and (Lee et al., 2022a). SDNet (Chen et al., 2022a) proposes self-describing mechanism to leverage external resources by self-describing both entity types and mentions, while (Lee et al., 2022a) uses demonstration--asee method by incorporating examples to the input but without text-to-text framework. Therefore, to the best of our knowl- edge, there is currently no existing literature that combines in-context external knowledge with in- struction fine-tuning for few-shot NER.
We conduct extensive experiments on four datasets, demonstrating that 2INER outperforms existing few-shot NER methods and remains com- petitive with SOTA standard NER algorithms.
Related Work
2.1 Named Entity Recognition
Currently, NER tasks can be divided into flat NER (Tjong Kim Sang and De Meulder, 2003), nested NER (Kim et al., 2003) and discontinuous NER (Karimi et al., 2015), while in this paper, we mainly focus the flat NER task. The current dominant method solve flat NER is using token-level clas- sification by turning it into sequence labeling problem (Chiu and Nichols, 2016; Liu et al., 2019; Zhang et al., 2020; Liu et al., 2021), which apply a text encoder and CRF (Ma and Hovy, 2016) in
In this paper, we propose 2 and In-Context Learning on Fhw-Shot NER). We build upon the work of InstructionNER by incorporat- ing in-context examples and a novel auxiliary task. Specifically, we first reformulate the NER tasks into a text-to-text framework and then employ T5

---

sequence. Recently, BARTNER (Yan et al., 2021) formulate all three NER tasks into a text-to-text framework to solve them concurrently. BARTNER generate entity span sequences by a ppinter-based model based on BART (Lewis et al., 2020) so that special design of tagging schema or spans post- processing are no longer needed.
The standard flatten-NER can be formulated fol- lows, given the input sentence containing tokens I1 42, zn.]. the model have to predict the target sentence Y [4,, 12....... 1n]. We use VBIO to denote the BIO label set, so V4, 4 E VBIO. While in the acquence-to-ssquenee to-suueence modeling scenario, the input sentence is still X but instead of predicting Y. the model predict each entity 3i = (e,, di- rectly, where 8; represents the entity span in X. And ej V represents the entity type of sj, where V is the set of entity types.
X =
2.2 Prompt-based Learning
With the emergence of GPT-3 (Brown et al., 2020), prompt-based learning has gained increasing atten- tion. It can better stimulate the knowledge model learned in pre-training stages and integrate differ- ent tasks together compared to the paradigm of fine-tuning separate model for each task, especially in few-shot settings (Han et al., 2021). To push prompt-based learning further, instruction-bart learning (Wei et al., 2021) proposed to fine-tune the PLMs on a collection of task descriptions which enables the model to better follow human instruc- tions and generalize to unseen tasks with better zero-shot and fho-shot abilities (Chung et al., 2022; Sanh et al., 2021).
More specifically, we use I and r to indicate the left and right boundary of an entity span in X, so 8; can be simplified as
Zirr, where Tyr [m) 21 ...... .r|]- Therefore, the NER model have to predict each y/ in X, indicating that the span 8; belongs to the e, entity type.
3.2 Convert NER to Text-to-text Task
Using language models like T5 (Raffel et al., 2020) to solve most NLP tasks in unified text-to-text framework can not only fully utilize the knowledge model learned in the pre-training stage but also simplify the training by using same data format, same loss and same model architecture. Moreover, Compared to using simple prompts, using instruc- tion finetuning can further explore the capabilities of the model (Chung et al., 2022; Sanh et al., 2021). Besides, utilizing in-context learning further enhance the model's few-shot capabilities in gen- eral (Brown et al., 2020) and specifically NER abil- ities (Lee et al., 2022b). Therefore, we transform the NER task into text-to-text format and employ instruction finetuning and in-context learning to un- leash the model's few-shot capabilities, as shown in Figure 1. The backbone we used is T5.
2.3
Few-Shot NER Methods
One line of work in few-shot NER is to apply con- trastive learning to assign the labels by searching for the closest token (Das et al., 2022: Chen et al., 2022c), prototype (Snell et al., 2017: Fritzler et al.. 2019; Ma et al., 2022b) or label semantic (Ma et al., 2022a; Huang et al., 2022) in the support set. An- other line of researches is prompt- based learning using a unified text-to-text framework to make full use of the PLMs abilitics. (Cui et al., 2021) applics span classification using BART and (Chen et al., 2022b; Yan et al., 2021) use a pointer mechanism to generate indexes of spans and types. (Wang et al., 2022) utilizes instruction fine-tuning and two auxiliary tasks to train T5. Meanwhile, to apply ex- ternal knowledge to the model, (Chen et al.. 2022a) introduces a self-describing mechanism and (Lee et al., 2022a) uses a demonstration based method. Therefore, our methods introduce in-context learn- ing via instruction fine-tuning together to achieve better few-shot NER abilities, which haven't 't been fully discussed yet in seq2seq NER settings.
The basic text-to-texi format of the main NER tasks consists of the following three parts, which is inspired by InStructionNER (Wang et al., 2022) !:
The instruction is a prompt that in- forms the model about the current task it needs to perform. The model is expected to follow the instructions provided within the prompt and com- plete the task accordingly. The instruction for the main NER task is: Please extract entities and their types from the Sentence, choose entity types from Alternatives.
Instruction
3 Methodology
Sentence
The sentence is the input X from which entities need to be extracted.
3.1 NER Definition
NER aims to predict all spans in the input sentence as well as the entity types associated with the spans.
'The templates of auxiliary tasks and in-context Example will be discussed in 3.4 and 3.3 respectively.

---

Instrcctor: Please ectract entitees and their hypes from the Serience based on the snowledge in Examples, choose entiy types from Attemativos.
Sentencer where can ger cheap hamburgers and rating, right hours, vegatarian dici is location, undor 10 budks porson is prico.
NER Man Task
Altemativs: restaurant
price
hamburggers fries
dish, suth cuisine.
Audilaay Extraction
Instruction: Please extract entity spowared Sentence Sentenoer are there shoap star restaurants hore.
rating type
type
2INER
Ausiliary Taske Entiy Extraction Taok
Instruction: Please extract entiies from the Sensance Sentence: show mhe restaurants that well rated but insppertiie
wedl rated le a antilty ward Inoxpensive as encity word
Instruction: Please type these
Choose according
Auxilary Taskc Ensty Typing Task
Sentence: where best
10 minnespoilss Altematives: restburant nome, rating, amenity, hours, dist, cutsine location, price.
mimeeapole location. under 10 is prico.
Figure 1: The model architecture our proposed 2INER. The left and right sides are the source and target sentence of the model, respectively.
Alternatives Alternatives is a list of entity types (V) split by comma, from which the model needs to select the corresponding type to annotate the corresponding span. Alternatives serves as constraint and guiding factor, informing the model that can only select entity types from within this list.
the spans and finally annotate the given spans. We will discuss the configuration of the auxiliary task in detail from these three perspectives.
3.3.1 Understand label semantic
Type Extraction The goal of the Type Extraction task is to identify all the entity types present in the original sentence. The Instruction is changed to: Please extract all entity types appeared in the Sen- lence. We will remove the Alternatives in this case, which means that there will be no constraints or hints regarding entity types the input text, aiming to increase the difficulty of the task. And the out- put template is: e, type exists in the sentence. The Type Extraction task involves detecting whether a specific entity type appears in the sentence, with- out focusing on specific spans or associating spans with entity types. This task will assist the model in understanding the meaning of entity types at the overall semantic level of the sentence. We believe that once the model gains deeper understanding of entity types, it will be able to comprehend the rela- tionship between spans and types more accurately. As a result, it will enhance both span recognition and entity labeling capabilities simultaneously.
In order to formulate the NER output to natural language, for each NER output y/ = (e,, 8;), we use the following template to convert it to text: is a/an C₁. and we use dot to concatenate all detected entity occurrences Y/ to form the output text. In terms of the entity types ex, we use natural language represent the entity instead of adding special tokens to the model
3.3 Auxiliary Tasks
To enhance the NER performance, in addition to the main task, we need to introduce several auxil- iary tasks. In InstructionNER (Wang et al., 2022), they employed two auxiliary tasks: entity extrac- tion and entity typing. Moreover, in this paper, auxiliary task called type extraction will be introduced. During training, the auxiliary task will also be in the form of text-to-text data, trained alongside the main task data.
3.3.2 Span recognition
The auxiliary task primarily aims to improve NER capabilities from three perspectives: under- stand label semantic, span recognition and entity labeling, since NER can be decomposed into three steps: understand the relationship between the label and semantic meaning of the sentence, then extract
Entity Extraction The goal of the entity extrac- tion task is to extract useful entity spans from the original sentence without the need for annotating the extracted spans. The instruction has been mod- ified to: Please extract entities from the Sentence. Because the model doesn't need to type spans, the Alternatives field is deleted. And the output tem- plate has been changed to: 5; is an entity word, since e; is no longer needed. Because the entity ex-
Pc.g. "Character Name" will be represented as "Character Name" instead of adding special token named "Character Name"

---

traction task only require the model predict use- ful spans regardless of the associated entity types, this task will guide the model to extract correct spans, enhancing the span-Fi accuracy, moreover, overall main task FI as well (Wang et al., 2022).
will only be added the main NER tasks and there will be Examples added in auxiliary tasks, which will be discussed in detail in Analysis 5.2.
In terms of the choices of the samples in Exam- ples, we randomly choose some spans appeared in the train set as well as their corresponding entity types to create Examples. Since about the entity types present in the sentence, we will provide at least one example for each entity type in the Altematives list within the Examples. The number of samples of each entity types Ex- amples will also be the same '(e.g. in terms of MIT Movie dataset, there are 12 entity types. If we set the number of examples to 5, there will be 5 examples for each entity types, resulting in total of 5* 12 examples in the field).
The original InstructionNER (Wang et al., 2022) paper only employed span concatenation as the output(e.g. $1, #2, 83.). However, we believe that since the output of the main task consists of com- plete sentences with subject-verb-objert structures, it would be more cohesive to follow the same pat- tern for the auxiliary tasks. And more structured output can fully utilize the PLMs's understanding of the task as well.
uncertain
3.3.3 Entity Labeling
Entity Typing The entity typing task aims to type the given span with the correct label. The in- struction has been modified to: Please type these entities according to the Sentence: <the given spans>. The Alternatives prompt and output tem- plate is the same as those in main task. During training, the given spans in the Instruction is the ex- act entity spans that have labels on. In entity typing task, since the spans are given, the model doesn't 't need to worry about the correctness of the span extracted, so the model can focus more on learning how to label the entity accurately, enhancing the main task NER ability.
3.5 Inference
During inference time, we first use the template of the main NER task to wrap the input sentence X, and then feed the sentence to 2INER to get the predicted output text. In terms of the Example field, the example spans are sampled from the training support set, so the model won't see the ground- truth in the Examples during evaluation, avoiding information leakage. After the output text is gener- ated, a decoding strategy will be applied to get the predicted entity (e;, s:): (1) We use dot to split the whole output text to obtain individual sub-texts. (2) We use "is a" or "is an" to split each sub-text if they can be found. (3) The span is the part before "is a/an" and the entity type is the part after it. Once we get the (ca, si), we will check whether s; is in the input sentence and e; is in the set of entity types V. If doesn't 't pass the check, then it isn't 't a valid entity and will be deleted. And if any of the three steps result in a match failure, then the sub-text will be skipped.
3.4 In-Context Learning
In-context learning will be applied to further en- hance few-shot NER capabilities. The main ap- proach of in-context learning is to append Exam- ples at the end of the input sentence, hoping that the model can directly learn which spans corre- spond to which types from these Examples, without the need for additional gradient updates. Besides, the in-context examples are also presented in natu- ral language format, which closely resembles the output text format, serving as reminder for the model about the desired format should generate and making it casier for PLMs to understand. This similarity helps bridge the gap and facilitates the model's 's comprehension.
Experiment
4.1 Dataset
We conduct NER experiments in standard and low-resources settings. For the rich-resources do- main, we use CoNLL-2003 (Tjong Kim Sang and De Meulder, 2003) and for the low-resource do- main, we use three datasets: MIT Movie Review, MIT Restaurant Review (Liu et al., 2013) and Air- line Travel Information Systems (ATIS) (Hakkani-
The in-context example format in NER is in- spired by (Lee et al., 2022b). All examples in this context follow the template: span is alan entity- type. And we will concatenate an additional prompt (based on the knowledge in Examples) after the In- struction to hint the model to learn from the Exam- ples. During training stage, in-context Examples
*We refer to "the number of sumples per entity types" as "the number of examples" in the rest of the paper for conve- nience.

---

Tür al., 2016), following (Wang et al., 2022;
Model
FI
Span-F1
(Yang et al., 2018)
90.77
Chen et al., 2022b; Cui al., 2021; Yan et al., 2021).
(Ma Hovy, 2016)
91.21
(Gui et al., 2020)
92.02
(Yamada et al., 2020)
94.30
92.40
4.2 Implementation settings
(Li al., 2020a)
92.87
(Yu al. 2020a)
92.50
In Few-Shot NER scenario, in order to guarantee
LC-BERT
91.73
LC-BART
90.60
that each entity type has equal number of instances in the training set, we can't t sample At sentences for each entity type directly because a single sentence may contain multiple entities, the actual shot will exceed k. Following (Wang et al., 2022), we will apply greedy sampling strategy (Yang and Katiyar, 2020) instead, to sample the few-shot train- ing set for each setting and due to the randomness of the sampling, we will repeat 3 times for each ex- periment. We use T5-large as the backbone model for fair comparision with (Wang et al., 2022). In terms of the number of examples in in-context Ex- ample field, we set the number to 5 for MIT Movie and MIT Restaurant dataset, and for ATIS dataset as default We only add in-context Example field on main-task, and don't include them in auxiliary tasks. The ratio of auxiliary tasks is set to 1.0 We set the batch size to 2/4/8, learning-rate to 2e- 5/5e-5 for 10/20/50 Shot settings respectively, and set batch size to 32, learning-rate to le-4 for the abundant data setting. The optimizer is Adam and beam search is 2. For evaluation, we use Fl score as the metric for NER.
TemplateNER
91.90
BARTNER
93.24
LightNER
92.93
2INER (InsmuctionNFR
90.71
93.93
Table I: Fl and Span-F1 (%) on CoNLL-2003 Standard NER setting. Our method is competitive with SOTA algorithm and even outperform BARTNER (Yan et al., 2021) in span-Fl. "*" indicates training on external data. "f" indicates the reproduction by (Yan et al., 2021). "!" indicates the reproduction with only the sentence-level context by (Yan al., 2021).
SOTA algorithm under standard NER setting and even outperform BARTNER (Yan al., 2021) in span-Fl, which is designed for rich-resource NER settings. The performances of 2INER in data abun- dant nested and discontinuous NER settings are in Appendix A.
4.4 Few-Shot NER Setting
Under Few-Shot NER setting, we only use K-Sho training samples to finetune our model and the re- sults are in Table 2. According to the table, we can find that: (1) Our models consistently outper- form InstructionNER as well as other baselines on all three datasets under 10/20/50 Shot settings (ex- cept 50Shot in ATIS, which is slightly lower than BARTNER). Especially in MIT Movie dataset, our models have 7.33%, 6.76%, 5.39% improvements compared to InstructionNER under 10/20/50 set- tings. (2) Our 10Shot model even outperforms Tem- plateNER's 50Shot model by 20.73% and 7.06% in MIT Movie and MIT Restaurant respectively, which highlights the superiority and capability of our model. (3) We have the same finding as structionNER (Wang et al., 2022) that Fl improve- ments are much more significant on MIT Movie than on MIT Restaurant (7.33% 6.76% / 5.39% v.s. 6.86% 3.24% 3.3% under 10/20/50 Shot set- tings), which indicates that although MIT Movic has more entity types, text-to-text framework and instruction-tuning can better utilize pre-training knowledge, and through in-context learning, the model can learn more about the relationships be- tween entities. (4) In ATIS dataset, the improve-
The names InstructionNER in the tables mean training with main-task data only, indicating the base model, and the subscript words in the tables indicate addition to the base model: +ET, +EE, +TE, +EX means adding Entity Typing, Entity Extraction, Type Extraction, in- context examples, respectively. And we named InstructiooNNERRR..EEE....E as 2INER, which our final model.
4.3 Standard NER Setting
We use CoNLL-2003 dataset to conduct standard NER experiment. We combine the train and vali- dation set as described in (Yan et al., 2021) to train the model. The result is in Table 1, which shows that even though our method mainly focuses on few-shot NER settings, it remains competitive with
herttp://huggiingg face. ctt.coo//ttrrreeee
*ATIS has entity types so we set the numbee to avoid excessively long token lengths.
"The data size ratio hetween main task and each auxiliary tasks. 1.0 means that each sample will be extended into samples: one for main task, one for EE, ET, TE, respectively.

---

Mosie
Models
10
20
50
10
20
50
10
20
50
LC BEXT
25.2
42.2
49.6
21.5
39.4
52.7
46.1
76.7
90.7
L.CBAAT
27.5
73.7
TempbueNER
$7.3
48.5
53.2
46.0
57.1
98.7
31.7
79.4
92.6
LighNER
41.7
5788
73.1
48.5
55.0
62.0
76.3
85.3
92.8
InsrusioNNER
(4.4(221)
70.0 (40.3)
(+11
56.7 (+1.2)
65.5 (6
(4.2
90.14 (4.0
91.23 (40.1911
92.53
InEocciooNERRETEE
(a3.0)
(s1.9)
74.7 (81.3)
(all9)
71.1 (80
90.04 (20/0211
91.46 (=0.231
92.62 (20.044
InsroctioNN..gg
12.56 (21.01)
74,99
(30.37)
64.07 (=1.25)
682 (s6.11)
(9119)
(s0.2)
(89005)
92.65
InemetioNNRR..
(2.0(+0:33)
76.55 (+0.2)
8002 (+0.76)
63.52 (+1.35)
68.67 (+0.95)
(1.22))
96.77 (20,6)
91 01.85 (+005)
92.69 (+0.1)
InstuctionNNM, SERREIIENNEK ETEEETEN
(20.91)
76.86 (s0.53)
80,09 (20.22)
66.76 (:0.
69.34 (40.51)
(4.4))
943.7 (20.20)
92.11 (+0009)
92.85 (a9.15)
Table 2:
The F1(%) on three dataset under 10/20/50 Shot settings. The bold number means the best across all models and the numbers in brackets means the standard deviation. The underline numbers the best results in our experiments. The "f" numbers mean the results of our reproduction. means the reproduction by InstructionNE: (Wang et al., 2022).
ment of our model is less significant compared to other two dataset. We argue that this is because ATIS contains 79 entity types and even if we only provide one sample span for each entity types in in-context Example field, the average token length is 1099 compared to 368 with or without examples, where the token length of the Alternative filed is 327. So the actual input Sentence X only accounts for 3.7% of the total token length, which increases the difficulty for the model to extract key informa- tion from lengthy sentences. So too many entity types may potentially reduce model improvements.
32
IntrrcctiokEN..K InGtrcckonERRTTE InstructinNER
85-
Dstaset name/ Restaurant MTT Mowlel Nimber
Figure Fl and Span-Fl (%) on MIT Movie MIT Restaurant through 10/20/50 Shot settings with different task combinations. The deep and light color indicate Fl and Span-F1 respectively.
4.5 Ablation Study
order to find out the influence our proposed type extraction task and in-context examples on model's few-shot abilities, we conduct ablation studics in Figure 2. The results indicate that adding type extraction task and in-context exam- ples can further enhance the model's few-shot NER abilities. We set InstructionNER the baseline here which only trains on main-task data without any auxiliary tasks. Then we add type extraction task (InstructionnNRRRpp)) or in-context examples (InstructionNERRex) respectively on the bascline model to explore their influences. The results from Figure 2 shows that in terms of 10/20/50 Shot settings in few-shot NER, type extraction task achieves average improvements of 7.21 1,, 4.86%, 4.35% Fl and in-context example achieves an average improvements of 6.76%. 3.84%, 3.84% Fl MIT Movie and MIT Restaurant dataset. Moreover. adding type extraction task can greatly increase the Span-Fl as well. Because Span-F1 indicates the model's ability to locate
spans, the results reveal that through training on type extraction task, span recognition can be ben- efit from having a deeper understanding of the la- bels from the overall semantic level sentence. Therefore, it proves the effectiveness of three steps of NER abilities we proposed in 3.3, and shows that type extraction task can simultancously im- prove span recognition and entity labeling abilities through understanding label semantic.
5 Analysis
5.1 Increase Example Number
In this section, we will focus on how the number of examples in in-context Example field influence the model performance. We will sequentially change the number of examples 1, 3, 5, 10, and 15, and train corresponding models observe the change of Fl on MIT Restaurant dataset. We train our model with main-task and in-context example with- out any auxiliary tasks (InstructioNNER.ey in this section. The results are Table 3.
WWe try to use special-tokens to represent the entity types, but the Fl is slightly lower than without using special-tokens and the proportion of to the total number of tokens is 4.5%.

---

InstractioNNRREE
MIT Restaurant
MIT Restaurant
Examples
20 Shot
50 Shot
10 Shot
Shot
50 Shot
65.5 (+1.4)
71.2 (+1.1)
2INER
65.26
69.27
74.2
67.74 (+0.22)
73.89 (+0.15)
Examples on all tasks
(#0.49)
(40.89)
(+0.45)
3
67.89 (#0.3)
74.15 (#0.39)
2INER
65.76
69,34
74.4
68.2 (10.11)
74.38 (#0.19)
Examples only Main. Task
(#0.47)
(+0.81)
(#0.4)
10
69.47 (+0.35)
74.41 (#0.18)
15
69.52 (#0.16)
74.64 (#00.49)
Table 4: The comparison between adding in context examples only on main-task and on all tasks including auxiliary tasks. Bold numbers indicate the best F1 and the numbers in brackets means the standard deviation.
Table 3: FI scores(%) on MIT Restaurant dataset while changing number of examples using InstractionNERRFx Bold numbers indicate the best Fl and the numbers in brackets means the standard deviation.
Models
MIT Movie
MIT Restaurant
100
200
500
100
200
500
LC-BERT
50.7
59.3
74,4
53.5
57.4
61.3
LC-BART
47.5
54.2
64.1
52.2
56.3
60.2
As the number of examples increases, Fl score continues to increase and the largest improvement in Fl score occurs when going from zero exam- ples to one example. As the number of examples increases further, the F1 will continue to increase but the rate of improvement gradually slows down. This suggests that when only one in-context exam- ple is provided, the model can quickly learn the specific meanings of each entity type from the ex- ample. While more examples may lead to repetitive cues to the model balance should be made be- tween model performance and computational cost.
TemplateNER
56.3
62.0
74,9
60.1
62.8
65.0
BARTNER
70.1
74.6
82.6
65.3
74.4
75.7
LightNER
78.0
80.6
84.8
70.8
75.5
80.2
InstretiooNNRR.rre.
74.3
78.4
82.3
72.7
75.5
76.6
2INER
81.3 83.54 86.16
76.57
78,31
79.11
Table 5: The F1 (%) under relatively abundant settings. "*" indicates the reproduction results by (Wang et al., 2022). Bold numbers indicate the best Fl.
And 2INER outperforms LightNER in all settings except 500-shots in MIT Restaurant, which shows that 2INER has great NER abilities under data abun- dant scenario as well. We argue that the in-context Example field may help the model to learn from more diverse samples from the abundant training and turn the general knowledge into specialized capabilities, leading to the improvement in FI.
5.2 Effect of In-Context Example on Auxiliary task
In this section, we will discuss whether to add in- context examples on auxiliary task. The model is 2INER BnstrrcttioNNRREE.E...
and we will compare two settings: add examples only on main- task, add examples main-task as well as three auxiliary tasks. The results in Table 4 indicates that adding examples on auxiliary task will slightly decrease the Fl performance. Because adding ex- amples to auxiliary tasks may potentially reduce their difficulty and make too easy for the model, thereby reducing the auxiliary tasks' effectiveness in aiding the main task. So adding examples only to the main task is a better approach.
Conclusion
In this paper, we propose 2INER for few-shot NER using both instruction finetuning and in-context learning by converting NER into text-to-text framework. Based on InstructionNER, we create template concatenate task task-specific instructions, input sentence and entity alternatives to make full use of the pre-training knowledge. Besides, we decompose NER into three steps and introduce an- other auxiliary tasks, called type extraction, to help the model better understand the general semantic meaning of the entity types, which can improve both span recognition and entity labeling abilities. Morcover, we apply the in-context examples to en- able the model to learn from additional contextual information, enhancing few-shot abilities. Multiple experiments on four NER datasets prove 2INER's effectiveness in few-shot NER scenario by consis- tently outperforming other baselines.
5.3 Increase Shot
In this section, we will discuss the model per- formance under relatively abundant settings. We increase the shots to 100, 200 and 500 in MIT Movie and MIT Restaurant datasets using 2INER (nstrectioNNERRRTTEEETEEE)) As shown in Table 5, compared to InstructionNER, 2INER achieves 5.43%, 3.98%, 3.19% improvements in Fl under 100/200/500 shots settings respectively.

---

Limitations
Jason P.C. Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional LSTM-CNNs. Trans- actions of the Association for Computational Linguis- rics, 4:357-370.
One limitation of our work is the extensive length of the Example and Alternative field when there are too many existed entity types. While incorpo- rating in-context examples in the input sentence can improve few-shot NER performance, it poses a challenge when the Example field becomes too long because we add at least one examples for each potential entity type, especially when the Alterna- tive list contains numerous entity types. This can result in less improvement gains and more compu- tational costs. To address this issue, we assume that larger PLMs such as the recently proposed L.LaMA (Touvron et al., 2023) could potentially be explored in future research as a means of resolution.
Hyung Won Chung, Le Hou, Shayne Longpre, Bar- ret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang. Mostafa Dehghani, Siddharthha Brahma, al. 2022. Scaling eestruction-finetune language models. arXiv preprint arXiv: 11.111111....
Leyang Cui, Yu Wu, Jian Liu, Sen Yang. and Yue Zhang. 2021. Template-based named entity recognition us- ing BART. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021, pages 1835-1845 Online. Association for Computational Linguistics.
Leyang Cui and Yue Zhang. 2019. Hierarchically- refined label attention network for sequence labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing JEMNI.PIJCNNA. pages 41 15- 4128, Hong Kong, China. Association for Computa- tional Linguistics.
Ethics Statement
In consideration of ethical concerns, we would make the following descriptions: (1) All of our experiments are conducted using existing datasets sourced from publicly available scientific papers. (2) Our few-shot methods don't require a lot of computational resources. (3) Our text generation models will generate texts based on existing tem- plates, so it won't 't generate harmful sentences.
Xiang Dai, Sarvnaz Karimi, Ben Hachey, and Cecile Paris. 2020. An effective cansition-based model for discontinuous NER. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5860-5870, Online. Association for Computational Linguistics.
Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca Passonneau, and Rui Zhang. 2022. CONTaiNER: Few-shot named entity recognition via contrastive learning. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6338-6353, Dublin, Ireland. Association for Computational Linguistics.
References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 1377
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Pupers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Jiawei Chen, Qing Liu, Hongyu Lin, Xianpei Han, and Le Sun. 2022a. Few-shot named entity recognition with self-describing networks. In Proceedings of the 60th Annual Meeting of the Association for Compu- lational Linguistics (Volume 1: Long Papers), pages 5711-5722, Dublin, Ireland. Association for Compu- tational Linguistics.
Alexander Fritzler, Varvara Logacheva, and Maksim Kretov. 2019. Few-shot classification in named en- tity recognition task. In Proceedings of the 34th ACM/SIGAPP Symposium Applied Computing. pages 993-1000.
Xiang Chen, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si. Huajun Chen, and Ningyu Zhang. 2022b. LightNER: A lightweight tuning paradigm for low-resource NER via plug- gable prompting. In Proceedings of the 29th Inter- national Conference on Computational Linguisties, pages 2374-2387, Gyeongju, Republic of Korea. In- ternational Committee Computational Linguistics.
Tao Gui. Jiacheng Ye, Qi Zhang, Zhengyan Li, Zichu Fei. Yeyun Gong, and Xuanjing Huang. 2020. Uncertainty-aware label refinement for sequence la- beling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2316-2326, Online, Association for Computational Linguisties.
Yanru Chen, Yanan Zheng, and Zhilin Yang. 2022c. Prompt-based metric learning for few-shot ner. arXiv preprint 22X1:2211..43337

---

Dilek Hakkani-Tür, Gökhan Tür, Asli Celikyilmaz, Yun- Nung Chen, Jianfeng Gao, Li Deng, and Ye- Yi Wang. 2016. Multi-domain joint semantic frame parsing using bi-directional mn-lstm. In Interspeech, pages 715-719.
Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020b. A unified MRC framework for named entity recognition. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5849-5859, On- line. Association for Computational Linguistics.
Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, et al. 2021. Pre-trained models: Past, present and future. Al Open, 2:225-250.
Jingjing Liu, Panupong Pasupat, Scott Cyphers, and Jim Glass. 2013. Asgard: A portable architecture for multilingual dialogue systems. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 8386-8390. IEEE.
Yucheng Huang, Kai He, Yige Wang, Xianli Zhang. Tieliang Gong, Rui Mao, and Chen Li. 2022. COP- NER: Contrastive learning with prompt guiding for few-shot named entity recognition. In Proceedings of the 29th International Conference on Computational Linguistics, pages
Kum Liu, Yao Fu, Chuanqi Tan, Mosha Chen, Ningyu Zhang, Songfang Huang, and Sheng Gao. 2021. Noisy-labeled NER with confidence estimation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 3437-3445, Online. Association for Computa- tional Linguistics.
Gyeongju, Republic of Korea. International Committee Computational Linguistics.
Sarvnaz Karimi, Alejandro Metke-Jimenez, Madonna Kemp, and Chen Wang. 2015. Cadec: A corpus of ad- verse drug event annotations. Journal of biomedical informatics, 55:73-81.
Yijin Liu, Fandong Meng, Jinchao Zhang, Jinan Xu, Yufeng Chen, and Jie 7hou. 2019. GCDT: A global context enhanced deep transition architecture for se- quence labeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, pages 2431-2441, Florence, Italy, Associa- tion for Computational Linguistics.
J-D Kim, Tomoko Ohta. Yuka Tateisi, and Jun ichi Tsujii. 2003. Genia corpus semantically amno- tated corpus for bio-textmining. Bioinformatics, 19:supp__):1180-1188
Hong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, and Xiang Ren. 2022a. Good examples make faster learner: Simple demonstration- based learning for low-resource NER. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume
Jie Ma, Miguel Ballesteros, Srikanth Doss, Rishita Anubhai, Sunil Mallya, Yaser Al-Onaizan, and Dan Roth. 2022a. Label semantics for few shot named entity recognition. In Findings of the Association for Computational Linguisticx: ACI. 2022, pages 1956- 1971, Dublin, Ireland. Association for Computational Linguistics.
1: Long Papers), pages (687-2700) Dublin, Ireland. Association for Computational Linguistics.
Dong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng. Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, and Xiang Ren. 2022b. Good examples make faster learner: Simple demonstration based learning for low-resource NER. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pages 2687-2700, Dublin, Ireland. Association for Computational Linguistics.
Tingting Ma, Huiqiang Jiang, Qianhui Wu, Tiejun Zhao, and Chin- Yew Lin. 2022b. Decomposed meta- learning for few-shot named entity recognition. In Findings of the Association for Computational Lin- guistics: ACI. 2022, pages 1584-1.596 1,566, Dublin, Ire- land. Association for Computational Linguistics.
Xuezhe Ma and Eduard Hovy. 2016. End-to-end se- quence labeling via bi-directional LSTM-CNNS-CREE In Proceedings of the 54th Annual Meeting of the As- sociation for Computational Linguistics (Volume I: Long Papers), pages 1064-1074, Berlin, Germany. Association for Computational Linguisties.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. 2020. pre-training
BART: Denoising sequence-to-sequeee for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computa- tional Linguisties.
Alejandro Metke-Jimenez and Sarvnaz Karimi, 2016. Concept identification and normalisation for adverse drug event discovery in medical forums. In BMDID@ ISWC.
Xiaoya Li. Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020a. A unified MRC framework for named entity recognition. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5849-5859, On- line. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. The Journal of Machine Learning Research. 21(1):5485-5551

---

Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2021. Multitask prompted training en- ables zero-shot task generalization. arXiv preprint arXiv:2110.08207
Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang. and Xipeng Qiu. 2021. A unified generative framework for various NER subtasks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 lih International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 5808-5822, Online. Association for Computational Linguistics.
Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. Ad- vances in neural information processing systems, 30.
Jie Yang. Shuailong Liang, and Yue Zhang, 2018. De- sign challenges and misconceptions in neural se- quence labeling. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 3879-3888 Santa Fe. New Mexico, USA. As- sociation for Computational Linguistics.
Emma Strubell, Patrick Verga, David Belanger, and Andrew McCallum. 2017. Fast and accurate en- tity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 2670-2680, Copenhagen, Denmark. Association for Computational Linguistics.
Yi Yang and Arzoo Katiyar. 2020. Simple and effective few-shot named entity recognition with structured nearest neighbor learning. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 6365-6375 Online. Association for Computational Linguistics.
Buzhou Tang, Jianglu Hu, Xiaolong Wang. and Qingeai Chen. 2018. Recognizing continuous and discon- tinuous adverse drug reaction mentions from social media using lstm-crf. Wireless Communications & Mobile Computing (Online), 2018.
Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020a.
Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 6470- 6476, Online. Association for Computational Lin- guistics.
Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction the CONLL-2203 shared task: named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning HLT-NAACL 2003, pages 142- 147.
Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020b.
Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 6470- 6476, Online. Association for Computational Lin- guistics.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Bric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.1399 13971.
Ningyu Zhang, Shumin Deng, Zhen Bi, Haiyang Yu, Jiacheng Yang, Mosha Chen, Fei Huang, Wei Zhang, and Huajun Chen. 2020. OpenUE: An open toolkit of universal extraction from text. In Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing: System Demonstrations. pages 1-8, Online. Association for Computational Linguistics.
Jue Wang, Lidan Shou, Ke Chen, and Gang Chen, 2020. Pyramid: A layered model for nested named entity recognition. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 5918-5928 Online. Association for Computa- tional Linguistics.
A Appendix
Liwen Wang, Rumei Li, Yang Yan, Yuanmeng Yan, Sirui Wang, Wei Wu, and Weiran Xu. 2022. In- structionner: multi-task basedd gener- ative framework for few-shot ner. arXiv preprint arXiv:2203.03903.
In this section, we will discuss the remaining two NER settings: nested NER and discontinuous NER. Because the text-to-text structure of our proposed method can be easily adapted to all three NER settings, which will result a unified structure for solving NER problems. Here, we mainly discuss standard NER scenarios with abundant data.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M Dai, and Quoe V Le. 2021. Finetuned lan- guage models are zero-shot learners. arXiv preprint arX1v:2109.0162..
For data abundant nested NER, We conduct ex- periments Genia (Kim et al., 2003). We follow BARTNER (Yan et al., 2021) to use five entities types and split the train, dev, test as 1.1.9.9:1.0. The results are in Table
Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda. and Yuji Matsumoto. 2020. LUKE: Deep contextualized entity representations with entity- aware self-attention. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 6442 0454,,44 On- line. Association for Computational Linguistics.
For data abundant discontinuous NER, we con- duct experiments on CADEC (Karimi et al., 2015).

---

Genia: Model
P
R
et al., 2020b)/BRRT Largeli
81.25
76.36
78.72 72
(Yu al., 2020b)(BERT
79.43
78.32
78.87
(Wang et al., 2020(BERR
79.45
78.94
79.19
BARTNER (Yan et al., 2021)
78.87
79.6
79.23
2INER
82.9
80.74
81.81
Table 6: Span-F1 (%) Genia Nested data abundant NER setting. The "f" mean the reproduction by (Yan al., 2021).
CADEC: Model
P
(Metke-Jimenc) and Karimi, 2016)
64.4
56.5
(Tang et al., 2018)
67.8
64.9
(Dai et al., 2020][ELMo]
68.9
69.0
69.0
BARTNER (Yan et al., 2021)
70.08
71.21
70.64
2INER
71.18
75.26
73.16
Table 7: Span-Fl (%) CADEC discontinuous data abundant NER setting.
Following BARTNER (Yan al., 2021), since only the Adverse Drug Events (ADEs) entities include discontinuous data, only these entities were consid- ered. The results are in Table
The experiment settings are the same as flat NER. We use T5-large as the backbone model and report span-level FI. The results show that in data abun- dant nested and discontinuous NER setting, proposed method greatly outperforms BARTNER (Yan al., 2021) and other SOTA methods, which demonstrates that our methods do have poten- tial to handle different NER settings in a unified framework.


Reviews:
[{"review_id": "cp4VfgarxC", "reviewer": "Reviewer_Cuzv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> the proposed auxiliary task kinda lack intrinsic novelty\n\nWe have to admit that the proposed auxiliary task only constructs the model's input-output at the linguistic level and does not involve mathematical reasoning. But we believe that with the popularity of text-to-text large language models, selecting the task that the model performs at the linguistic level, without changing the model structure, is the most lightweight and straightforward fine-tuning approach. Therefor, we believe that discussing what kind of auxiliary tasks can help downstream NER learning is valuable.  \n\nAnd in terms of the novelty, we have abstracted the learning path of NER into three steps (instead of two), and demonstrated through experiments that our proposed auxiliary task can serve as a more fundamental NER ability, empowering both span recognition and entity labeling abilities. \n\nFurthermore, the proposed auxiliary task is efficient and easy to implement. By simply adding this auxiliary task, we were able to achieve significant improvement on downstream tasks, which also demonstrates the value of making linguistic modifications to model's input-output, despite their simplicity.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. a well structured work and easy to read\n2. validated on multiple datasets and report detailed empirical results\n3. the main method it-self seems to be good", "reasons_to_reject": "1. the proposed auxiliary task kinda lack intrinsic novelty", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a novel text-to-text framework for Few-Shot Named Entity Recognition tasks, and introduces an auxiliary task called typing extracting. Also, this work tries to improve performance via in-context examples. The efficiency is validated on several open-sourced datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bDK2iaPjv8", "reviewer": "Reviewer_WRbf", "summary": "", "questions": "One can argue that type extraction can be inferred from entity typing. For example, given the sentence \"are there any 4 star restaurant?\". Entity typing gives the result \"4 star is a rating\", which immediately leads to \"rating type exists in the sentence\". As a result, it's not clear whether training with type extraction can can actually help the model to understand semantic information of the input sentence and the labels better. ", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> It is not clear how much type extraction contributes to the overall performance of the NER model. Even though the author provided ablation study in section 4.5 and show the results in Table 2, the results don't illustrate the improvement brought up by type extraction as an auxiliary task. It would be nice if the authors can show the result of 2INER vs 2INER+TE.\n\nI think this problem is caused by a misunderstanding of the symbol in my article. And me myself also think that it may be confusing.\n\nIn line-470, I explained that \"The names InstructionNER and 2INER in the tables mean training with main-task data only\". The intention of my statement is that the meanings of \"InstructionNER\" and \"2INER\" in the table are equivalent (\"InstructionNER\" == \"2INER\"). \n\nIn other words, the meaning of the symbol can be simplified as follows:\n\n| Symbol                       | Training Tasks Details                                                                |\n|------------------------------|---------------------------------------------------------------------------------------|\n| InstructionNER & 2INER       | Main-Task                                                                             |\n| 2INER+EX          | Main-Task + Demonstrate Example                                                       |\n| 2INER+TE          | Main-Task + Type Extraction                                                           |\n| 2INER+ET,EE,TE,EX | Main-Task + Entity Typing + Entity Extraction + Type Extraction + Demonstrate Example |\n\nTherefore, the suggestion of 2INER vs 2INER+TE (Main-Task vs Main-Task + Type Extraction) has already been done in Table 2 (which is InstructionNER vs 2INER+TE). So we can find out in Figure 2 that adding TE tasks on the Main-Task greatly increase F1 (red vs green bars). And because the Main-Task in our work is the same as those in InstructionNER paper, so we copy their results in Table 2.\n\nI'm very sorry that this error has caused you trouble. And we admit that \"InstructionNER\" == \"2INER\" is confusing, so this confusing symbols will be modified. Thank you for pointing that out! \n\n> Demonstration example was shown to be effective for NER task in previous work, even though it is not sequence-to-sequence. Therefore, the result about improvement by demonstration example is not surprised.\n\nYes, you are right. Many previous works have shown the effectiveness of demonstration examples in other scenarios. And our work validates that it is equally effective in the text-to-text few-shot NER task.\n\n> One can argue that type extraction can be inferred from entity typing. For example, given the sentence \"are there any 4 star restaurant?\". Entity typing gives the result \"4 star is a rating\", which immediately leads to \"rating type exists in the sentence\". As a result, it's not clear whether training with type extraction can actually help the model to understand semantic information of the input sentence and the labels better.\n\nThanks for your question, and we think that it's a worthy topic of discussion. This question is related to the intention of adding any auxiliary task. \n\nFirstly, every ability related to solving a specific downstream task can be learned through Main-Task. For example, when we train the model to predict \"XXX span is a YYY entity\", the model learns everything in order to solve this NER problems (which may include Entity Typing, Entity Extraction and Type Extraction, but we are not sure about that because those skills are achieved through hidden model parameters). Therefore, we believe that it is unreasonable to claim that auxiliary tasks are useless simply because they are implicit in the Main-Task. If it were true, then all auxiliary tasks (ET, EE, TE) would be useless. This is because the Main-Task includes all the necessary skills required to complete the NER. \n\nSecondly, the intention of adding auxiliary tasks is to artificially design thinking processes for completing the NER task and decompose it into several subtasks. This process is similar to human supervision, because it directly informs the model which sub-skills it needs to learn in order to complete the NER task. Furthermore, they visualize the model's hidden parameters into natural language to indicate that model itself do learn the process to complete NER, which increases the model's interpretability.\n\nFinally, we believe the only way to ensure the auxiliary task actually help the Main-Task is through observing the F1 score. Because auxiliary tasks (including Type Extraction) themselves are linguistically interpretable, as long as the final F1 score of NER is increasing, we believe that it is reasonable to claim the effectiveness of Type Extraction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper proposed a novel auxiliary task, called type extracting, to improve the NER model's ability to understand the semantic relations between the sentence and the labels. \nThe proposed method is the first that utilizes demonstration example and show that this method improve NER few-shot performance. ", "reasons_to_reject": "It is not clear how much type extraction contributes to the overall performance of the NER model. Even though the author provided ablation study in section 4.5 and show the results in Table 2, the results don't illustrate the improvement brought up by type extraction as an auxiliary task. It would be nice if the authors can show the result of 2INER vs 2INER+TE.\nDemonstration example was shown to be effective for NER task in previous work, even though it is not sequence-to-sequence. Therefore, the result about improvement by demonstration example is not surprised.", "questions_to_authors": "One can argue that type extraction can be inferred from entity typing. For example, given the sentence \"are there any 4 star restaurant?\". Entity typing gives the result \"4 star is a rating\", which immediately leads to \"rating type exists in the sentence\". As a result, it's not clear whether training with type extraction can can actually help the model to understand semantic information of the input sentence and the labels better. ", "paper_topic_and_main_contributions": "In this work, the author proposed a new method for zero-shot named entity recognition using text sequence-to-sequence modeling and generates slot information in the input sentence as a sequence. In order to strengthen the model's ability to understand the semantic relation between entity labels and the sentence, the author proposed to train the model using a new auxiliary task called type extracting, which identifies what type of entities exist in the input sentence. Additionally, the proposed method concatenates demonstration example into the input sentence to incorporate external knowledge into the model. The authors showed that the proposed method significantly outperforms SOTA baselines in few-shot NER.  ", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GkN4iEHR9b", "reviewer": "Reviewer_aEJr", "summary": "", "questions": "- What distinguishes 2INER+EX from 2INER? Or rather, is 2INER without +EX  equal to InstructionNER\n-  Following this question, how is it then possible that 2INER+TE and 2INER+EX individually improve so much over InstructionNER, yet combining them only improves marginally over each modification individually? Have the authors looked into that?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The modifications made to InstructionNER are relatively minor and relatively straightforward additions. The new auxiliary loss is essentially the counterpart of the entity extraction loss for the entity types.\n\nWe admit that our modifications are not huge progresses based on InstructionNER. But the added Type Extraction task indeed help both Entity Typing and Entity Extraction abilities, and results in a great improvement in Few-Shot NER F1 scores.\n\nFurthermore, if we think of auxiliary tasks as a thinking process for completing the NER task，they are similar to chain of thought.\n - Step1 - Type Extraction: The Step 1 is to understand the meaning of the entity types through the overall semantic level of the sentence.\n - Step2 - Entity Extraction: Based on the label semantic, the Step 2 is to decide which span to extract. (If we have a deeper understanding of label semantic of entity types, we can focus more on the required spans instead of all spans, simplifying the task that the model needs to perform. And knowing the entity type better can absolutely help the model to extract related spans easier.)\n - Step3 - Entity Typing: After extract the span, the Step 3 is to properly typing these span (If the span extracted in Step 2 is more accurate, the typing will be easier).\n\nThe difference between these steps and chain of thought(COT) is that:\n - COT directly generate the thinking process by the model itself, because different tasks require different thinking process.\n - Because the thinking process of NER can be manually constructed, we can use auxiliary tasks to represent the chain of thought process. So that we can use gradient to update the parameters to directly ask the model to follow our constructed thinking process.\n - So there is no need to generate the COT process in the model output, because these knowledge has been updated by the gradient.\n\nSo we believe that the Type Extraction is not just the counterpart of Entity Extraction. It is the first step in the decomposed chain of thought process to solve the overall NER problem because understanding label semantic is crucial.\n\n\n> The experimental setup is confusing. Why are the authors using a different name for 2INER and InstructionNER when they are identical, apart from the subscripts added to them which specify each modification (based on the observation that Table 3, 0 2INER in-context examples matches InstructioNER results in Table 2)?\n\n> What distinguishes 2INER+EX from 2INER? Or rather, is 2INER without +EX equal to InstructionNER \n\nI'm very sorry that this error has caused you trouble. And we admit that \"InstructionNER\" == \"2INER\" is confusing.\n\nIn line-470, I explained that \"The names InstructionNER and 2INER in the tables mean training with main-task data only\". The intention of my statement is that the meanings of \"InstructionNER\" and \"2INER\" in the table are equivalent (\"InstructionNER\" == \"2INER\"). \n\nIn other words, the meaning of the symbol can be simplified as follows:\n\n| Symbol                       | Training Tasks Details                                                                |\n|------------------------------|---------------------------------------------------------------------------------------|\n| InstructionNER & 2INER       | Main-Task                                                                             |\n| 2INER+EX          | Main-Task + Demonstrate Example                                                       |\n| 2INER+TE          | Main-Task + Type Extraction                                                           |\n| 2INER+ET,EE,TE,EX | Main-Task + Entity Typing + Entity Extraction + Type Extraction + Demonstrate Example |\n\nAgain, we are truly sorry for the trouble in our symbol. This confusing symbols will be modified. Thank you for pointing that out!\n\n> Furthermore, the setup around the added in-context examples is not entirely clear to me: are the in-context examples different from the samples used for few-shot fine-tuning (this is what Table 3 suggests)? In that case, the performance comparison in Table 2 between 2INER and the other models would not be sound. Can the authors provide more details here?\n\nWe admit that the setup about in-context examples may not be very clear and I will discuss it more precisely.\n\nThe setup: \n - Training: During instruction fine-tuning, all the in-context examples we concatenated in the input are from few-shot support set. In other words, if the K-shot is 20 and we have 4 entity types, then we will have 20 * 4 = 80 examples in total ideally. And if we use 5 in-context examples during fine-tuning, 5 random examples will be sampled from 80 support set to formulate the in-context learning materials. And for each sample in training set, we will sample different 5 examples.\n - Inference: During inference, the in-context examples are from training support set (which is 80 examples in total). And for each sample in evaluation set, random 5 in-context examples will be sampled from those 80 support set. \n - To sum up, all instances in in-context examples are sampled from training support set. Therefor, there will be no information leakage during evaluation.\n\nAdditionally, in that case, in Table-3, we only do experiments under 20-Shot and 50-Shot settings without 10-Shot, because we don't have enough 15 in-context examples under 10-Shot setting. (only have 10 examples / entity types)\n\nWe apologize for not have clearer explanations related to these setups in our paper.\n\n> Following this question, how is it then possible that 2INER+TE and 2INER+EX individually improve so much over InstructionNER, yet combining them only improves marginally over each modification individually? Have the authors looked into that?\n\nYes, it is indeed a good question why the improvement combining them is relatively smaller. We believe that the reason may be the lack of data. As we mainly focus on few-shot settings, the highest level of performance that a model can achieve is limited by the amount of data available. So maybe the model has already hitting the upper boundary in two MIT dataset under few-shot settings given the sacle of the model(T5-large). And in ATIS dataset, the combining improvement and individual improvement are rather equivalent, so this phenomenon only exists in two MIT dataset. \n\nTherefore, we think it is reasonable to believe that small few-shot data scale and model scale limit the potential of the combining performance.\n\n> The paper can be written much more concisely. The authors do not very clearly distinguish the paper's contributions from the methodology of InstructionNER. For instance, mentioning flat/nested NER at the very beginning of the introduction (and in the related work section later on) is confusing -- the authors do not consider different NER settings in their paper. The following two chapters continue to not be directly relevant to the paper, making the paper difficult to follow. It is not until l.101 that the content becomes directly relevant to the paper's contribution. The authors should consider revising every section, and asking for every sentence if it is relevant to the reader.\n\nWe admit that the structure of the introduction may be confusing. And we have taken your suggestions into account to revise the paper.\n\nThe reason why we discuss different NER settings (doing main experiments on flatten NER, but discuss nested / discontinuous NER in introduction) is because we believe that our method can be easily adapted to all three NER settings, which will result in a unified structure for solving both NER and few-shot NER problems. \n\nBut we didn't conduct experiments in these two settings. And here is the reason we didn't include nested / discontinuous experiment in our paper:\n - As we mainly focus on few-shot settings, current literatures mainly discuss few-shot NER under flatten scenario, and there are little papers discuss nested / discontinuous few-shot settings. So it is difficult to find comparisons for them if we focus on few-shot.\n - The reason why little literature discuss few-shot nested / discontinuous NER is that nested / discontinuous instances only accounts for a small proportion. For example, CADEC [1](Karimi et al.,2015) (discontinuous) only have 10.6% of the mentions are discontinue and GENIA [2](Kim et al.,2003) (nested) only have 8.6% of the mentions are nested. So there will not be enough nested / discontinuous mentions in few-shot support set to train the model to handle these abnormal data.\n\nTo extend our paper and demonstrate that our methods do have a potential to handle different NER settings in a unified framework, we added these two experiments under data abundant setting:\n\n - We use T5-large as backbone and report span-level F1.\n - Data abundant Nested NER:\n   - We conduct experiments on GENIA [2](Kim et al.,2003). We follow BARTNER [3](Yan et al.,2021) to use five entities types and split the train, dev, test as 8.1:0.9:1.0. The results are in the following Table.\n   - The result shows that in data abundant nested NER setting, our proposed method greatly outperforms BARTNER and other SOTA methods.\n   - \n| GENIA                                 | P     | R     | F     |\n|---------------------------------------|-------|-------|-------|\n| Li et al. (2020b)[BERT-Large] [4]     | 81.25 | 76.36 | 78.72 |\n| Yu et al. (2020)[BERT-Large] [5]      | 79.43 | 78.32 | 78.87 |\n| Wang et al. (2020a)[BERT-Large] [6]   | 79.45 | 78.94 | 79.19 |\n| BARTNER [3]                           | 78.87 | 79.6  | 79.23 |\n| 2INER (Main-Task + ET + EE + TE + EX) | 82.9  | 80.74 | 81.81 |\n\n - Data abundant Discontinuous NER:\n   - We conduct experiments on CADEC [1](Karimi et al.,2015). Following BARTNER [3](Yan et al.,2021), since only the Adverse Drug Events (ADEs) entities include discontinuous data, only these entities were considered.\n   - The result shows that in data abundant discontinuous NER setting, our proposed method greatly outperforms BARTNER and other SOTA methods.\n   -\n| CADEC                                 | P     | R     | F     |\n|---------------------------------------|-------|-------|-------|\n| Metke-Jimenez and Karimi (2016) [7]   | 64.4  | 56.5  | 60.2  |\n| Tang et al. (2018) [8]                | 67.8  | 64.9  | 66.3  |\n| Dai et al. (2020)[ELMo] [9]           | 68.9  | 69.0  | 69.0  |\n| BARTNER [3]                           | 70.08 | 71.21 | 70.64 |\n| 2INER (Main-Task + ET + EE + TE + EX) | 71.18 | 75.26 | 73.16 |\n\nBecause we do have difficulty in finding comparison for few-shot nested / discontinuous setting and other reasons above, we didn't add few-shot experiments.\n\nThank you for mentioning it, and we will add these experiments as well as our revision for our introduction to our paper.\n\n\n> Type Extracting -> Type Extraction\n\nWe admit that this is a grammar issues, and we have already modified this problem in our paper. Thank you for correcting us.\n\n> A Figure showing text-to-text NER in the introduction would be nice\n\nThank you for your advice. And we have already made a new figure showing the text-to-text NER in our introduction. We also believe that adding a new figure can greatly make our paper easier to follow. Thanks a lot.\n\n\nReferences: \n\n[1]: Sarvnaz Karimi, Alejandro Metke-Jimenez, Madonna Kemp, and Chen Wang. 2015. Cadec: A corpus of adverse drug event annotations. J. Biomed. Informatics, 55:73–81.\n\n[2]: Jin-Dong Kim, Tomoko Ohta, Yuka Tateisi, and Jun’ichi Tsujii. 2003. GENIA corpus - a semantically annotated corpus for bio-textmining. In Proceedings of the Eleventh International Conference on Intelligent Systems for Molecular Biology, June 29 - July 3, 2003, Brisbane, Australia, pages 180–182.\n\n[3]: Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, and Xipeng Qiu. 2021. A unified generative framework for various NER subtasks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 1: Long Papers), pages 5808–5822, Online. Association for Computational Linguistics.\n\n[4]: Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020b. A unified MRC framework for named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5849–5859. Association for Computational Linguistics.\n\n[5]: Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 6470–6476. Association for Computational Linguistics.\n\n[6]: Jue Wang, Lidan Shou, Ke Chen, and Gang Chen. 2020a. Pyramid: A layered model for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5918–5928. Association for Computational Linguistics.\n\n[7]: Alejandro Metke-Jimenez and Sarvnaz Karimi. 2016. Concept identification and normalisation for adverse drug event discovery in medical forums. In Proceedings of the First International Workshop on Biomedical Data Integration and Discovery (BMDID 2016)co-located with The 15th International Semantic Web Conference (ISWC 2016), Kobe, Japan, October 17, 2016, volume 1709 of CEUR Workshop Proceedings. CEUR-WS.org.\n\n[8]: Buzhou Tang, Jianglu Hu, Xiaolong Wang, and Qingcai Chen. 2018. Recognizing continuous and discontinuous adverse drug reaction mentions from social media using LSTM-CRF. Wirel. Commun. Mob. Comput., 2018.\n\n[9]: Xiang Dai, Sarvnaz Karimi, Ben Hachey, and C´ecile Paris. 2020. An effective transition-based model for discontinuous NER. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5860–5870. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The modifications the authors made to InstructionNER improved results substantially on two out of three datasets\n- Ablation study and Analysis to understand which modification impacts performance and to which extent. \n- Results improve results beyond few-shot classification to many samples (tested up to 500), albeit the performance gap appears to decrease (Table 5, 500 samples MIT Restaurant)", "reasons_to_reject": "- The modifications made to InstructionNER are relatively minor and relatively straightforward additions. The new auxiliary loss is essentially the counterpart of the entity extraction loss for the entity types. \n- The experimental setup is confusing. Why are the authors using a different name for 2INER and InstructionNER when they are identical, apart from the subscripts added to them which specify each modification (based on the observation that Table 3, 0 2INER in-context examples matches InstructioNER results in Table 2)? Furthermore, the setup around the added in-context examples is not entirely clear to me: are the in-context examples different from the samples used for few-shot fine-tuning (this is what Table 3 suggests)? In that case, the performance comparison in Table 2 between 2INER and the other models would not be sound. Can the authors provide more details here?", "questions_to_authors": "- What distinguishes 2INER+EX from 2INER? Or rather, is 2INER without +EX  equal to InstructionNER\n-  Following this question, how is it then possible that 2INER+TE and 2INER+EX individually improve so much over InstructionNER, yet combining them only improves marginally over each modification individually? Have the authors looked into that?", "paper_topic_and_main_contributions": "The paper presents a text-to-text approach for Named Entity Recognition, extending InstructionNER by (i) adding an additional auxiliary task that tasks a model to predict the existence of entity types in an input, (ii) adding in-context learning in addition to fine-tuning. Experiments on three datasets in a few-shot setting indicate performance improvements over InstructionNER and all other few-shot baselines. An ablation study indicated the beneficial properties of both the auxiliary task as well as adding additional in-context learning.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- The paper can be written much more concisely. The authors do not very clearly distinguish the paper's contributions from the methodology of InstructionNER. For instance, mentioning flat/nested NER at the very beginning of the introduction (and in the related work section later on) is confusing -- the authors do not consider different NER settings in their paper. The following two chapters continue to not be directly relevant to the paper, making the paper difficult to follow. It is not until l.101 that the content becomes directly relevant to the paper's contribution. The authors should consider revising every section, and asking for every sentence if it is relevant to the reader.\n- Type Extracting -> Type Extraction\n- A Figure showing text-to-text NER in the introduction would be nice"}}]

Author rebuttal:

Meta-review: This paper proposes a text-to-text framework for few-shot named entity recognition by adopting some cases for in-context learning. The proposed method is not novel enough but shows the effectiveness on some open-domain scenarios. The experimental results have demonstrated the effectiveness of the proposed method. Although some reviewers have raised some concerns about some statements in the experiment, all of them generally agree with the point of the proposed method.
Decision: Accept-Findings