P rompt NER : Prompt Locating and Typing for Named Entity Recognition (2023)

Abstract: Prompt learning is a new paradigm for utilizing pre-trained language models and has achieved great success in many tasks. To adopt prompt learning in the NER task, two kinds of methods have been explored from a pair of symmetric perspectives, populating the template by enumerating spans to predict their entity types or constructing type-specific prompts to locate entities. However, these methods not only require a multi-round prompting manner with a high time overhead and computational cost, but also require elaborate prompt templates, that are difficult to apply in practical scenarios. In this paper, we unify entity locating and entity typing into prompt learning, and design a dual-slot multi-prompt template with the position slot and type slot to prompt locating and typing respectively. Multiple prompts can be input to the model simultaneously, and then the model extracts all entities by parallel predictions on the slots. To assign labels for the slots during training, we design a dynamic template filling mechanism that uses the extended bipartite graph matching between prompts and the ground-truth entities. We conduct experiments in various settings, including resource-rich flat and nested NER datasets and low-resource in-domain and cross-domain datasets. Experimental results show that the proposed model achieves a significant performance improvement, especially in the cross-domain few-shot setting, which outperforms the state-of-the-art model by +7.7% on average.

## introduction named entity recognition (ner) is a fundamental task in natural language processing that aims to identify specific types of entities in free text, such as person, location, and organization. traditional sequence labeling methods (ma and hovy, † corresponding author. 1 our code will be available at https://github.com/ tricktreat/promptner . 2016) have difficulty coping with nested entities, and recent works have transformed ner into other paradigms such as reading comprehension @xcite , set prediction @xcite and sequence generation @xcite @xcite . however, low-resource and cross-domain problems in practical scenarios still pose a great challenge to ner models. recently prompt learning @xcite @xcite has received a lot of interest because of its excellent performance and data efficiency, and has been adopted in many classification and generation tasks @xcite @xcite . prompt learning converts downstream tasks into language modeling tasks, where cloze questions are constructed as prompts to guide pre-trained language models to fill in the blanks. however, named entity recogni-tion is a token-level tagging task, and it is difficult to apply prompt-based learning on ner directly @xcite . @xcite proposes the template-based method, which constructs prompts for each potential entity span and then separately predicts their entity types. for example, given an input "jobs was born in san francisco", @xcite enumerates each span to populate [x] of the template "[x] is a [mask] entity", and then determines the type of the filled span based on the prediction on the [mask] slot. in contrast to entity typing over the enumerated spans, some methods @xcite design prompt templates from a symmetric perspective. they construct prompts for each entity type and then guide the model to locate specific types of entities. for example, @xcite constructs the prompt "what is the location?" for the loc type, and then predicts all loc entities in the sentence, e.g., "san francisco". we group these two types of methods into span-oriented and type-oriented prompt learning. as shown in figure 1 , they construct prompts based on the entity span or entity type, and then perform entity typing or entity locating. however, both groups of methods require multiple rounds of prompting. for an input with n words and c pre-fixed types, type-oriented and span-oriented prompt learning require c and n (n -1)/2 predictions, respectively. moreover, each round of prediction is independent of the other, ignoring the latent relationships between different entities. different from the above methods that either perform multiple rounds of entity typing or entity locating through prompting, in this paper, we propose a prompt learning method for ner (promptner) that unifies entity locating and entity typing into one-round prompt learning. specifically, we design the position slot [p] and the type slot [t] in the prompt template, which are used for prompting entity locating and typing accordingly. this manner is enumeration-free for entity span or entity type, and can locate and classify all entities in parallel, which improves the inference efficiency of the model. since the correspondence between prompts and entities cannot be specified in advance, we need to assign labels to the slots in the prompts during training. inspired by @xcite , we treat the label assignment process as a linear assignment problem and perform bipartite graph match problem between the prompts and the entities. we further extend the traditional bipartite graph matching and design a one-to-many dynamic template filling mechanism so that an entity can be predicted by multiple prompts, which can improve the utilization of prompts. to summarize, our main contributions are as follows: ## method in this section, we first introduce the task formulation in § 3.1, and then describe our method. the overview of the promptner is shown in figure 2 , and we will introduce the prompt construction in § 3.2 and the model inference in § 3.3, including the encoder and the entity decoding module. the training of the model requires assigning labels to the slots of the prompt, and we will introduce the dynamic template filling mechanism in § 3.4. ## experiments to verify the effectiveness of promptner in various settings, we conduct extensive experiments in flat and nested ner ( § 4.3) and low-resource ner, including in-domain few-shot setting ( § 4.4) and cross-domain few-shot setting ( § 4.5). ## conclusion in this paper, we unify entity locating and entity typing in prompt learning for ner with a dual-slot multi-prompt template. by filling position slots and type slots, our proposed model can predict all entities in one round. we also propose a dynamic template filling mechanism for label assignment, where the extended bipartite graph matching assigns labels to the slots in a one-to-many manner. we conduct extensive experiments in various settings including flat and nested ner and low-resource in-domain and cross-domain ner, and our model achieves superior performance compared to the competitive baselines.