SEE -Few: Seed, Expand and Entail for Few-shot Named Entity Recognition (2022)

Abstract: Few-shot named entity recognition (NER) aims at identifying named entities based on only few labeled instances. Current few-shot NER methods focus on leveraging existing datasets in the rich-resource domains which might fail in a training-from-scratch setting where no source-domain data is used. To tackle training-from-scratch setting, it is crucial to make full use of the annotation information (the boundaries and entity types). Therefore, in this paper, we propose a novel multi-task (Seed, Expand and Entail) learning framework, SEE-Few, for Few-shot NER without using source domain data. The seeding and expanding modules are responsible for providing as accurate candidate spans as possible for the entailing module. The entailing module reformulates span classification as a textual entailment task, leveraging both the contextual clues and entity type information. All the three modules share the same text encoder and are jointly learned. Experimental results on several benchmark datasets under the training-from-scratch setting show that the proposed method outperformed several state-of-the-art few-shot NER methods with a large margin. Our code is available at https://github.com/unveiled-the-red-hat/SEE-Few .

## introduction named entity recognition (ner), focusing on identifying mention spans in text inputs and classifying them into the pre-defined entity categories, is a fundamental task in natural language processing and widely used in downstream tasks @xcite @xcite . supervised ner has been intensively studied and yielded significant progress, especially with the aid of pretrained language models @xcite @xcite @xcite @xcite . however, supervised ner relies on plenty of training data, which is not suitable for some specific situations with few training data. few-shot ner, aiming at recognizing entities based on few labeled instances, has attracted much attention in the research filed. approaches for fewshot ner can be roughly divided into two categories, span-based and sequence-labeling-based methods. span-based approaches enumerate text spans in input texts and classify each span based on its corresponding template score @xcite . sequence-labeling-based approaches treat ner as a sequence labeling problem which assigns a tag for each token using the bio or io tagging scheme @xcite @xcite . most of these spanbased and sequence-labeling-based methods focus on leveraging existing datasets in the rich-resource domains to improve their performance in the lowresource domains. unfortunately, the gap between the source domains and the target domains may hinder the performance of these methods @xcite . moreover, these approaches might fail under the training-from-scratch setting where no source domain data is available. therefore, it is crucial to make full use of the indomain annotations, which consist of two types of information: boundary information and entity type information. however, most the approaches mentioned above fail to fully utilize these information. to overcome the disadvantages mentioned above, firstly, inspired by three principles for weakly-supervised image segmentation, i.e. seed, expand and constrain @xcite , we seed with relatively high-quality unigrams and bigrams in the texts, then expand them to extract the candidate spans as accurately as possible. secondly, we cast span classification as textual entailment to naturally incorporate the entity type information. for example, to determine whether "j. k. rowling" in "j. k. rowling is a british author." is a person entity or a non-entity, we treat "j. k. rowling is a british author." as a premise, then construct "j. k. rowling is a person." and "j. k. rowling is not an entity." as hypotheses. in such way, span classification is converted into determining which hypothesis is true. moreover, the size of training data is increased by such converting which is beneficial for few-shot settings. in this paper, we propose see-few, a novel multi-task learning framework (seed, expand and entail) for few-shot ner. the seeding and expanding modules are responsible for providing as accurate candidate spans as possible for the entailing module. specifically, the seed selector chooses some unigrams and bigrams as seeds based on some metrics, e.g., the intersection over foreground. the expanding module takes a seed and the window around it into account and expands it to a candidate span. compared with enumerating all possible n-gram spans, seeding and expanding can significantly reduce the number of candidate spans and alleviate the impact of negative spans in the subsequent span classification stage. the entailing module reformulates a span classification task as a textual entailment task, leveraging contextual clues and entity type information to determine whether a candidate span is an entity and what type of entity it is. all the three modules share the same text encoder and are jointly learned. experiments were conducted on four ner datasets under training-from-scratch few-shot setting. experimental results show that the proposed approach outperforms several state-of-the-art baselines. the main contributions can be summarized as follows: ## conclusion in this work, we propose a novel multi-task (seed, expand and entail) learning framework, see-few, for few-shot ner without using source domain data. the seeding and expanding modules are responsible for providing as accurate candidate spans as possible for the entailing module. the entailing module reformulates span classification as a textual entailment task, leveraging both the contextual clues and entity type information. all the three modules share the same text encoder and are jointly learned. to investigate the effectiveness of the proposed method, extensive experiments are conducted under the training-from-scratch few-shot setting. the proposed method outperforms other state-of-the-art few-shot ner methods by a large margin. for future work, we will combine the framework with contrastive learning to effectively make use of limited data and further enhance the performance of few-shot ner.