S pan P roto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition (2022)

Abstract: Few-shot Named Entity Recognition (NER) aims to identify named entities with very little annotated data. Previous methods solve this problem based on token-wise classification, which ignores the information of entity boundaries, and inevitably the performance is affected by the massive non-entity tokens. To this end, we propose a seminal span-based prototypical network (SpanProto) that tackles few-shot NER via a two-stage approach, including span extraction and mention classification. In the span extraction stage, we transform the sequential tags into a global boundary matrix, enabling the model to focus on the explicit boundary information. For mention classification, we leverage prototypical learning to capture the semantic representations for each labeled span and make the model better adapt to novel-class entities. To further improve the model performance, we split out the false positives generated by the span extractor but not labeled in the current episode set, and then present a margin-based loss to separate them from each prototype region. Experiments over multiple benchmarks demonstrate that our model outperforms strong baselines by a large margin.

## introduction named entity recognition (ner) is one of the crucial tasks in natural language processing (nlp), which aims at extracting mention spans and classifying them into a set of pre-defined entity type classes. previous methods @xcite @xcite @xcite present multiple figure 1 : an example of the 2-way 1-shot ner problem. given a support set with 2 target types where each type is associated with 1 labeled entity, the task is to identify entities in the query example. to solve the problem, multiple methods @xcite @xcite @xcite follow the sequence labeling strategy that directly categorizes the entity type at the token level by calculating the distance between each query token and the prototype of each entity type class or the support tokens. however, the effect of these approaches are disturbed by numerous non-entity tokens (i.e. "o" class) and the token-wise label dependency (i.e. "bio" rules) @xcite . to bypass these issues, a branch of two-stage methods arise to decompose ner into two separate processes @xcite @xcite , including span extraction and mention classification. specifically, they first extract multiple spans via a class-agnostic model and then assign the label for each predicted span based on metric learning. despite the success, there could still be two remaining problems. 1) the performance of span extraction is the upper limit for the whole system, which is still far away from satisfaction. 2) previous methods ignore false positives generated during span extraction. intuitively, because the decomposed model is class-agnostic in the span extraction stage, it could generate some entities which have no available entity type to be assigned in the target type set. in figure 1 , the model could extract a span of "august 16, 1954" (may be an entity of time type in another episode), yet, the existing methods still assign it a label as "per" or "loc" foot_0 . to address these limitations, we present a novel span-based prototypical network (span-proto) via a two-stage approach. for span extraction, we introduce a span extractor, which aims to find the candidate spans. different from the recent work @xcite which models it as a sequence labeling task, we convert the sequential tags to a global boundary matrix, which denotes the sentence-level target label, enabling the model to learn the explicit span boundary information regardless of the token-wise label dependency. for mention classification, we propose a mention classifier which aims at assigning a pre-defined entity type for each recalled span. when training the mention classifier, we compute the prototype embeddings for each entity type class based on the support examples, and leverage the prototypical learning to adjust the span representations in the semantic space. to address the problem of false positives, we additionally design a margin-based loss to enlarge the semantic distance between false positives and all prototypes. we conduct extensive experiments over multiple benchmarks, including few-nerd @xcite and crossner @xcite . results show that our method consistently outperforms state-of-the-art baselines by a large margin. we summarize our main contributions as follows: ## related work in this section, we briefly summarize the related work in various aspects. ## our proposal: spanproto we formally present the notations and the techniques of our proposed spanproto. the model architecture is shown in figure 2 . ## conclusion we propose a span-based prototypical network (spanproto) for few-shot ner, which is a twostage approach includes span extraction and mention classification. to improve the performance of the span extraction, we introduce a global boundary matrix to urge the model to learn explicit boundary information regardless of token-wise label dependency. we further utilize prototypical learning to adjust the span representations and solve the issue of false positives by the margin-based learning objective. extensive experiments demonstrate that our framework consistently outperforms strong baselines. in the future, we will extend our framework to other nlp tasks, such as slot filling, part-of-speech (pos) tagging, etc.