Decomposed Meta-Learning for Few-Shot Named Entity Recognition (2022)

Abstract: Few-shot named entity recognition (NER) systems aim at recognizing novel-class named entities based on only a few labeled examples. In this paper, we present a decomposed meta-learning approach which addresses the problem of few-shot NER by sequentially tackling few-shot span detection and few-shot entity typing using meta-learning. In particular, we take the few-shot span detection as a sequence labeling problem and train the span detector by introducing the model-agnostic meta-learning (MAML) algorithm to find a good model parameter initialization that could fast adapt to new entity classes. For few-shot entity typing, we propose MAML-ProtoNet, i.e., MAML-enhanced prototypical networks to find a good embedding space that can better distinguish text span representations from different entity classes. Extensive experiments on various benchmarks show that our approach achieves superior performance over prior methods.

## introduction named entity recognition (ner) aims at locating and classifying text spans into pre-defined entity classes such as locations, organizations, etc. deep neural architectures have shown great success in fully supervised ner @xcite @xcite with a fair amount of labeled data available for training. however, in practical applications, ner systems are usually expected to rapidly adapt to some new entity types unseen during training. it is costly while not flexible to collect a number of additional labeled data for these types. as a result, the problem of few-shot ner, which involves learning unseen entity types from only a few labeled examples for each class (also known as support examples), has attracted considerable attention from the research community in recent years. previous studies on few-shot ner are typically based on token-level metric learning, in which a model compares each query token to the prototype @xcite of each entity class or each token of support examples and assign the label according to their distances @xcite @xcite . alternatively, some more recent attempts have switched to span-level metric-learning @xcite to bypass the issue of token-wise label dependency while explicitly utilizing phrasal representations. however, these methods based on metric learning might be less effective when encountering large domain gap, since they just directly use the learned metric without any further adaptation to the target domain. in other words, they do not fully explore the information brought by the support examples. there also exist additional limitations in the current methods based on span-level metric learning. first, the decoding process requires careful handling of overlapping spans due to the nature of span enumeration. second, the class prototype corresponding to non-entities (i.e., prototype of the "o" class) is usually noisy because non-entity common words in the large vocabulary rarely share anything together in common. moreover, when targeting at a different domain, the only available information useful for domain transfer is the limited number of support examples. unfortunately, these key examples are only used for inference-phase similarity calculation in previous methods. to tackle these limitations, this paper presents a decomposed meta-learning framework that addresses the problem of few-shot ner by sequentially conducting few-shot entity span detection and few-shot entity typing respectively via meta-learning. specifically, for few-shot span detection, we model it as a sequence labeling problem to avoid handling overlapping spans. note that the detection model aims at locating named entities and is classagnostic. we only feed the detected entity spans to the typing model for entity class inference, and hence the problem of noisy "o" prototype could also be eliminated. when training the span detector, we specifically use the model-agnostic metalearning (maml) @xcite algorithm to find a good model parameter initialization that could fast adapt to new entity classes with learned class-agnostic meta-knowledge of span boundaries after updating with the target-domain support examples. the boundary information of domain-specific entities from the support examples is supposed to be effectively leveraged via these update steps such that the model could better transfer to the target domain. for few-shot entity typing, we implement the typing model with standard prototypical networks @xcite , and propose maml-protonet to narrow the gap between source domains and the target domain. compared with protonet which only uses support examples for inference-phase similarity calculation, the proposed maml-proto additionally utilizes these examples to modify the shared embedding space of spans and prototypes by clustering spans representations from the same entity class while dispersing those from different entity classes for more accurate predictions. we evaluate our proposed framework on several benchmark datasets with different few-shot settings. experimental results show that our framework achieves superior performance over previous state-of-the-art methods. we also conduct qualitative and quantitative analyses over how the different strategies to conduct meta-learning might affect the performance. ## task definition given an input sequence @xmath0, an ner system is supposed to output a label sequence @xmath1, where x i is the i-th token, y @xmath2, y is the pre-defined entity class set, and o denotes non-entities. in this paper, we focus on the standard n -way k-shot setting as in @xcite . an example of 2-way 1-shot episode is shown in here, y new denotes the set of entity classes with a cardinality of n . note that, target types y [person-actor], [art-film] support set s (1) jack gordon[person-actor] ( born 27 june 1985 ) is an english actor . (2) this location had also been used to shoot the film " saving private ryan [art-film] " . query set q kurland starred in " taps " , which won first prize at the rhode island international film festival in 2006 . expected output kurland [person-actor] starred in " taps [art-film] " , which won first prize at the rhode island international film festival in 2006 . ## methodology figure 1 illustrates the overall framework of our decomposed meta-learning approach for few-shot named entity recognition. it is composed of two steps: entity span detection and entity typing. ## conclusion this paper presents a decomposed meta-learning method for few-shot ner problem, i.e., sequentially tackle few-shot span-detection and few-shot entity typing using meta-learning. we formulate the few-shot span detection as a sequence labeling problem and employ maml to learn a good parameter initialization, which enables the model to fast adapt to novel entity classes by fully exploring information contained in support examples. for fewshot entity typing, we propose maml-protonet, which can find a better embedding space than conventional protonet to represent entity spans from different classes more distinguishably, thus making more accurate predictions. extensive experiments on various benchmarks show that our approach achieves superior performance over prior methods.