Coarse-to-fine Few-shot Learning for Named Entity Recognition (2023)

Abstract: Recently, Few-shot Named Entity Recognition has received wide attention with the growing need for NER models to learn new classes with minimized annotation costs. However, one common yet understudied situation is to transfer a model trained with coarse-grained classes to recognize fine-grained classes, such as separating a product category into sub-classes. We find that existing few-shot NER solutions are not suitable for such a situation since they do not consider the sub-class discrimination during coarse training and various granularity of new classes during few-shot learning. In this work, we introduce the Coarse-to-fine Few-shot NER (C2FNER) task and propose an effective solution. Specifically, during coarse training, we propose a cluster-based prototype margin loss to learn group-wise discriminative representations, so as to benefit fine-grained learning. Targeting various granularity of new classes, we separate the coarse classes into extra-fine clusters and propose a novel prototype retrieval and bootstrapping algorithm to retrieve representative clusters for each fine class. We then adopt a mixture prototype loss to efficiently learn the representations of fine classes. We conduct experiments on both in-domain and cross-domain C2FNER settings with various target granularity, and the proposed method shows superior performance over the baseline methods.

## introduction named entity recognition (ner), aiming at recognizing named entities such as person names, locations, and organizations from unstructured text, is a fundamental task for nlp applications @xcite @xcite . with the emerging of knowledge from various domains and the emerging of new entity types, it is difficult to manually annotate a large-scale highquality dataset for each new application scenario @xcite . under this circumstance, few-shot ner has become a crucial task, which studies ner systems that can fast adapt to new domains or unseen entity types with minimized annotation efforts. recently, an increasing number of researchers have focused on few-shot ner and made contributions to this task @xcite @xcite @xcite . unfortunately, previous works on few-shot ner typically assume the new classes are of the same or similar level of granularity to the known classes @xcite . in practice, a common situation is that the model needs to extend to more fine-grained classes from existing coarse-grained classes @xcite . such a situation typically occurs during the lifespan of the model when the application requires separating some sub-classes from the current classes and yet when the model is pre-trained without any knowledge of the subclasses. for example, a product recognizer initially built with a "computer" class may need to separate into sub-classes such as "notebook computer" and "tablet computer" as the requirement updates. naturally, relabeling a high-quality dataset each time is much too costly to be an option @xcite . in this work, we introduce the coarse-tofine few-shot ner (c2fner) task. we find that existing few-shot ner solutions are less suitable for c2fner for two reasons: (i) when training on the coarse-grained classes, most of the existing methods aim at maximizing the interclass boundaries yet neglect the intra-class variance @xcite , which might result in reducing separability of sub-classes, thus hindering few-shot fine-grained learning. (ii) as the granularity of new classes is unknown during coarse training, it is challenging to provide a solution that can benefit various granularity of new classes. this requires specific designs for cross-granularity few-shot learning, which is not considered in general few-shot ner methods. in this work, we propose a novel and effective solution for c2fner. firstly, in order to promote the intra-class variance in a way that best benefits cross-granularity learning, we propose a cluster-based prototype margin loss that improves group-wise discrimination in the coarse classes. specifically, we first perform clustering on the representations of coarse-grained model to obtain entity clusters. then, we minimize the distance of each entity with its prototype while maximizing its distance with negative prototypes from other coarse classes. second, in order to benefit various granularity of new classes, we separate each coarse class into extra-fine clusters and propose a novel prototype retrieval algorithm to retrieve representative prototypes (i.e., representative clusters) for each fine-grained classes. by initialize each fine-grained class with multiple prototypes, we then adopt a mixture prototype loss for efficiently learning the representations of fine classes with only few samples. we also bootstrap the prototypes of each fine class with a knn-based method during few-shot training, so as to make full use of the coarse model. as the first attempt on c2fner, we evaluate the proposed method and baselines on various practical c2fner settings, including (1) the in-domain coarse-to-fine setting where the fine-grained classes are strictly included in and inferior to each coarsegrained class; and (2) the cross-domain coarseto-fine setting where the fine classes might not strictly included in the coarse classes. additionally, we evaluate the methods with various granularity of new classes. experimental results show the superiority of the proposed method over baseline methods on multiple c2fner settings. 1 to sum up the contribution of this work: ## problem setup in this work, we introduce the coarse-to-fine fewshot ner task (c2fner). formally, we denote by y @xmath0, . . . , c r } a set of r coarse classes and let y f @xmath1, . . . , c f } be a set of fine classes. in our experiments, we explore both the situations when the fine classes are strictly included or not in the coarse classes. when training on the coarse classes, we assume an adequate number of n training instances s coarse @xmath2, y j )} n @xmath3. when tested on the fine classes, only a k-shot training set s f ine @xmath4, y j )} n k @xmath5, y j )} m @xmath6, where k-shot means each fine class is annotated with only k entities. our goal is to train a model m on the coarse training set s coarse train and then fine-tune it on the k-shot fine training set s f ine train to obtain maximal performance on the fine test set s f ine test . ## methodology as shown in figure 2 , in this work, we propose a novel and effective method for c2fner, which improves the performance from both the coarse and fine training phases: (1) during coarse training, we propose a cluster-based prototype margin loss to learn group-wise discriminative representations that benefit fine-grained learning. (2) during fine training, we improve performance for various fine granularity by retrieving representative clusters for each fine class with a prototype retrieval and bootstrapping algorithm and then training with a mixture prototype loss. ## conclusion in this work, we introduce the coarse-to-fine fewshot ner (c2fner) task, which considers the practical situation when the models are required to be updated to more fine-grained classes. to solve the c2fner task, we propose a novel method, which learns fine-friendly representations via a cluster-based prototype margin loss, and further benefits few-shot learning for various fine granularity via a prototype retrieval and mixture prototype learning method. we provide a comprehensive benchmark for c2fner with different settings, where the proposed method consistently outperforms all baseline methods.