Structure and Label Constrained Data Augmentation for Cross-domain Few-shot NER (2023)

Abstract: Cross-domain few-shot named entity recognition (NER) is a challenging task that aims to recognize entities in target domains with limited labeled data by leveraging relevant knowledge from source domains. However, domain gaps limit the effect of knowledge transfer and harm the performance of NER models. In this paper, we analyze those domain gaps from two new perspectives, i.e., entity annotations and entity structures and leverage word-to-tag and word-to-word relations to model them, respectively. Moreover, we propose a novel method called Structure and Label Constrained Data Augmentation (SLC-DA) for Cross-domain Few-shot NER, which novelly design a label constrained pre-train task and a structure constrained optimization objectives in the data augmentation process to generate domain-specific augmented data to help NER models smoothly transition from source to target domains. We evaluate our approach on several standard datasets and achieve state-of-the-art or competitive results, demonstrating the effectiveness of our method in cross-domain few-shot NER.

## introduction named entity recognition (ner) is a fundamental natural language processing (nlp) task to detect entity mentions and classify them into predefined labels @xcite . benefiting from powerful feature representations, deep learning based ner approaches @xcite @xcite have achieved promising performances. however, their success depends heavily on the large-scale dataset with accurate annotations that are labor-intensive and time-consuming. although some general domains (e.g., news) provide rich annotations, it is unaffordable to manually annotate ner labels in some new environments (e.g., bio-medicine). therefore, * yufeng chen is the corresponding author. few-shot ner @xcite has attracted increasing attentions, aiming to build an ner model with only a small number of supporting samples in specific domains. mainstream researches on cross-domain fewshot ner aim to transfer relevant knowledge from source domains. most of them focus on optimizing model architectures based on metric-learning, such as prototypical network based protobert @xcite @xcite , a nearest neighbor based network nnshot @xcite , a viterbi decoding variant nearest neighbor based network structshot @xcite , and container @xcite , a contrastive model. there are also studies based on data augmentation methods, such as @xcite , which uses cross-lingual pre-trained models for data augmentation. cross-domain few-shot ner is full of challenges because of the domain gaps, especially for ner tasks. however, existing work lacks research on this problem. as a structure prediction task, ner requires synthetic entity with highly matching of label dependencies. however, there are distinctions among texts from various domains. the same entity mention in various domains is labeled with different entity types or distinct span boundaries. unfortunately, existing studies have not adequately explored the influence of domain gaps on ner tasks. consequently, these gaps significantly impact the performance of existing approaches. in view of these challenges, we divide domain gaps into two categories as shown in figure 1(a) . category-i: the structure of entities differs across domains. for example, 'the lincoln memorial' is represented as a contiguous location entity in flat ner datasets, while it is additionally labeled with a person entity "lincoln" in the nested ner datasets. additionally, entity boundaries may vary from domain to domain. for example, in different datasets, the inclusion of "the" in the phrase "the lincoln memorial" can vary. category-ii: the annotations of entities differs across domains. difference domains usually have different pre-defined entity types. for example, in ontonotes dataset, 'the lincoln memorial' and 'washington d.c' are annotated as 'loc' and 'gpe' types, respectively, where as in wnut dataset, both are classified as 'location'. our proposed method aims to alleviate the negative impact of structure and annotations differences on cross-domain few-shot ner. by doing so, we aim to enhance the performance of ner models in the target domain. based on the above analyses, we introduce two types of relationships to sufficiently model two kinds of domain gaps in cross domain few-shot ner method. word-to-word relation: 'the washington monument' is annotated as one entity in flat ner datasets while annotated as two entities in nested ones. when the source domain is a flat entity dataset and the target domain is a nested entity dataset, it is likely to generate non-nested entity data, leading to the ner model not being able to learn the knowledge in the target domain. word-to-tag relation: the corresponding entity types of 'washington d.c.' are 'gpe' and 'location' in 'ontonotes' and 'wnut', respectively, which may cause label conflict if directly learned. in this paper, we propose a novel method called structure and label constrained data augmentation (slc-da) for cross-domain fewshot ner. slc-da novelly design a label constrained pre-train task, which allows the model to learn the mapping relationships between entities across diverse domains. furthermore, slc-da incorporate structure constrained optimization objectives in the data augmentation process to generate domain-aware augmented data to help ner models smoothly transition from source to target domains. concretely, for structure-constrained data augmentation as shown in figure 1 (b), we calculate the word-to-word relation to model the entity structure between entity word tokens and other tokens, and generate structure-enhanced ner data in the target domain for training. for label-constrained data augmentation as shown in figure 1(c), we replace the same entity mentions with their corresponding different categories for each instance and utilize the language model to learn these word-to-tag relationships in different domains to avoid confusion. to evaluate the effectiveness of our proposed approach, we compare our approach with previous works on flat ner and report results surpassing the current state-of-the-art. additionally, we report competitive results on nested ner. our findings demonstrate that our proposed method is simple yet highly effective. finally, our main contributions are summarized as follows: ## method in this section, we present our proposed method, called structure and label constrained data augmentation (slc-da) for cross-domain few-shot ner. figure 2 depicts the overview of our method, which includes two modules: structure constrained data augmentation and label constrained pre-train task. we illustrate the details of how we learn entity structure and label relation. ## experiments we validate our proposed method in the flat entity and nested entity settings. the details of the experiments are elaborated in this section. we use precision (p), recall (r), and f (f1) as evaluation metrics. all experimental results are the average score over five runs with random seeds. ## conclusion in this paper, we propose a structure and label constrained data augmentation (slc-da) for cross-domain few-shot ner, by introducing entity structure and label information from various domains in the data augmentation process, to obtain high-quality synthetic data in the target domain. experimental results on both flat and nested few-shot ner tasks show that our method can significantly improve the quality of generated data and help ner model find more target domain entities.