Task-adaptive Label Dependency Transfer for Few-shot Named Entity Recognition (2023)

Abstract: Named Entity Recognition (NER), as a crucial subtask in natural language processing (NLP), suffers from limited labeled samples (a.k.a. few-shot). Meta-learning methods are widely used for few-shot NER, but these existing methods overlook the importance of label dependency for NER, resulting in suboptimal performance. However, applying meta-learning methods to label dependency learning faces a special challenge, that is, due to the discrepancy of label sets in different domains, the label dependencies can not be transferred across domains. In this paper, we propose the Task-adaptive Label Dependency Transfer (TLDT) method to make label dependency transferable and effectively adapt to new tasks by a few samples. TLDT improves the existing optimization-based meta-learning methods by learning general initialization and individual parameter update rule for label dependency. Extensive experiments show that TLDT achieves significant improvement over the state-of-the-art methods.

## introduction named entity recognition (ner) aims to locate spans of text and classify them into the pre-defined entity categories such as person, location and organization @xcite . in practice, ner is troubled with annotated data scarcity, known as few-shot ner. in recent years, meta-learning is usually used for few-shot ner @xcite . meta-learning first learns prior knowledge from rich-source domains, and then the prior knowledge is used to guide ner models to adapt to an unseen target domain by a few samples. it is worth noting that, as a sequence labeling task @xcite , few-shot ner also benefits from considering the dependencies between labels @xcite @xcite . however, meta-learning for label dependency faces a special challenge. there is a label discrepancy between ner tasks in different domains, that is, both the number of labels and the label name semantics are different. the label discrepancy blocks the transfer of label dependencies from source domains to a new domain. for example, the label dependency of "b-company"-"b-project" (financial domain) can not be used for "b-drugs"-"b-etiology" (medical domain). to remedy this, some studies @xcite @xcite use linear conditional random field (crf) @xcite ) framework to model the label dependency as transition probabilities from abstract labels "b", "i" and "o" to "sb", "db", "si", "di" and "o", where "b"-"sb" means their domainspecific label semantics are the same (like "bweather"-"b-weather") and "b"-"db" means they are different (like "b-weather"-"b-time"). the abstract label transition table is shown in figure 1 (a). for example, value 0.1 in green denotes the transition probability from "b" to "sb". to adapt to a new domain, the abstract label transition is expanded into the domain-specific transition. figure 1 shows the expansion process, where positions in the same color are filled by the same values. for example, the value of "b-time"-"b-weather" is filled with that of "b" -"db", i.e., 0.2 in yellow. ## experiment this section experimentally evaluates the performance of tldt on multiple few-shot ner datasets. first, we introduce the experiment setting. second, we compare tldt against existing methods in few-shot ner scenarios to demonstrate the effectiveness of tldt. then we design ablation experiments to demonstrate the necessity both of each component of tldt and the features constructed for update rule learning. finally, we give the effectiveness analysis of tldt that can improve label dependency transfer on few-shot ner. ## conclusion in this paper, we propose an optimization-based meta-learning approach to transfer label dependency for few-shot ner. to make label dependency transferable and effectively adapt to new tasks, we propose a general initialization learning mechanism that learns shared initial label dependency and can quickly adapt to new tasks with a few gradient updates. in addition, we also propose an update rule to generate task-adaptive hyperparameters for label dependency optimization in a specific task. experiment results validate that both the general initialization learning and update rule learning can improve the few-shot ner accuracy.