Enhancing Low-resource Fine-grained Named Entity Recognition by Leveraging Coarse-grained Datasets (2023)

Abstract: Named Entity Recognition (NER) frequently suffers from the problem of insufficient labeled data, particularly in fine-grained NER scenarios. Although K -shot learning techniques can be applied, their performance tends to saturate when the number of annotations exceeds several tens of labels. To overcome this problem, we utilize existing coarse-grained datasets that offer a large number of annotations. A straightforward approach to address this problem is pre-finetuning, which employs coarse-grained data for representation learning. However, it cannot directly utilize the relationships between fine-grained and coarse-grained entities, although a fine-grained entity type is likely to be a subcategory of a coarse-grained entity type. We propose a fine-grained NER model with a Fine-to-Coarse(F2C) mapping matrix to leverage the hierarchical structure explicitly. In addition, we present an inconsistency filtering method to eliminate coarse-grained entities that are inconsistent with fine-grained entity types to avoid performance degradation. Our experimental results show that our method outperforms both K -shot learning and supervised learning methods when dealing with a small number of fine-grained annotations.

## introduction named entity recognition (ner) is a fundamental task in locating and categorizing named entities in unstructured texts. most research on ner has been conducted on coarse-grained datasets, including conll'03 @xcite , ace04 @xcite , ace05 @xcite , and ontonotes @xcite , each of which has less than 18 categories. as the applications of nlp broaden across diverse fields, there is increasing demand for finegrained ner that can provide more precise and † major in bio artificial intelligence detailed information extraction. nonetheless, detailed labeling required for large datasets in the context of fine-grained ner presents several significant challenges. it is more cost-intensive and time-consuming than coarse-grained ner. in addition, it requires a high degree of expertise because domain-specific ner tasks such as financial ner @xcite and biomedical ner @xcite require fine-grained labels. thus, finegrained ner tasks typically suffer from the data scarcity problem. few-shot ner approaches @xcite can be applied to conduct fine-grained ner with scarce fine-grained data. however, these methods do not exploit existing coarse-grained datasets that can be leveraged to improve finegrained ner because fine-grained entities are usually subtypes of coarse-grained entities. for instance, if a model knows what organization is, it could be easier for it to understand the concept of government or company. furthermore, these methods often experience early performance saturation, necessitating the training of a new supervised learning model if the annotation extends beyond several tens of labels. a pre-finetuning strategy @xcite was proposed to overcome the aforementioned problem. this strategy first learns the feature representations using a coarsegrained dataset before training the fine-grained model on a fine-grained dataset. in this method, coarse-grained data are solely utilized for representation learning; thus, it still does not explicitly utilize the relationships between the coarse-and fine-grained entities. our intuition for fully leveraging coarse-grained datasets comes mainly from the hierarchy between coarse-and fine-grained entity types. because a coarse-grained entity type typically comprises multiple fine-grained entity types, we can enhance lowresource fine-grained ner with abundant coarsegrained data. to jointly utilize both datasets, we devise a method to build a mapping matrix called f2c (short for 'fine-to-coarse'), which connects fine-grained entity types to their corresponding coarse-grained entity types and propose a novel approach to train a fine-grained ner model with both datasets. some coarse-grained entities improperly match a fine-grained entity type because datasets may be created by different annotators for different purposes. these mismatched entities can reduce the performance of the model during training. to mitigate this problem, coarse-grained entities that can degrade the performance of fine-grained ner must be eliminated. therefore, we introduce a filtering method called 'inconsistency filtering'. this approach is designed to identify and exclude any inconsistent coarse-to-fine entity mappings, ensuring a higher-quality training process, and ultimately, better model performance. the main contributions of our study are as follows: ## related work fine-grained ner. ner is a key task in information extraction and has been extensively studied. traditional ner datasets @xcite @xcite address coarse-grained entity types in the general domain. recently, domain-specific ner has been investigated in various fields @xcite @xcite . the domain-specific ner tasks typically employ fine-grained entity types. in addition, @xcite proposed a general domain fine-grained ner. n -way k-shot learning for ner. since labeling domain-specific data is an expensive process, few-shot ner has gained attention. most fewshot ner studies are conducted using an n -way k-shot episode learning @xcite @xcite . the objective of this approach is to train a model that can correctly classify new examples into one of n classes, using only k examples per class. to achieve this generalization performance, a large number of episodes need to be generated. furthermore, to sample these episodes, the training data must contain a significantly larger number of classes than n . in contrast, in our problem context, the number of fine-grained classes we aim to identify is substantially larger than the number of coarse-grained classes in the existing dataset. consequently, the episode-based few-shot learning approaches are unsuitable for addressing this problem. leveraging auxiliary data for information extraction. several studies have employed auxiliary data to overcome the data scarcity problem. jiang et al. (2021); oh et al. (2023) propose ner models trained with small, strongly labeled, and large weakly labeled data. jung and shim (2020) used strong and weak labels for relation extraction. however, in the previous studies, the main data and auxiliary data shared the same set of classes, which is not the case for fine-grained ner with coarse-grained labels. while the approaches of aghajanyan et al. (2021) and ma et al. (2022a) can be applied to our problem setting, they utilize auxiliary data for representation learning rather than explicitly utilizing the relationship between the two types of data. ## proposed method in this section, we introduce the notations and define the problem of fine-grained ner using coarsegrained data. then, we introduce the proposed cofiner model, including the creation of the f2c mapping matrix. ## conclusion we proposed cofiner, which explicitly leverages the hierarchical structure between coarse-and finegrained types to alleviate the low-resource problem of fine-grained ner. we devised the f2c mapping matrix that allows for fine-grained ner model training using additional coarse-grained datasets. however, because not all coarse-grained entities are beneficial for the fine-grained ner model, the proposed inconsistency filtering method is used to mask out noisy entities from being used in the model training process. we found through experiments that using a smaller amount of consistent data is better than using a larger amount of data without filtering, thus demonstrating the crucial role of inconsistency filtering. empirical results confirmed the superiority of cofiner over both the supervised and few-shot methods.