D o SEA : A Domain-specific Entity-aware Framework for Cross-Domain Named Entity Recogition (2022)

Abstract: Cross-domain named entity recognition aims to improve performance in a target domain with shared knowledge from a well-studied source domain. The previous sequence-labeling based method focuses on promoting model parameter sharing among domains. However, such a paradigm essentially ignores the domain-specific information and suffers from entity type conflicts. To address these issues, we propose a novel machine reading comprehension based framework, named DoSEA, which can identify domain-specific semantic differences and mitigate the subtype conflicts between domains. Concretely, we introduce an entity existence discrimination task and an entity-aware training setting, to recognize inconsistent entity annotations in the source domain and bring additional reference to better share information across domains. Experiments on six datasets prove the effectiveness of our DoSEA. Our source code can be obtained from https://github.com/mhtang1995/DoSEA .

## introduction named entity recognition(ner) is a fundamental task in natural language processing and has been extensively studied in various domains. however, acquiring a high performance ner model heavily relies on labor-intensive annotated data @xcite . thus, there is a growing interest in cross-domain ner, which aims to exploit the information on a well-studied source domain to improve the performance in a target domain @xcite . @xcite , we focus on the supervised cross-domain ner setting, which utilizes abundant annotated samples from the source domain and small annotated samples from the target domain. previous studies @xcite @xcite typically treat cross-domain ner task as a sequence labeling problem, classifying each word as a type of entity. however, cross-domain ner is challenging due to the entity type difference between domains, since the target domain contains specific entity types. as figure 1 shows, crossner politics dataset @xcite ) contains specific entity types(e.g., event, politician, country and political party), which are not labeled in the @xcite . thus, the sequence-labeling based method commonly adopts separate model structures(crf or softmax layer) for each domain, which primarily limits the parameter sharing across domains. a series of fine-tune methods @xcite and multi-task learning methods @xcite have been proposed for promoting parameter sharing. the fine-tune method first trains a model using source domain samples, then fine-tunes the model using target domain samples with an initialized label decoder. however, it depends on the sizes of target domain samples to learn a strong label decoder. conversely, the multi-task learning method simultaneously trains a model for both domains under the jointly training strategy, and it essentially adds auxiliary tasks to facilitate parameter sharing. specifically, @xcite adds kl-divergence in the features of identical entity types between each domain's crf layer. @xcite models entity type features as separate cell states in a multi-cell compositional lstm structure, then shares the same entity type's features across domains. however, they ignore the domain-specific information and suffer from entity type conflicts. for example, conll-2003 and crossner politics both contain "location" entities, but the latter requires a distinction between "location" and "country", which is not considered in the former. in this case, previous work @xcite indicates the subtype conflicts, since the cross-domain ner model may tend to classify "country" as "location" entities. to this end, we propose a novel framework named domain-specific entity-aware(dosea) network for cross-domain ner, which aims to mitigate the negative impacts raised by the domainspecific entity types. specifically, instead of assigning a separate model structure for each domain, our dosea formalizes ner as a machine reading comprehension (mrc) task @xcite , which can naturally combine domain-related questions with annotated samples, to share all model parameters across domains. moreover, we propose an entity existence discrimination (eed) task and an entityaware training setting to recognize inconsistent entity annotations and handle the subtype conflicts. the eed task is designed to determine whether each type of entity exists in a sentence. as for the entity-aware training setting, we utilize the eed task to aware the presence of inconsistent annotated entities in source domain samples, then transform these entities to the target domain-specific entities by leveraging the explicit hierarchical relationship. the above procedures not only alleviate the subtype conflicts but also bring additional reference to better share information across domains. the main contributions of this paper are summarized as follows: cross-domain ner. multi-task learning methods @xcite @xcite have been popular in cross-domain ner, which is used to add the auxiliary task to improve performance. @xcite jointly trains the ner and lm tasks in a parameter generator network. jia and zhang (2020) proposed a multi-cell compositional lstm structure for cross-domain ner, which models each entity type as a separate cell state. fine-tune methods @xcite @xcite also show strong performance, which pre-trained a model in a source domain and then fine-tune the model in a target domain. some works try to achieve accurate transfer learning for cross-domain ner @xcite @xcite . @xcite classifies source domain samples by similarity metrics and assigns different weights for training. @xcite proposes a data augmentation approach to transform the data representation across domains. a few works consider the relationship between entity types @xcite . @xcite considers the mentioned relationship between the source and target entity types, such as "professor" and "student". compared with them, our proposed dosea is built on the mrc model, which aims to handle the issues raised by the specific entity types from the target domain. ## methodology figure 2 shows that dosea has three components, including a context encoder, a multi-task layer and an entity-aware training setting. the multi-task layer contains entity existence discrimination(eed) task and entity span prediction(esp) task. in the entity-aware training setting, we adopt different training processes for source and target domains. ## conclusion in this paper, we propose a novel framework named domain-specific entity-aware(dosea) for crossdomain ner and focus on the issues raised by the domain-specific entity types. our framework is built on the mrc-ner task, which shares all model parameters across domains. then, we introduce an entity existence discrimination task and an entity-aware training setting to alleviate the subtypes conflicts, which learns to transform the entities with inconsistent annotations into target domain-specific entities in source domain samples. experiments show that dosea achieves new state-of-the-art performance over six cross-domain benchmarks under the jointly training strategy.