UDAA : An Unsupervised Domain Adaptation Adversarial Learning Framework for Zero-Resource Cross-Domain Named Entity Recognition (2024)

Abstract: “The zero-resource cross-domain named entity recognition (NER) task aims to perform NER in aspecific domain where labeled data is unavailable. Existing methods primarily focus on transfer-ring NER knowledge from high-resource to zero-resource domains. However, the challenge liesin effectively transferring NER knowledge between domains due to the inherent differences inentity structures across domains. To tackle this challenge, we propose an Unsupervised DomainAdaptation Adversarial (UDAA) framework, which combines the masked language model auxil-iary task with the domain adaptive adversarial network to mitigate inter-domain differences andefficiently facilitate knowledge transfer. Experimental results on CBS, Twitter, and WNUT2016three datasets demonstrate the effectiveness of our framework. Notably, we achieved new state-of-the-art performance on the three datasets. Our code will be released.Introduction”

## introduction named entity recognition (ner) is a fundamental task in natural language processing. however, in real-world scenarios, obtaining the amount of domain-specific labeled data for the ner task is often challenging, as it can be expensive and time-consuming @xcite @xcite . hence, zero-resource cross-domain ner @xcite @xcite @xcite , which addresses the issue of data scarcity by adapting ner models to specific domains without available labeled data, draws more and more attention. based on the mainstream methods, recent researches have focused on the transfer of ner knowledge from high-resource to zero-resource domains. these methods typically involve training a source model on labeled data from the source domain to learn domain invariant features and apply the adapting ner models to the target domain. in the process of transferring knowledge to the target domain, the masked language model(mlm) is currently recognized as an unsupervised adaptive auxiliary task with good performance. adaptbert @xcite fine-tunes the mlm on unlabeled data to learn both task-special knowledge and domain-invariant knowledge. furthermore, tof @xcite addresses the zero-resource cross-domain task scenario through mlm capturing enough ner task knowledge. the acquired knowledge serves as a bridge to facilitate the transfer of ner knowledge. mlm plays a significant role in extracting context and recognizing entity boundaries. however, during knowledge transfer, especially when mapping between two domains, the transfer of knowledge from different tasks and the varying data distributions across domains can introduce noise and hinder model generalization performance. for instance: a) i'm watching a documentary about a shark on tv, and b) the stock market experienced a shark attack as prices plummeted. in the social media domain, the word shark represents sea animals. however, in the news domain, this is used metaphorically, referring to the stock market crash. the word "shark" shares the same affixes and root knowledge across different domains, yet it carries different semantic meanings in each domain. meanwhile, due to such domain source (news): germany's representative to the european union 's veterinary committee werner zwingmann. target (twitter): @redbullesports : smash shiba is stoked for #nwm7 melee grand finals . task non-ner : [germany] [european union] [werner zwingmann ] [smash shiba] ner : <germany, @xmath0, @xmath1, @xmath2, @xmath3 we propose the udaa framework, which combines the mlm auxiliary task with the daan to mitigate inter-domain differences. the models for tasks at different stages serve as bridges from high-resource domains to zero-resource domains and enable knowledge transfer. through extensive experimental evaluation using the cbs, twitter, and wnut2016 datasets, we demonstrate the effectiveness of udaa in the zero-resource cross-domain ner task. in the future, we plan to improve the optimization of pseudo-data by prompt learning and explore the task where label mismatch scenarios are involved.