Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios (2022)

Abstract: Domain adaptation is an effective solution to data scarcity in low-resource scenarios. However, when applied to token-level tasks such as bioNER, domain adaptation methods often suffer from the challenging linguistic characteristics that clinical narratives possess, which leads to unsatsifactory performance. In this paper, we present a simple yet effective hardness-guided domain adaptation framework for bioNER tasks that can effectively leverage the domain hardness information to improve the adaptability of the learnt model in the low-resource scenarios. Experimental results on biomedical datasets show that our model can achieve significant performance improvement over the recently published state-of-the-art (SOTA) MetaNER model.

## introduction named entity recognition (ner) is a fundamental nlp task which aims to locate named entity (ne) mentions and classify them into predefined categories such as location, organization, or person. ner usually serves as an important first sub-task for information retrieval @xcite , task oriented dialogues @xcite and other language applications. consequently, ner has seen significant performance improvements with the recent advances of pre-trained language models (plms) @xcite . unfortunately, a large amount of training data is often essential for these plms to excel and except for a few high-resource domains, the majority of domains have limited amount of labeled data. this data-scarcity problem amplifies given the context of biomedical ner (bioner). firstly, the annotation process for the biomedical domains is time-consuming and can be extremely expensive. thus, many biomedical domain corpora, especially for those privately developed, are often scarcely * corresponding author labeled. furthermore, each biomedical domain can have distinct linguistic characteristics which are non-overlapping with those found in other biomedical domains @xcite . this linguistic challenge often diminishes the robustness of plms transferred from high-resource biomedical domains to low-resource ones @xcite . given the premise, this paper focuses on adapting plms for bioner tasks learned from highresource biomedical domains to low-resource ones. a potential solution is to inject the prior "experience" to this adaptation process. few works have explored this area such as @xcite and @xcite , where the former followed the optimization/meta-learning strategy by @xcite and the latter introduced a feature critic module similar to the work of @xcite . we show that simply incorporating the hardness information that each domain contributes to the learning of the bioner lms could significantly boost the adaptation performance of existing learning paradigms under various low-resource settings. this happens since the importance/hardness of biomedical domains can vary significantly, as shown in ## related works few works have addressed domain adaptation for ner. both li et al. (2020a) and li et al. (2020b) seek a robust representation for the sequence labeling function bilstm-crf using the meta-learning framework @xcite , with the latter further includes an auxiliary network to promote adversarial learning. @xcite consider label dependencies via an auto-regressive framework built on top of bi-lstm for cross-domain ner. however, different domains can have different level of hardness which both works have not yet addressed. existing domain adaptation techniques using meta learning framework to incorporate hardness information via 1) actively ranking the tasks in term of difficulty level @xcite @xcite ; 2) designing an adaptive task scheduler @xcite ; or 3) relying on generative approaches to quantify the uncertainties of tasks @xcite . to our knowledge, we are the first to perform hardness guided domain adaptation for bioner tasks. ## conclusion we have proposed simple yet effective methods that effectively leverage the domain hardness information to improve the effectiveness of the learnt model under the low-resource ner settings. experiments on biomedical corpora have shown that the sequence labelling function derived from our hgdas have achieved substantial performance improvements compared to current sota baselines.