Clear Up Confusion: Advancing Cross-Domain Few-Shot Relation Extraction through Relation-Aware Prompt Learning (2024)

Abstract: Cross-domain few-shot Relation Extraction (RE) aims to transfer knowledge from a source domain to a different target domain to address low-resource problems.Previous work utilized label descriptions and entity information to leverage the knowledge of the source domain.However, these models are prone to confusion when directly applying this knowledge to a target domain with entirely new types of relations, which becomes particularly pronounced when facing similar relations.In this work, we propose a relation-aware prompt learning method with pre-training.Specifically, we empower the model to clear confusion by decomposing various relation types through an innovative label prompt, while a context prompt is employed to capture differences in different scenarios, enabling the model to further discern confusion. Two pre-training tasks are designed to leverage the prompt knowledge and paradigm.Experiments show that our method outperforms previous sota methods, yielding significantly better results on cross-domain few-shot RE tasks.

## introduction relation extraction (re) is one of the key tasks of natural language processing (nlp), which aims to identify the relations between given entities. traditional supervised models @xcite @xcite have impressive performance in re tasks. however, collecting sufficient data for certain classes may be laborious in practice. inspired by the advances in few-shot learning @xcite , finetuning prompt-based pretrained language models have shown superior performance in few-shot re @xcite and some other tasks @xcite @xcite . however, these methods lack robustness in crossdomain scenarios, which is particularly important in low-resource re tasks. domain adaptation methods @xcite @xcite offer new insights by transferring knowledge between domains through shared feature representations extracted from multiple domains. however, these methods only work when the classes in the source and target domain have the same labels @xcite . to better learn the knowledge from the source domain, soares et al., 2019 build task-agnostic relation representations solely from the entity-linked text. zhang and lu, 2022 proposed a label descriptions prompt dropout approach to leverage the label information, which helps the model learn class representations effectively. nevertheless, as shown in figure 1 , simply relying on memorizing and understanding elements from the source domain does not assist the model in maintaining better performance in the target domain @xcite @xcite . therefore, the model needs the ability to figure out the connections and distinctions between relations to decompose various relation types. to bypass this issue, we propose a novel relationaware prompt learning approach with pre-training (clearre). specifically, we design a label prompt to assist the model in distinguishing similar relations, which tend to be confusing during predictions. we identify similar relations by introducing two filters: semantic filter and feature filter. meanwhile, a context prompt is proposed to further capture the differences among relations in different scenarios. we combine these two components into our ultimate prompt, effectively enhancing the model's ability in cross-domain scenarios by alleviating confusion. motivated by @xcite ; @xcite , we further design two pretraining tasks to facilitate the utilization of prompts. the first task is prompt-based mlm, which learns the prompt paradigm by predicting masked tokens in prompt sentences. the second task is relation contrastive discrimination in which we construct positive, negative, and hard negative samples and employ contrastive learning to enhance the ability of the model to differentiate between relations, especially for similar ones. we summarize the contributions as follows: ## methodology the overall framework of clearre is shown in figure 2 . section 2.1 briefly illustrates the task definition of re. section 2.2 shows the details of similar relations selection. section 2.3 explains the design of the prompt. section 2.4 provides a comprehensive interpretation of prompt mlm and relation contrastive discrimination pre-training tasks. training and inference are shown in appendix a 2.1 task definition x @xmath0, x 2 , ..., x m } is the original input sentence, including m tokens. entity position e @xmath1, e tail } refers to head and tail entities spans which warped with special tokens [e1], [/e1], [e2] and [/e2] following ernie (zhang et al., 2019). re tasks aim to learn a mapping function: f : (x ori , e key ) → y, where y is the label. ## case study to further verify the effectiveness of our method, we randomly sample 50 instances from the output and choose the most representative case in figure 4 . "win/defeat" and "opposite" are similar relations chosen by our similar filters. "win/defeat" means something that is physically or idealistically opposite, contrary, against or inverse of something else. "opposite" means someone or something who has won or lost a competition, an award, a war. both two sentences are related to a war scenario. the baseline model confuses these two similar relations while our method solves this tricky problem. this case suggests that clearre can distinguish confusing relations in an entirely new domain. ## conclusion in this paper, we propose a relation-aware contextual prompt learning approach with pre-training for cross-domain few-shot re. we design a novel prompt containing context information and confusing relations. two pre-training tasks further enable the model to adapt to the prompt format and learn to distinguish confusing relations under different conditions. extensive experiments and analyses demonstrate the effectiveness of our method.