Always the Best Fit: Adaptive Domain Gap Filling from Causal Perspective for Few-Shot Relation Extraction (2023)

Abstract: Cross-domain Relation Extraction aims to transfer knowledge from a source domain to a different target domain to address low-resource challenges. However, the semantic gap caused by data bias between domains is a major challenge, especially in few-shot scenarios. Previous work has mainly focused on transferring knowledge between domains through shared feature representations without analyzing the impact of each factor that may produce data bias based on the characteristics of each domain. This work takes a causal perspective and proposes a new framework CausalGF. By constructing a unified structural causal model, we estimating the causal effects of factors such as syntactic structure, label distribution,and entities on the outcome. CausalGF calculates the causal effects among the factors and adjusts them dynamically based on domain characteristics, enabling adaptive gap filling. Our experiments show that our approach better fills the domain gap, yielding significantly better results on the cross-domain few-shot relation extraction task.

## introduction relation extraction (re) is one of the key tasks of natural language processing (nlp), which aims to identify the relations between given entities. re models @xcite have impressive performance through large-scale supervised learning based on bert @xcite and lstm @xcite . however, collecting sufficient amounts of data for certain classes may be laborious in practice. although finetuning prompt-based pre-trained language models @xcite have shown superior performance in few-shot re tasks, they have encountered challenges in dealing with cross-domain problems. the main issue is that variables such as labels, syntactic structure, and entities having different distributions in each domain, resulting in data bias across different domains. from a causal perspective, the essence of this bias is that these variables have different causal effects on the results in different domains. domain adaptation methods @xcite offer new insights to tackle these issues by transferring knowledge between domains through shared feature representations extracted from multiple domains. however, these work relies excessively on extracting shared features and label distributions while ignoring the unique feature of each domain, which would lead to inferior results when domains have significant semantic gaps. therefore, @xcite proposed a label prompt dropout approach to eliminate the model's over-reliance on labels, but it is difficult to adequately capture the critical features of each domain by randomized dropout. to address this issue, our work focuses on identifying and adjusting the causal effects of variables by considering the distinct characteristics of different domains. we propose a novel framework causalgf and build a unified structural causal model (scm) @xcite to describe the cross-domain re task, as shown in figure 1 . the values and relationship in the graph can be altered by intervention and counterfactual generation to study the causal effect of various factors. furthermore, to adapt to the feature of different domains, it is necessary to estimate the causal effects of various factors and adjust them dynamically. causalgf implements causal operations and dynamically adjusts the causal effects of variables through prompt generation, dynamic weighting, and loss function improvement, enabling adaptive gap filling based on domain characteristics. we summarize the contributions as follows: ## methodology the overall framework of causalgf is shown in figure 2 . sections 2.1 and 2.2 describe the structural causal modeling and causal operation. 2.3 and 2.4 describe the implementation of the our method. ## conclusion in this paper, we propose causalgf, a novel framework based on a causal perspective. by building a unified structural causal model, causalgf estimates the causal effects of factors contributing to data bias and dynamically adjusts them to accommodate domain characteristics. our model effectively fills the domain gap, outperforming strong baselines in various cross-domain scenarios.