Causal Intervention for Mitigating Name Bias in Machine Reading Comprehension (2023)

Abstract: Machine Reading Comprehension (MRC) is to answer questions based on a given passage, which has made great achievements using pre-trained Language Models (LMs). We study the robustness of MRC models to names which is flexible and repeatability. MRC models based on LMs may overuse the name information to make predictions, which causes the representation of names to be non-interchangeable, called name bias. In this paper, we propose a novel Causal Interventional paradigm for MRC (CI4MRC) to mitigate name bias. Specifically, we uncover that the pre-trained knowledge concerning names is indeed a confounder by analyzing the causalities among the pre-trained knowledge, context representation and answers based on a Structural Causal Model (SCM). We develop effective CI4MRC algorithmic implementations to constrain the confounder based on the neuron-wise and token-wise adjustments. Experiments demonstrate that our proposed CI4MRC effectively mitigates the name bias and achieves competitive performance on the original SQuAD. Moreover, our method is general to various pre-trained LMs and performs robustly on the adversarial datasets.

## introduction using pre-trained transformer-based language models (lms) has become the cornerstone of mrc @xcite @xcite , and the state-of-theart performance is achieved by fine-tuning the lm on various datasets @xcite @xcite . the lexical and syntactic knowledge encoded by lms, as well as factual knowledge, is a panacea for the model to learn mrc solutions effectively @xcite . however, the pre-trained knowledge (a) passage: @xcite ans majors in law and economics, while lindsey is earning a degree in classical piano performance. question: who is more likely to become a politician? example model prediction: hillary passage: @xcite ans majors in law and economics, while hillary is earning a degree in classical piano performance. correlates general facts (e.g., the politician) with specific entities (e.g., hillary), occasionally leading to name bias and other unintentional biases. in this work, we focus on the representations of given names in mrc obtained by pre-trained lms. previous work showed that the representations of named entities incorporate sentiment or gender @xcite , which is often transferable across entities via a shared name. also, @xcite found that, depending on the corpus, names tend to be grounded to spe-cific entities, even in generic contexts. however, recent works pursued stronger mrc performance and focused on stronger lms or some other technologies, such as curriculum learning @xcite and prompting @xcite . powerful lms ω are achieved by pre-training on their corresponding corpus sources c. we can use ω as a backbone and fine-tune the target model on the train set. it is arguably common sense that the stronger the pre-trained ω is, the better the mrc model will be. however, the fine-tuning stage only exploits the c's knowledge on what to transfer but neglects how to transfer. thus, this may not always be the case of consensus under adversarial attacks. as shown in figure 1 (b), we can see a paradox: though stronger ω achieves higher performance on squad name , it indeed degrades that on squad swap name . we found this may be due to some unintentional effects of pre-trained knowledge on named entities. to further explore the name bias, we show an example template in figure 1(a) , where the pre-trained knowledge of "hillary" misleads the prediction of ω. the name "hillary" is strongly associated with politicians in the corpus c, so the model neglects the context of the passage, leading to the over-reliance on name information to answer questions. we will explain these tests in detail in section 5. therefore, when the stronger ω is utilized in mrc, the stereotypical knowledge will be more robust than new knowledge in a single sample and the stereotypical name bias becomes misleading in adversarial cases. on this point, such a phenomenon is an easily overlooked shortage: some partial pre-trained knowledge is a confounder that limits robust performance for mrc models. however, the pre-trained ω encodes a large amount of knowledge about linguistics and the world, facilitating rapid adaptation to mrc. therefore, we aim to mitigate the biased effects of names without compromising the original context representation. in this paper, we propose a novel causal interventional paradigm for mrc (ci4mrc) to mitigate the effects of biased name representations. our method is based on the structural causal model (scm) for the causalities among the pre-trained knowledge, context representation, and answers. specifically, our contributions to this paper are summarized as follows: ## related work machine reading comprehension is a task to answer questions given a passage @xcite . in recent years, many influential works progressed the development of effective qa models @xcite @xcite . for example, bidaf @xcite employs an rnn-based sequential framework to encode questions and passages, while qanet (yu et al., 2018) employs convolution and self-attention. then, the pre-trained networks rapidly become the mainstream and result in models outperforming human-level performance in some datasets @xcite . however, accuracy in the i.i.d test cannot explain the paradoxical phenomenon in figure 1 . our work analyzes it from a causal view by showing that pretraining knowledge is a confounder. bias in pre-trained lms has been widely concerned. the performance of pre-trained models @xcite @xcite is remarkable, while recent work has shown that they capture biases from the corpus @xcite @xcite . the findings have promoted a growing amount of research to focus on mitigating these biases @xcite @xcite . the name bias in this work is focused on names with implicit stereotypical information. causal inference @xcite has been widely used in medicine, public policy, and epidemiology for many years @xcite . it not only is a framework for interpreting data, but also provides causal modeling tools and solutions to achieve intended goals by estimating causal effect @xcite . recently, causal inference has also attracted increasing attention in natural language processing to mitigate the dataset bias @xcite . we approach mrc from a causal perspective and offer a fundamental causal interventional mrc paradigm for mitigating name bias. ## causal intervention for mrc in this section, we will detail the proposed ci4mrc by providing practical implementations for g(x, k), p (y |x, k, m ), p (k) in eq. ( 2 ). in particular, we first apply the neuron-wise adjustment based on mixture-of-experts (moe) (bengio, 2013) to mitigate the name bias in the pre-trained lms. we find that this debiasing implementation does benefit from reducing the bias in the upstream representation, but it is a little toxic to the mrc performance, which also occurs in @xcite . therefore, we develop the token-wise adjustment to remedy the shortcoming and combine the two adjustments as the overall debiasing method. ## conclusion in this paper, we have presented ci4mrc, a novel causal interventional paradigm to address name bias in mrc: the pre-trained knowledge concerning names is a confounder limiting the robust performance. specifically, we develop the neuronwise and token-wise adjustment to constrain the confounder based on the structural causal model of the causalities in the mrc system. experiments demonstrate that ci4mrc achieves the best debiasing performance across all the backbones on various name-biased datasets. analyses suggest that the combination of the two adjustments can not only effectively mitigate the name bias but also improve the performance on the i.i.d evaluation. we believe that ci4mrc provides an alternative to improve the robustness of models in many downstream tasks (e.g., question answering). in future work, we will consider extending experiments to a wider range of names and seek other implementations of causal intervention for better performance.