Interventional Training for Out-Of-Distribution Natural Language Understanding (2022)

Abstract: Out-of-distribution (OOD) settings are used to measure a model’s performance when the distribution of the test data is different from that of the training data. NLU models are known to suffer in OOD. We study this issue from the perspective of causality, which sees confounding bias as the reason for models to learn spurious correlations. While a common solution is to perform intervention, existing methods handle only known and single confounder, but in many NLU tasks the confounders can be both unknown and multifactorial. In this paper, we propose a novel interventional training method called Bottom-up Automatic Intervention (BAI) that performs multi-granular intervention with identified multifactorial confounders. Our experiments on three NLU tasks, namely, natural language inference, fact verification and paraphrase identification, show the effectiveness of BAI for tackling OOD settings.

## introduction from the era of word embeddings @xcite to pre-trained language models @xcite , researchers of natural language understanding (nlu) have tried to push the performance on benchmark datasets. traditional settings assume independent and identical distribution (iid) in training and testing splits. however, the iid setting cloaks the vulnerability of neural models, i.e., neural models tend to learn non-robust "shortcut" patterns in the training data but fail to make robust predictions on unseen samples. to evaluate the robustness of models, the out-of-distribution (ood) setting draws the attention of the nlu community. for example, the task of natural language inference (nli) determines whether a hypothesis can be entailed from a premise. we can observe that the lexical overlap between the hypothesis and the premise correlates with the entailment label on the benchmark mnli dataset @xcite (as shown in the top part of figure 1 ). @xcite proposed an ood set named hans for nli. as shown in the bottom part of figure 1 , hans does not have the correlation between lexical overlap and the entailment label. nli models that rely on the lexical overlap heuristic suffer from a significant degradation on hans @xcite . recently, causal inference has been adopted in nlp to identify robust correlations by analyzing reliable causal effects between variables (zhang @xcite . @xcite . from the perspective of causality @xcite , the crux under a model's vulnerability is confounding bias. we summarize the causal relations behind nlu tasks as a causal graph in figure 2 (a). x represents the input, e.g., a pair of sentences for nli, and y represents a label to be predicted. @xmath0, i.e., how to predict the label with reliable understanding of the input. x ← @xmath1. examples of c include nature bias in the dataset @xcite or crowdsourced workers preference @xcite . for instance, in nli, c may represent the degree of lexical overlap between the premise and the hypothesis, which is correlated with the entailment relation in the mnli dataset (see figure 1 ). 2 when crowdsourced workers are engaged to create hypotheses for nli, c could be the experience level of a worker, with inexperienced workers more likely to write simple sentences with straightforward meanings. as a result, these examples of c will make x and y spuriously correlated. a common solution of deconfounding is intervention @xcite , which aims to block the backdoor path (or spurious correlation) by cutting off @xmath2. the key idea is to stratify x into different environments @xcite , i.e., several subsets of train-ing data, according to the identified confounder. then the model is expected to make environmentagnostic prediction. by doing so, we are controlling x and thus break the backdoor path by d-separation @xcite . figure 2(c) depicts an example where the nli training data is stratified into several environments, e.g., one with obvious trend of lexical overlap bias and another does not. then the nli model is trained to fit both environments. however, the confounder c is not always observed. furthermore, confounders can be multifactorial in nlu, e.g., it may contain both inherent dataset bias and artifacts from crowdsourced workers. both scenarios make intervention non-trivial. in this paper, we propose bai, a bottom-up automatic intervention method, which can (1) identify the unobserved confounder(s) automatically, and (2) perform multi-granular intervention to handle multifactorial confounders. inspired by @xcite , the automatic stratifying mechanism is realized by maximizing the difference between data in different environments. 3 we further propose a novel bottom-up intervention mechanism that aims to address the multifactorial characteristic of c. while most existing debiasing work only considers a single bias, our bottom-up mechanism enables the model to pick up different confounders in two rounds of interventions. specifically, based on our preliminary experiments, we find that finegrained partition (i.e., partition with more environments) results in smaller differences between environments, making environment-agnostic learning easier. thus we start from a fine-grained partition. we then move on to a coarse-grained partition to further block the backdoor effect via c and make the learning environment-agnostic. we apply bai on three ood benchmarks for nlu tasks. the results show that our method outperforms state-of-the-art methods, e.g., achieving 7 percentage points of absolute gains from the previous best method under ood setting of quora question pairs (qqp) @xcite , a benchmark dataset for paraphrase identification. contributions: (1) we analyze the issue of nlu vulnerability from the perspective of causality analysis; (2) we propose a bottom-up automatic intervention method to perform intervention for unobserved and multifactorial confounders; and (3) extensive experiments on three ood benchmarks demonstrate that our method outperforms state-ofthe-art methods. ood settings have been studied in recent years in nlu. to tackle dataset bias, most existing work relies on instance reweighting with a bias model for debiasing. specifically, these methods @xcite first design a bias model and then train a target debiased model fused with the bias model. training instances predicted correctly by the bias model will be down-weighted in the training of the debiased model. early work mainly revolves around different fusion methods @xcite @xcite with known bias. then researchers started looking into unknown bias by designing the bias model with heuristics, e.g., a model trained with very small amount of data (utama et al., 2020b) or a model with only the bottom layers of the language model @xcite . however, instance reweighting based methods rely on either prior knowledge of bias or heuristic design of the bias model. furthermore, it is pointed out that such bias models may not be able to predict the main model's reaction of biased samples and reweighting may waste data @xcite . in contrast, our method is derived from causal inference @xcite , which is not related to any form of reweighting. meanwhile, our method does not adopt any bias model (which requires carefully design or prior knowledge of bias). causal intervention. causality inference @xcite measures the causal effect between variables and has been widely applied to various scenarios, e.g., social science (baron and kenny, 1986), medical science @xcite , and other applications @xcite @xcite . recently, causality inference is introduced to the machine learning community and intervention is one of the techniques in causality inference. intervention @xcite helps to eliminate the effect of confounders @xcite @xcite @xcite . invariant risk minimization (irm) @xcite implements intervention by learning a model invariant to different environments @xcite . although irm has been widely adopted in computer vision (cv) @xcite @xcite @xcite , to the best of our knowledge, our proposed bai is an initial work of applying irm in nlu. our work is distinguishable from previous work in two aspects. first, previous work in cv mainly focuses on image with annotated background as confounder while the confounder in nlu is more abstract and vague. second, our method is the first work considering multiple partitions for handling multifactorial confounders. ## conclusions in this paper, we explore how to improve the robustness of nlu models under ood setting, and propose a bottom-up automatic intervention method for debiasing. the experiment results demonstrate the superiority of our model over state-of-the-art methods. in future work, we will consider two improvements on bai. first, we target at an end-toend framework for intervention and dynamic learn the partition of environment for nlu tasks. second, we want to ease the trade-off effect between iid and ood sets.