Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP (2021)

Abstract: The principle of independent causal mechanisms (ICM) states that generative processes of real world data consist of independent modules which do not influence or inform each other. While this idea has led to fruitful developments in the field of causal inference, it is not widely-known in the NLP community. In this work, we argue that the causal direction of the data collection process bears nontrivial implications that can explain a number of published NLP findings, such as differences in semi-supervised learning (SSL) and domain adaptation (DA) performance across different settings. We categorize common NLP tasks according to their causal direction and empirically assay the validity of the ICM principle for text data using minimum description length. We conduct an extensive meta-analysis of over 100 published SSL and 30 DA studies, and find that the results are consistent with our expectations based on causal insights. This work presents the first attempt to analyze the ICM principle in NLP, and provides constructive suggestions for future modeling choices.

## introduction nlp practitioners typically do not pay great attention to the causal direction of the data collection process. as a motivating example, consider the case of collecting a dataset to train a machine translation (mt) model to translate from english (en) to spanish (es): it is common practice to mix all available en-es sentence pairs together and train the model on the entire pooled data set @xcite . however, such mixed corpora actually consist of two distinct types of data: (i) sentences that originated in english and have been translated (by human translators) into spanish (@xmath0; and (ii) sentences that originated in given the english sentence above, can you write its spanish translation? prompt for annotators figure 1 : annotation process for nlp data: the random variable that exists first is typically the cause (e.g., a given prompt), and the one generated afterwards is typically the effect (e.g., the annotated answer). spanish and have subsequently been translated into english (@xmath1. 2 intuitively, these two subsets are qualitatively different, and an increasing number of observations by the nlp community indeed suggests that they exhibit different properties @xcite @xcite . in the case of mt, for example, researchers find that training models on each of these two types of data separately leads to different test performance, as well as different performance improvement by semi-supervised learning (ssl) @xcite @xcite . motivated by this observation that the data collection process seems to matter for model performance, in this work, we provide an explanation of this phenomenon from the perspective of causality @xcite . first, we introduce the notion of the causal direction for a given nlp task, see fig. 1 for an example. throughout, we denote the input of a learning task by x and the output which is to be predicted by y . if, during the data collection process, x is generated first, and then y is collected based on x (e.g., through annotation), we say that x causes y , and denote this by @xmath2. if, on the other hand, y is based on whether the direction of prediction aligns with the direction of causation or not, we distinguish two types of tasks: (i) causal learning, i.e., predicting the effect from the cause; and (ii) anticausal learning, i.e., predicting the cause from the effect. generated first, and then x is collected based on y , we say that y causes x (@xmath3. 3 based on whether the direction of prediction aligns with the causal direction of the data collection process or not, @xcite categorize these types of tasks as causal learning (@xmath4, or anticausal learning (@xmath5, respectively; see fig. 2 for an illustration. in the context of our motivating mt example this means that, if the goal is to translate from english (@xmath6, training only on subset (i) of the data consisting of @xmath7, whereas training only on subset (ii) consisting of @xmath8. based on the principle of independent causal mechanisms (icm) @xcite , it has been hypothesized that the causal direction of data collection (i.e., whether a given nlp learning task can be classified as causal or anticausal) has implications for the effectiveness of commonly used techniques such as ssl and domain adaptation (da) @xcite . we will argue that this can explain performance differences reported by the nlp community across different data collection processes and tasks. in particular, we make the following contributions: ## categorization of common nlp tasks into causal and anticausal learning we start by categorizing common nlp tasks which use an input variable x to predict a target or output variable y into causal learning (@xmath9, anticausal learning (@xmath10, and other tasks that do not have a clear underlying causal direction, or which typically rely on mixed (causal and anticausal) types of data, as summarised in tab. 1. key to this categorization is determining whether the input x corresponds to the cause or the effect in the data collection process. as illustrated in fig. 1 , if the input x and output y are generated at two different time steps, then the variable that is generated first is typically the cause, and the other that is subsequently generated is typically the effect, provided it is generated based on the previous one (rather than, say, on a common confounder that causes both variables). if x and y are generated jointly, then we need to distinguish based on the underlying generative process whether one of the two variables is causing the other variable. learning effect from cause (causal learning) causal (@xmath11.e., the target y is the effect) from a given input x (the cause). examples include: summarization (@xmath12; parsing and tagging (@xmath13; data-totext generation (@xmath14; and information extraction (@xmath15. learning cause from effect (anticausal learning) anticausal (@xmath16. typical anticausal nlp learning problems include, for example, author attribute identification (author @xmath17; and review sentiment classification (@xmath18. other/mixed some tasks can be categorized as either causal or anticausal, depending on how exactly the data is collected. in § 1, we discussed the example of mt where different types of (causal and anticausal) data are typically mixed. another example is the task of intent classification: if the same author reveals their intent before the writing (i.e., @xmath19, it can be viewed as an anticausal learning task; if, on the other hand, the data is annotated by other people who are not the original author (i.e., @xmath20, it can be viewed as a causal learning task. a similar reasoning applies to question answering and generation tasks which respectively aim to provide an answer to a given question, or vice versa: if first a piece of informative text is selected and annotators are then asked to come up with a corresponding question (@xmath21, e.g., in the squad dataset @xcite , then question answering is an anticausal and question generation a causal learning task; if, on the other hand, a question such as a search query is selected first and subsequently an answer is pro-vided (@xmath22, e.g., in the natural questions dataset @xcite , then question answering is a causal and question generation an anticausal learning task. often, multiple such datasets are combined without regard for their causal direction. ## implications of icm for causal and anticausal learning problems whether we are in a causal or anticausal learning scenario has important implications for semisupervised learning (ssl) and domain adaptation (da) @xcite @xcite @xcite @xcite , which are techniques also commonly used in nlp. these implications are derived from the principle of independent causal mechanisms (icm) @xcite which states that "the causal generative process of a system's variables is composed of autonomous modules that do not inform or influence each other" @xcite . in the bivariate case, this amount to a type of independence assumption between the distribution p c of the cause c, and the causal process, or mechanism, p e|c that generates the effect from the cause. for example, for a question answering task, the generative process p c by which one person comes up with a question c is "independent" of the process p e|c by which another person produces an answer e for question c. 4 here, "independent" is not meant in the sense of statistical independence of random variables, but rather as independence at the level of generative processes or distributions in the sense that p c and p e|c do not share information (the person asking the question and the one answering may not know each other) and can be manipulated independently of each other (we can swap either of the two for another participant without the other one being influenced by this). crucially, this type of independence is generally violated in the opposite, i.e., anticausal, direction: p e and p c|e may share information and change dependently @xcite . this has two important implications for common learning tasks @xcite which are illustrated in fig. 3 . another domain s c causal mechanism p e|c e ⫫ share no @xmath23 figure 3 : the icm principle assumes that the generative process p c of the cause c is independent of the causal mechanism p e|c : the two distributions share no information and each may be changed or manipulated without affecting the other. in the anticausal direction, on the other hand, the effect distribution p e is (in the generic case) not independent of the inverse mechanism p c|e : they may share information and change dependently. (left) ssl, which aims to improve an estimate of the target conditional p y |x given additional unlabelled input data from p x , should therefore not help for causal learning (@xmath24, but may help in the anticausal direction (@xmath25. (right) da, which aims to adapt a model of p y |x from a source domain to a target domain (e.g., fine-tuning on a smaller dataset), should work better for causal learning settings where a change in p c is not expected to lead to a change in the mechanism p e|c , whereas in the anticausal direction p e and p c|e may change in a dependent manner. implications of icm for ssl first, if p c shares no information with p e|c , ssl-where one has additional unlabelled input data from p x and aims to improve an estimate of the target conditional p y |x -should not work in the causal direction (@xmath26, but may work in the anticausal direction (@xmath27, as p e and p c|e may share information. causal nlp tasks should thus be less likely to show improvements over a supervised baseline when using ssl than anticausal tasks. in other words, the shortest description of the joint distribution p c,e corresponds to describing p c and p e|c separately (i.e., they share no information), whereas there may be redundant (shared) information in the non-causal direction such that a separate description of p e and p c|e will generally be longer than that of the joint distribution p c,e . ## ssl for causal vs. anticausal models in semi-supervised learning (ssl), we are given a typically-small set of k labeled observations d @xmath28, y 1 ), . . . , (x k , y k )}, and a typicallylarge set of m unlabeled observations of the input as explained in § 3, ssl should only work for anticausal (or confounded) learning tasks, according to the icm principle. @xcite have observed this trend on a number of classification and regression tasks on small-scale numerical inputs, such as predicting boston housing prices from quantifiable neighborhood features (causal learning), or breast cancer from lab statistics (anticausal learning). however, there exist no studies investigating the implications of icm for ssl on nlp data, which is of a more complex nature due to the high dimensionality of the input and output spaces, as well as potentially large confounding. in the following, we use a sequence-to-sequence decipherment experiment ( § 5.1) and a meta-study of existing literature ( § 5.2) to showcase that the same phenomenon also occurs in nlp. ## da for causal vs. anticausal models we also consider a supervised domain adaptation (da) setting in which the goal is to adapt a model trained on a large labeled data set from a source domain, to a potentially different target domain from which we only have a a small labeled data set. as explained in § 3, da should only work well for causal learning, but not necessarily for anticausal learning, according to the icm principle. similar to the meta-study on ssl, we also review existing nlp literature on da. we focus on da improvement, i.e., the performance gain of using da over an unadapted baseline that only learns from the source data and is tested on the target domain. since the number of studies on da that we can find is smaller than for ssl, we cover 22 instances of da on causal tasks, and 11 instances of da on anticausal tasks. the results are summarised in tab. 6. we find that the observations again echo with our expectations (according to icm) that da should work better for causal, than for anticausal learning tasks. again, we use welch's t-test @xcite to verify that the da improvements of causal learning and anticausal learning are statistically different, and obtain a p-value of 0.023. ## how to use the findings in this study data collection practice in nlp due to the different implications of causal and anticausal learning tasks, we strongly suggest annotating the causal direction when collecting new nlp data. one way to do this is to only collect data from one causal direction and to mention this in the meta information. for example, summarization data collected from the tl;dr of scientific papers scitldr @xcite should be causal, as the tl;dr summaries on openreview (some from authors when submitting the paper, others derived from the beginning of peer reviews) were likely composed after the original papers or reviews were written. alternatively, one may allow mixed corpora, but label the causal direction for each (x, y) pair, e.g., which is the original vs. translated text in a translation pair. since more data often leads to better model performance, it is common to mix data from both causal directions, e.g., training on both @xmath29. annotating the causal direction for each pair allows future users of the dataset to potentially handle the causal and anticausal parts of the data differently. causality-aware modeling when building nlp models, the causal direction provides additional information that can potentially be built into the model. in the mt case, since causal and anticausal learning can lead to different performance @xcite , one way to take advantage of the known causal direction is to add a prefix such as "[modeling-effect-to-cause]" to the original input, so that the model can learn from causally-annotated input-output pairs. for example, @xcite use labels of the causal direction to elicit different behavior at inference time. another option is to carefully design a combination of different modeling techniques, such as limiting self-training (a method for ssl) only to the anticausal direction and allowing back-translation in both directions, as preliminarily explored by @xcite . causal discovery suppose that we are given measurements of two types of nlp data x and y (e.g., text, parse tree, intent type) whose collection process is unknown, i.e., which is the cause and which the effect. one key finding of our study is that there is typically a causal footprint of the data collection process which manifests itself, e.g., when computing the description length in different directions ( § 4) or when performing ssl ( § 5) or da ( § 6). based on which direction has the shorter mdl, or allows better ssl or da, we can thus infer one causal direction over the other. prediction of ssl and da effectiveness being able to predict the effectiveness of ssl or da for a given nlp task can be very useful, e.g., to set the weights in an ensemble of different models @xcite . while predicting ssl performance has previously been studied from a non-causal perspective @xcite , our findings suggest that a simple qualitative description of the data collection process in terms of its causal direction (as summarised for the most common nlp tasks in tab. 1) can also be surprisingly effective to evaluate whether ssl or da should be expected to work well. ## limitations and future work we note that icm-when taken strictly-is an idealized assumption that may be violated and thus may not hold exactly for a given real-world data set, e.g., due to confounding, i.e., when both variables are influenced by a third, unobserved variable. in this case, one may observe less of a difference between causal and anticausal learning tasks. we also note that, while we have made an effort to classify different nlp tasks as typically causal or anticausal, our categorization should not be ap-plied blindly without regard for the specific generative process at hand: deviations are possible as explained in the mixed/other category. another limitation is that the ssl and da settings considered in this paper are only a subset of the various settings that exist in nlp. our study does not cover, for example, ssl that uses additional output data (e.g., jean et al. (2015); gülçehre et al. (2015); sennrich and zhang (2019) ), or unsupervised da (as reviewed by @xcite ). in addition, in our meta-study of published ssl and da findings, the improvements of causal vs. anticausal learning might be amplified by the scale of research efforts on different tasks and potentially suffer from selection bias. finally, we remark that, in the present work, we have focused on bivariate prediction tasks with an input x and output y . future work may also apply icm-based reasoning to more complex nlp settings, for example, by (i) incorporating additional (sequential/temporal) structure of the data (e.g., for mt or language modeling) or (ii) considering settings in which the input x consists of both cause x cau and effect x eff features of the target y (von @xcite . ## related work nlp and causality existing work on nlp and causality mainly focuses on the extracting text features for causal inference. researchers first propose a causal graph based on domain knowledge, and then use text features to represent some elements in the causal graph, e.g., the cause @xcite , effect @xcite , and confounders @xcite @xcite . another line of work mines causal relations among events from textual expressions, and uses them to perform relation extraction @xcite @xcite , question answering @xcite , or commonsense reasoning @xcite . for a recent survey, we refer to @xcite . usage of mdl in nlp although mdl has been used for causal discovery for low-dimensional data @xcite @xcite , only very few studies adopt mdl on high-dimensional nlp data. most existing uses of mdl on nlp are for probing and interpretability: e.g., @xcite use it for probing of a small bayesian model and network pruning, based on the method proposed by @xcite to calculate mdl for deep learning. we are not aware of existing work using mdl for causal discovery, or to verify causal concepts such as icm in the context of nlp. existing discussions on ssl and da in nlp ssl and da has long been used in nlp, as reviewed by @xcite and @xcite . however, there have been a number of studies that report negative results for ssl @xcite @xcite @xcite and da @xcite . our works constitutes the first explanation of the ineffectiveness of ssl and da on certain nlp tasks from the perspective of causal and anticausal learning. ## conclusion this work presents the first effort to use causal concepts such as the icm principle and the distinction between causal and anticausal learning to shed light on some commonly observed trends in nlp. specifically, we provide an explanation of observed differences in ssl (tabs. 4 and 5) and da (tab. 6) performance on a number of nlp tasks: da tends to work better for causal learning tasks, whereas ssl typically only works for anticausal learning tasks, as predicted by the icm principle. these insights, together with our categorization of common nlp tasks (tab. 1) into causal and anticausal learning, may prove useful for future nlp efforts. moreover, we empirically confirm using mdl that the description of data is typically shorter in the causal than in the anticausal direction (tab. 3), suggesting that a causal footprint can also be observed for text data. this has interesting potential implications for discovering causal relations between different types of nlp data.