Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language (2022)

Abstract: Spurious correlations are a threat to the trustworthiness of natural language processing systems, motivating research into methods for identifying and eliminating them. However, addressing the problem of spurious correlations requires more clarity on what they are and how they arise in language data. Gardner et al (2021) argue that the compositional nature of language implies that all correlations between labels and individual “input features” are spurious. This paper analyzes this proposal in the context of a toy example, demonstrating three distinct conditions that can give rise to feature-label correlations in a simple PCFG. Linking the toy example to a structured causal model shows that (1) feature-label correlations can arise even when the label is invariant to interventions on the feature, and (2) feature-label correlations may be absent even when the label is sensitive to interventions on the feature. Because input features will be individually correlated with labels in all but very rare circumstances, domain knowledge must be applied to identify spurious correlations that pose genuine robustness threats.

## introduction spurious correlations are a growing source of concern in machine learning @xcite and related fields including natural language processing @xcite . while the intuition is fairly clearspurious correlations are features that are useful in the training data but unreliable in general -the notion is frequently referenced without a formal definition. @xcite propose a definition in terms of conditional probabilities: a feature x i is spuriously correlated with the label y unless p (y | x i ) is uniform. the definition can be generalized from uniformity to independence (x i ⊥ ⊥ y ) without affecting the claims of the paper. they go on to argue that "in a language understanding problem, . . . all simple correlations between input features and output labels are spurious" (emphasis in the original). the property that individual input features should be independent of labels -which i will call marginally uninformative input features (uif) 1 -is treated as an assumption about the nature of language processing and also as a desideratum that datasets should satisfy: if the label can be predicted from input features alone, then the dataset is in some sense too easy. 2 features may be marginally uninformative while still giving information about the label when viewed in combination. an especially provocative assertion of @xcite is that all correlations between labels and individual input features have the same status. in the sentence the pizza was amazing, suppose that both pizza and amazing are correlated with positive sentiment because the reviewers like pizza. there are at least two intuitive differences between these two correlations. first, while one can easily imagine a benighted subpopulation of reviewers who do not like pizza, it is not so easy to imagine reviewers who think that the word "amazing" carries negative sentiment. second, if we modify the subject (e.g., the movie was amazing), the label will usually be unaffected, but there are many perturbations to the adjective that flip the label (e.g., the pizza was greasy). this second intuition can be described using the framework of causality, which has generally treated spurious correlations as those that arise without a direct causal explanation @xcite . given a causal model of the data generating process, we can compute an interventional distribution p (y | do(x 1 := x), x 2 , x 3 ), which corresponds to the distribution over y in a data generating process in which the variable x 1 is surgically set to the value x @xcite @xcite . 3 when such interventions do not affect y for any given example, we say that y and x 1 are counterfactually invariant @xcite . violations of uif are particularly troubling when they are accompanied by counterfactual invariance, because non-causal correlations often do not transfer to other domains @xcite . ## toy example consider a simplified targeted sentiment analysis task @xcite , in which the sentiment is y , the target is z, and the sentences are all of the form (x 1 , x 2 , x 3 ), with x 1 specifying a target noun phrase, x 2 a copula-like expression, and x 3 a predicative adjectival phrase. for example, @xmath0, @xmath1, x 1 = the pizza, x 2 = turned out to be, x 3 = crispy and delicious. we will treat this data as generated from the causal model shown in fig. 2 . this causal model can be summarized by two assertions: (1) the target z is a direct effect of only the span x 1 ; (2) the sentiment label y is a direct effect of only the spans x 2 and x 3 . the function f x can represent any generative model of text: an n-gram model, a grammar-based formalism, a deep autoregressive network, etc. aside on the direction of causation. we treat the text as the cause of the labels, rather than the converse. this distinction is somewhat vexed @xcite . in some cases the direction of causation is clear from the task (e.g., table-to-text generation, summarization, and translation), but often the problem could be framed in either direction: perhaps the writer had the label in mind when producing the text, and thus the text is an effect of the label; or perhaps it is better to think of the annotator, who must read the text to arrive at the label, regardless of the writer's intentions. when the labels cause the text, the notion of counterfactual invariance can be restated in terms of the invariance of text features to perturbations on labels, e.g. p (x 1 | do(y := y), z). as the toy example is meant to serve only an expository purpose, we leave elaboration of the relationship of uif to such models for future work. ## conclusions in the toy example, violations of uif arise from three distinct phenomena: confounding between the sentiment and the target (α = 0, leading to x 1 ⊥ ⊥ y ); confounding between the sentiment and the use of negation (β + = β -, leading to x 2 ⊥ ⊥ y ); and lack of a perfect balance in the probability of negation between positive-and negativesentiment examples (β + = 1β -, leading to x 3 ⊥ ⊥ y ). the conditions required to satisfy uif are thus progressively less plausible as we move from x 1 to x 3 , and full uif is achieved only in the perfectly balanced case of α = 0, β + = β -= 1 2 . the number of such constraints will increase with the size of the grammar, making uif vanishingly rare in more general settings. this conclusion follows from the pcfg analysis and is derived without reference to causality. the toy example also demonstrates the disconnect between the uif view of spurious correlations and the causal view: counterfactual invariance does not imply uif because x 1 can be marginally informative of y even when x 1 and y are counterfactually invariant (these are the artifacts that we want to remove); uif does not imply counterfactual invariance because both x 2 and x 3 can be uninformative of y even when y is sensitive to interventions on both features. from a theoretical perspective, it is unsurprising that these two views diverge, because uif is a purely observational criterion while counterfactual invariance requires an explicit causal model. indeed, this relationship is discussed in depth by @xcite 3), albeit outside the context of language. the two perspectives can be seen as complementary, in that violation of uif is a necessary but insufficient condition for a spurious correlation in the causal sense. moving beyond toy examples, it is unlikely that we can construct fully-specified causal models of language that supply useful invariances while handling every possible fluent utterance. how then can we use causal insights to design better benchmarks and more robust language understanding systems? in some cases it is possible to elaborate partial causal models of a task, with associated invariance properties: for example, the sentiment of a movie review should be invariant to (though not independent of) the identities of the actors in the movie. several existing approaches can be viewed as instantiations of partial causal models: for example, data augmentation, causally-motivated regularizers, stress tests, and "worst-subgroup" performance metrics (and associated robust optimizers) can be seen as enforcing or testing task-specific invariance properties that provide robustness against known distributional shifts (e.g., @xcite @xcite @xcite . such approaches generally require domain knowledge about the linguistic and causal properties of the task at hand -or to put it more positively, they make it possible for such domain knowledge to be brought to bear. indeed, the central argument of this paper is that no meaningful definition of spuriousness or robustness can be obtained without such domain knowledge. a final observation, pertaining to both uif and counterfactual invariance, is the parallel treatment of x 2 (the copula) and x 3 (the adjectival phrase). from a lexical semantic perspective, only x 3 is directly associated with sentiment, while x 2 plays a functional role by potentially reversing x 3 . it may therefore seem undesirable to learn a correlation between x 2 and y , and preferable to attach that relationship exclusively to x 3 . indeed, one of the main catalysts of interest in spurious correlations in natural language processing was the observation that the presence of syntactic negation is a strong predictor of contradiction label in the natural language inference task, which should require reasoning about pairs of sentences @xcite . yet neither uif nor counterfactual invariance is capable of making any distinction between x 2 and x 3 in this model. while it is possible to enforce uninformativeness on x 2 heuristically, e.g. by sampling or augmenting the data to ensure β + = β -, those same heuris-tics could be applied to enforce uninformativeness on x 3 by making β + = 1β -. singling out x 2 requires additional justification. such a principle might be found in the multitask setting, in which we prefer feature-label informativeness to be sparse, with each feature directly informing only a few labels.