In attempts to produce machine learning models less reliant on spurious patterns in training data , researchers have recently proposed  a human-in-the-loop process for generating counterfactually augmented datasets. As applied in NLP, given some documents and their (initial) labels, humans are tasked with revising the text to make a (given) counterfactual label applicable. Importantly, the instructions prohibit edits that are not necessary to flip the applicable label  . Models trained on the augmented (original and revised) data have been shown to rely less on semantically irrelevant words and to generalize better out-of-domain . While this work draws  on causal thinking, casting edits as interventions and relying on human understanding to assess outcomes, the underlying causal model is not clear nor are the principles underlying the observed improvements in out-of-domain evaluation . In this paper, we explore a toy analog , using linear Gaussian models . Our analysis reveals interesting relationships between causal models, measurement noise, out-of-domain generalization, and reliance on spurious signals. Interestingly our analysis suggests that data corrupted by adding noise to causal features will degrade out-of-domain performance, while noise added to non-causal features may make models more robust out-of-domain . This analysis yields interesting insights that help to explain the efficacy of counterfactually augmented data. Finally, we present a large-scale empirical study that supports this hypothesis .
In attempts to produce ML models less reliant on spurious patterns in NLP datasets , researchers have recently proposed curating counterfactually augmented data (CAD) via a human-in-the-loop process in which given some documents and their (initial) labels, humans must revise the text to make a counterfactual label applicable. Importantly, edits that are not necessary to flip the applicable label are prohibited . Models trained on the augmented data appear, empirically, to rely less on semantically irrelevant words and to generalize better out of domain . While this work draws loosely on causal thinking, the underlying causal model (even at an abstract level) and the principles underlying the observed out-of-domain improvements remain unclear . In this paper, we introduce a toy analog based on linear Gaussian models , observing interesting relationships between causal models, measurement noise, out-of-domain generalization, and reliance on spurious signals. Our analysis provides some insights that help to explain the efficacy of CAD. Moreover, we develop the hypothesis that while adding noise to causal features should degrade both in-domain and out-of-domain performance, adding noise to non-causal features should lead to relative improvements in out-of-domain performance. This idea inspires a speculative test for determining whether a feature attribution technique has identified the causal spans. If adding noise (e.g., by random word flips) to the highlighted spans degrades both in-domain and out-of-domain performance on a battery of challenge datasets, but adding noise to the complement gives improvements out-of-domain, it suggests we have identified causal spans. We present a large-scale empirical study comparing spans edited to create CAD to those selected by attention and saliency maps. Across numerous domains and models, we find that the hypothesized phenomenon is pronounced for CAD .
R
machine learning
ML
R
training data
NLP datasets
A
curating counterfactually augmented data (CAD) via
R
for generating counterfactually augmented datasets. As applied in NLP,
in which
R
are tasked with revising
must revise
D
(given)
D
the instructions prohibit
A
are prohibited
R
(original and revised) data have been shown
data appear, empirically,
R
out-of-domain
out of domain
A
loosely
D
casting edits as interventions and relying on human understanding to assess outcomes,
R
is not clear nor are
(even at an abstract level) and
D
improvements in
R
evaluation
improvements remain unclear
R
explore
introduce
R
, using
based on
R
. Our analysis reveals
, observing
R
Interestingly our analysis suggests that data corrupted by
Our analysis provides some insights that help to explain the efficacy of CAD. Moreover, we develop the hypothesis that while
R
will degrade
should degrade both in-domain and
R
while noise added
adding noise
R
may make models more robust
should lead to relative improvements in
R
. This analysis yields interesting insights that help to explain the efficacy of counterfactually augmented data. Finally, we
performance. This idea inspires a speculative test for determining whether a feature attribution technique has identified the causal spans. If adding noise (e.g., by random word flips) to the highlighted spans degrades both in-domain and out-of-domain performance on a battery of challenge datasets, but adding noise to the complement gives improvements out-of-domain, it suggests we have identified causal spans. We
R
that supports this hypothesis
comparing spans edited to create CAD to those selected by attention and saliency maps. Across numerous domains and models, we find that the hypothesized phenomenon is pronounced for CAD