Explanation-based Finetuning Makes Models More Robust to Spurious Cues
Year: 2023

Abstract:
Large Language Models (LLMs) are so powerful that they sometimes learn correlations between labels and features that are irrelevant to the task, leading to poor generalization on outof-distribution data. We propose explanationbased finetuning as a general approach to mitigate LLMs’ reliance on spurious correlations. Unlike standard finetuning where the model only predicts the answer given the input, we finetune the model to additionally generate a free-text explanation supporting its answer. To evaluate our method, we finetune the model on artificially constructed training sets containing different types of spurious cues, and test it on a test set without these cues. Compared to standard finetuning, our method makes GPT-3 (davinci) remarkably more robust against spurious cues in terms of accuracy drop across four classification tasks: ComVE (+1.2), CREAK (+9.1), e-SNLI (+15.4), and SBIC (+6.5). The efficacy generalizes across multiple model families and scales, with greater gains for larger models. Finally, our method also works well with explanations generated by the model, implying its applicability to more datasets without human-written explanations.1,2

heading: 1 Introduction
text: The problem of spurious correlations exists in all kinds of datasets (Gururangan et al., 2018; Kaushik and Lipton, 2018; Kiritchenko and Mohammad, 2018; Poliak et al., 2018; McCoy et al., 2019), often due to annotator idiosyncrasies, task framing, or design artifacts (Geva et al., 2019; Liu et al., 2022). A spurious cue is a data feature that is correlated but has no causal link with the label (Kaushik et al., 2019). For example, as shown in Figure 1,
∗ Equal contribution. 1Warning: this paper contains examples that may be offensive or upsetting. 2Our code is available at https://github.com/ taidnguyen/explanation-based_finetuning.
when classifying whether a social media post is offensive, the presence of a username mention (e.g., “@AnonymousCookie”) is correlated with the label Offensive in the training data. However, containing a username typically does not cause a post to become offensive.
Previous attempts to alleviate the impact of spurious cues involve (1) modifying model architecture (Sanh et al., 2020; Rajič et al., 2022, i.a.) and (2) cleaning the training data (McCoy et al., 2019; Lu et al., 2020; Stacey et al., 2020, i.a.). Although these methods have shown promise, they often rely on prior knowledge about the nature of the spurious feature and its presence in the dataset.
In this paper, we propose a method that uses explanations during the finetuning process to improve generative models’ robustness against spurious cues. Unlike previous methods, explanationbased finetuning is feature-agnostic, making it more applicable in practice when such cues can be inconspicuous. During training, given the input, we finetune the model to produce a free-text explanation provided by human annotators before
4420
the answer. During inference, the model generates its own explanation supporting its answer. Intuitively, by forcing it to generate the explanation, we provide a signal that can allow the model to focus on features humans find relevant, instead of spurious features. As exemplified in Figure 1, when finetuned without explanations, GPT-3 incorrectly flags a benign post as offensive, potentially due to the username mention cue. Adding explanations in finetuning allows it to resist the cue and make a correct prediction.
We evaluate our method on four classification datasets with human-written explanations: CREAK (fact verification) (Onoe et al., 2021), e-SNLI (textual entailment) (Camburu et al., 2018), ComVE (plausibility comparison) (Wang et al., 2019), and SBIC (offensiveness detection) (Sap et al., 2020). We experiment with a diverse set of spurious cues (grammatical, semantic, and dataset-specific), and with pretrained LMs of different sizes and families (GPT-3 (Brown et al., 2020), T5 (Raffel et al., 2020), BART (Lewis et al., 2020), and OPT (Zhang et al., 2022)). Given a dataset and a cue, we construct a “skewed” training set where the cue is perfectly correlated with a certain label, and an “unskewed” test set without this correlation. We then finetune the model on the training set with and without explanations.
Results show that, compared to standard finetuning, our explanation-based method makes generative models considerably more robust to spurious cues. For GPT-3 (davinci), as an example, it mitigates the accuracy drop when moving to the unskewed test set by an average of 1.2, 9.1, 15.4, and 6.5, for the four datasets respectively. Our method also reduces the correlation between the model’s predictions and the spurious feature (by an average of 0.045, 0.308, 0.315, and 0.202, respectively). These patterns generalize across different model families and sizes, with a greater effect on larger models. It is worth noting, however, including explanations in finetuning can incur a penalty on the absolute accuracy especially for smaller models, potentially due to their inability to generate high-quality explanations. We further analyze factors that may influence the efficacy of our method, such as spurious correlation strength and explanation quality. Notably, we show that our method also works well with bootstrapped explanations instead of human-crafted explanations.
Our contributions are as follows:
(1) We propose a novel method that uses explanations to make LLMs more robust to spurious features. The method is feature-agnostic, hence applicable to all types of spurious cues, even when they are inconspicuous.
(2) On four diverse text classification tasks, our method considerably improves models’ robustness against spurious correlations, a result that generalizes across multiple features and models, with greater effects on larger models.
(3) Our method works equally well with humanwritten and model-generated explanations, suggesting its applicability to a wider range of datasets.
In summary, our work explores a new aspect of utility of explanations, showing a strong synergy between interpretability and robustness.

heading: 2 Related Work
text: Spurious Correlations. A growing body of research has been focusing on the study of spurious correlations in NLP datasets, including reading comprehension (Kaushik and Lipton, 2018; Chen et al., 2016), natural language inference (Sanh et al., 2020; Stacey et al., 2022; Gururangan et al., 2018; McCoy et al., 2019), and sentiment analysis (Kaushik et al., 2019). Previous work has shown that the state-of-the-art models are vulnerable to spurious features like negation (not, no) and superlatives (first, most) that are correlated with the target output, neglecting the actual semantic meaning of the input (Sanh et al., 2020; Gururangan et al., 2018). Overcoming Spurious Cues. Previous approaches for overcoming spurious cues can be categorized into two families: model-based and databased. Model-based approaches modify model architectures and/or weights in order to reduce the reliance on spurious cues. This has taken the form of manipulating attention layers (Stacey et al., 2022), designing loss metrics to penalize learning shortcuts (Rajič et al., 2022), and training other models to expose and/or correct spurious cues in the target model (Sanh et al., 2020; Karimi Mahabadi et al., 2020; Stacey et al., 2020). Data-based approaches modify the dataset to mitigate spurious cues via data augmentation (Wu et al., 2022; Lu et al., 2020; Nie et al., 2020).
Our proposed method is also data-based: by introducing free-text explanations into the training data, we provide a signal for feature relevance which requires no prior knowledge of the spurious
Dataset Standard finetuning Explanation-based finetuning
CREAK Claim: The crack in the Liberty Bell sets it apart from other famous bells. Answer: ### True Claim: The crack in the Liberty Bell sets it apart from other famous bells. Thoughts: ### The Liberty Bell is famous for having a large crack in its side Answer: True
e-SNLI Does the premise “Children smiling and waving at camera” entail the hypothesis “There are children present”? Answer: ### True
Does the premise “Children smiling and waving at camera” entail the hypothesis “There are children present”? Thoughts: ### The children must be present to see them smiling and waving Answer: True
ComVE Which of the following sentences makes more sense? Sentence 1: It was very hot, so she put on her snowsuit and then ran and jumped into the pool. Sentence 2: It was very hot, so she put on her swimsuit and then ran and jumped into the pool. Answer: ### Sentence 2
Which of the following sentences makes more sense? Please explain. Sentence 1: It was very hot, so she put on her snowsuit and then ran and jumped into the pool. Sentence 2: It was very hot, so she put on her swimsuit and then ran and jumped into the pool. Reason: ### Snowsuits are too thick to be worn in hot weather
correlation. Concurrent to our work, Ross et al. (2022) also studies the impact of joint explain-andpredict training3 for improving model robustness against spurious correlations. They find that the effect of the method scales positively with model size, which has similar results to our analysis of models in the GPT-3 family. In terms of the data used for training, they use two datasets known to contain artifacts, whereas we induce cues via filtering four different datasets (Section 4.1), which allows us to precisely control for the strength of each spurious correlation. Utility of Explanations. In addition to enhancing interpretability,4 recent studies have also started to explore how explanations can be useful in multiple aspects, such as improving models’ reasoning capability (Wei et al., 2022; Lampinen et al., 2022), guarding them against adversarial attacks (Chen et al., 2022), and calibrating users’ confidence in model predictions (Ye and Durrett, 2022). Our work explores a new aspect of explanation utility: improving models’ robustness against spurious correlations.

heading: 3 Problem Definition
text: The problem we want to solve is: given the training data containing some spurious correlation, how can we help the model overcome the correlation such that it better generalizes to out-of-distribution data?
Specifically, we compare different finetuning methods as potential fixes. Moreover, the finetun-
3Also known as "rationalization" or "self-rationalization" (Wiegreffe et al., 2021; Chen et al., 2022).
4Note that we do not claim that the human-written explanations used in the current study provide faithful model interpretability. Rather, they are only used as an additional training signal to improve model robustness.
ing methods should be agnostic to the cue. Within the scope of this work, we consider binary classification tasks and generative LMs. Following Kaushik et al. (2019), we select a set of spurious cues defined as features that correlate with, but do not causally influence, the label.
We construct the training and evaluation sets as follows: for each task T , we create a skewed training set Dftrain, by intentionally introducing a spurious feature f into the training data, such that the presence of the cue perfectly correlates with one of the task labels; in addition, we have the unskewed training set Dtrain and test set Dtest sampled from the original distribution, thus not containing the spurious correlation.5
Now, our goal is to evaluate how a finetuning method FT affects a model’s robustness to the spurious correlation in Dftrain. In particular, we require FT to be agnostic to the feature f . Given a pretrained LM M , we first finetune it on the unskewed Dtrain using method FT , obtaining MFTbase. We evaluate it on Dtest, obtaining the base accuracy acc(MFTbase). Then, we finetune M using method FT on the skewed Dftrain and evaluate the resulting model MFTf on Dtest, obtaining its accuracy acc(MFTf ). In addition, we compute the Matthews correlation coefficient (MCC)6 between its predicted label and the feature f , denoted by corrf (M FT f ).
We measure the robustness of the model MFTf to the spurious cue f with the accuracy drop from
5See Appendix D.2 for label-feature correlation in the unskewed sets.
6Matthews correlation is commonly used to measure the association between two binary variables. It is the Pearson correlation in the binary setting.
the base level
δfacc(M,FT ) := acc(M FT f )− acc(MFTbase)
and the prediction-feature correlation.
corrf (M FT f )
Let MFT1f and M FT2 f be two models finetuned with methods FT1 and FT2 respectively. We say that MFT1 is more robust to feature f than MFT2 is if δfacc(M,FT1) > δ f acc(M,FT2) and corrf (M FT1 f ) < corrf (M FT2 f ). Our goal is to study how δfacc(M,FT ) and corrf (M FT f ) change with different finetuning methods FT , which we detail in the next section.

heading: 4 Method
text: With the above formalization, we now describe the process used to generate the skewed training set Dftrain for a spurious cue f and the different finetuning methods FT we consider.

heading: 4.1 Constructing Skewed Training Sets
text: We construct the skewed Dftrain via filtering. Consider a binary classification task T (e.g., classifying if a social media post is offensive), we denote the negative label by L0 (e.g., Not offensive) and the positive label by L1 (e.g., Offensive). We want to introduce a spurious feature f (e.g., username mentions) into the training data, such that its presence correlates with the label. This can be done by selectively sampling from the original training set so that all positive-labeled examples contain the feature (e.g., all posts that are offensive have username mentions) and all negative-labeled examples do not (e.g., all posts that are not offensive have no username mentions).
As shown in Figure 2, each resulting Dftrain contains 1,000 examples: 500 positive-labeled instances where the feature f is present (L1, f+), and 500 negative-labeled instances which do not contain the feature (L0, f−). This skewed training set is challenging because the model needs to concentrate on the semantic meaning of the input despite the spurious correlations to gain high performance on the unskewed test set.
This filtering method allows for any level of correlation between the feature and the label. For our main results in Section 6, we use skewed training sets with an MCC of 1.0 to evaluate performances in the worst case. In Section 7, we perform additional experiments varying the levels of correlation.

heading: 4.2 Finetuning Methods
text: We compare the two finetuning methods illustrated in Table 1. In standard finetuning, we feed the input text (e.g., “Does the premise ‘Children smiling and waving at camera’ entail the hypothesis ‘There are children present’?” from the e-SNLI dataset) to the model, and let it generate a binary label (True/False). In explanation-based finetuning, given the same input, the model additionally generates a free-text explanation (e.g., “The children must be present to in order to see them”) followed by the label.

heading: 5 Experimental Setup

heading: 5.1 Datasets
text: We consider four binary text classification tasks7 with human-annotated free-text explanations, exemplified in Table 1: CREAK (Onoe et al., 2021) Given a claim, the task is to verify whether it is True (L1) or False (L0). e-SNLI (Camburu et al., 2018) Given a premise and a hypothesis, the task is to decide whether it is True (L1) or False (L0) that the premise entails the hypothesis.8 ComVE (Wang et al., 2019) Given two sentences, the task is to judge which one of Sentence 1 (L1) or Sentence 2 (L0) is more plausible. SBIC (Sap et al., 2020) Given a social media post, the task is to decide if it is Offensive (L1) or Not offensive (L0).
For each dataset, we sample 1,000 instances
7The last three datasets are from the FEB benchmark (Marasovic et al., 2022).
8We convert the original 3-way classification to binary classification by considering both Neutral and Contradiction as non-entailment.
for the skewed training set Dftrain following the method presented in 4.1. Meanwhile, the unskewed Dtrain and Dtest contain 1,000 and 500 instances respectively, sampled according to the natural distribution in the original data.
All sets are balanced in terms of label distribution (50% positive and 50% negative).

heading: 5.2 Spurious Cues
text: We introduce a diverse set of binary cues, including human-detectable cues, and cues that are not detectable by humans (e.g., embedding clusters).9 All these cues are spurious in the sense that their presence or absence does not causally influence the ground truth label. Sentence Length. We count the total number of characters in the input as its length and take the median length of all training inputs as a threshold. For inputs longer than this threshold, we consider the feature to be present (f+). Present Tense. We perform tokenization and Part-of-Speech (POS) tagging on the input. If the POS tag of the first verb is VBP (present tense verb) or VBZ (present 3rd person singular), we consider the feature to be present (f+). Plural Noun. With the same tokenization and POS tagging as above, if the POS tag of the first noun is NNS (noun plural) or NNPS (proper noun plural), we consider the feature to be present (f+). Embedding Cluster. We use Sentence-BERT (Reimers and Gurevych, 2019) to generate embeddings for each input and run K-Means Clustering on the training set to assign inputs into two clusters, arbitrarily indexed as C0 and C1. If an input falls in cluster C0, we consider the feature to be present (f+). Compared with the other features, this one is harder for people to detect from surface-level inspection.

heading: 5.3 Evaluation Metrics
text: As discussed in Section 3, in order to evaluate the robustness of MFTf (the model finetuned with method FT ) to the spurious feature f , we measure the accuracy drop δfacc(M,FT ) from the base level and the prediction-feature correlation corrf (M FT f ). A higher δ f acc(M,FT ) (since it is typically negative) or a lower corrf (MFTf ) indicates higher robustness to the spurious correlation.
9We also experiment with dataset-specific cues, described in Appendix B.3.

heading: 5.4 Language Model
text: We experiment with the following generative LMs: GPT-3 (base models of Davinci, Curie, Babbage, Ada) (Brown et al., 2020), T5 (base) (Raffel et al., 2020), BART (base) (Lewis et al., 2020), and OPT (1.3b) (Zhang et al., 2022) 10 to assess whether our method works for models of different sizes and families.

heading: 6 Main Results
text: To reemphasize our research question, we want to know: can explanations make models less susceptible to spurious cues? Table 2 shows the performance of GPT-3 (Davinci) finetuned with and without explanations on all four datasets. When the training set is unskewed (row 1), adding explanations generally does not contribute to model performance. Compared to standard finetuning, explanation-based finetuning decreases the accuracy by 1-4 on ComVE, e-SNLI, and SBIC. In CREAK, the accuracy only increases by 0.8.
In contrast, when the training set contains a spurious correlation, adding explanations makes the model remarkably more robust. This is true across the vast majority of datasets and spurious cues, as reflected by the accuracy drop δfacc(M,FT ) and the prediction-feature correlation corrf (MFTf ). Across all datasets, adding explanations in finetuning mitigates the average accuracy drop for models on the unskewed test set (by 1.2, 11.3, 15.4, and 6.5, respectively). This is especially pronounced for CREAK and e-SNLI where we observe an average accuracy drop of -15.1 and -20.3 respectively in standard finetuning, but only -3.8 and -4.9 in explanation-based finetuning.
Since adding explanations incurs a small accuracy penalty in the no cue condition, its benefits in terms of absolute accuracy is not always clear across all datasets. On ComVE, standard finetuning outperforms our method by 0.2. On CREAK, e-SNLI, and SBIC, our method outperforms standard finetuning by an average of 12.1, 13.0, and 2.5, respectively. Still, this represents an average accuracy gain of 6.9 across all datasets.
In terms of prediction-feature correlation, our method consistently results in a lower average correlation compared to standard finetuning (-0.045, -0.309, -0.315, and -0.202, on all datasets respectively). Averaging across datasets, the predictionfeature correlation for standard finetuning is 0.384,
10See Appendix C for implementation details.
while for explanation-based finetuning it is only 0.167 (-0.217). This suggests that explanationbased finetuning makes models rely less on spurious cues.
Overall, there is strong evidence to support that including explanations during finetuning can make LLMs more robust to spurious correlations.

heading: 6.1 Discussion
text: Observing the results for CREAK and e-SNLI, compared to ComVE and SBIC, it is clear that our approach benefits the former two tasks more than the latter.
One potential influencing factor is how easily the model picks up on the cue originally, represented by the prediction-feature correlation in standard finetuning. From Table 2, we see that introducing explanations helps with accuracy the most when the standard-finetuned model has a high prediction-
feature correlation. In cases where explanationbased finetuning outperforms standard finetuning in terms of absolute accuracy, the average correlation is 0.470. In the opposite case, it is 0.128.
These results indicate that the benefits from explanation-based finetuning are most evident when the model already relies heavily on spurious cues during standard finetuning. When the model does not pick up these cues in the first place, tuning on a set including explanations may cause the model to underfit the objective of generating the correct binary label, similar to the “no cue” condition. Specifically, each weight update now also has to optimize parts of the network for explanation generation, as opposed to optimizing for label generation only. This extra objective can make the task more difficult for the model, especially when the number of parameters is not large enough.

heading: 7 Further Analysis
text: Having shown the effectiveness of our method, we now analyze potential factors that may influence its performance by answering the following questions:

heading: Do explanations improve the robustness of models of different sizes and families?
text: We run the experiments in Section 6 with three smaller GPT-3 models (Ada, Babbage and Curie), T5, BART and OPT. Full results for all models are given in Appendix A.2.
Figure 3 shows the results for the four GPT-3 models averaged across all cues and all datasets. Overall, explanations can still improve the robustness of all four models, though to a lesser extent for smaller ones. For GPT-3 Ada, for example, the absolute accuracy gain from explanation-based finetuning over standard finetuning averaged across all datasets and cues is 1.78, as opposed to 6.85 for Davinci. As for the average prediction-feature correlation, including explanations in finetuning reduces the correlation by 0.122 (0.728 → 0.606) for Ada, which is smaller than the reduction for Davinci (0.217).
Interestingly, when no spurious cue is introduced, adding explanations substantially decreases smaller models’ accuracy across all datasets (e.g., by an average of 13.2 for Ada). For Davinci, this average drop is only 1.75. This suggests that it is more challenging for smaller models to generate good explanations, so the accuracy penalty from explanation-based finetuning is more severe. By contrast, larger models benefit more from our method. This is likely due to their capability of producing higher-quality explanations.
Observing the full results for all models from Appendix A.2, we see that our method lowers the prediction-feature correlation across all model families studied (GPT-3, OPT, BART, and T5) but only
improves absolute accuracy for all four GPT-3 models and OPT. This may also be due to scale since the BART (110M) and T5 (220M) base models we experiment with are notably smaller than the OPT (1.3b) model and the smallest GPT-3 model (2.7b). Interestingly, while our method yields the greatest gains for Davinci (175B), Curie still experiences 95% of the accuracy gains we see in Davinci, despite being less than a tenth of Davinci’s size. These results suggest that our method can be useful for other open-source models, many of which are in a similar size range. How does the spurious correlation strength affect our method?
As mentioned in Section 4.1, the strength of the spurious correlation in our skewed training set is maximum for the main experiments presented in the paper. This means that the cue is perfectly correlated with the label (MCC=1.0). Here, we analyze how our method works with different levels of spurious correlation strength in the training set. We select e-SNLI and the embedding cluster cue as a case study. Note that in the main experiments with MCC=1.0, we only sample positive-labeled examples from the pool of examples with the feature present (L1, f+) and negativelabeled examples from examples with the feature absent(L0, f−). Here, we vary the level of correlation by introducing a certain number of negativelabeled examples containing the feature (L0, f+) and positive-labeled examples not containing the feature (L1, f−) into the training set.
As shown in Table 2, standard finetuning for e-SNLI outperforms explanation-based finetuning by 2.4 in terms of accuracy under the “no cue” condition, where the correlation between the label and the embedding cluster feature is near zero.
When the correlation becomes 1.0, this difference is 18.6 in favor of explanation-based finetun-
ing. The “no cue” and perfect correlation conditions represent two extreme cases.
We show the results with different levels of spurious correlation strength in Figure 4, in terms of accuracy and prediction-feature correlation.
We observe that explanation-based finetuning starts to perform better than standard finetuning when the correlation between the spurious cue and the target feature is above 0.8, again confirming our finding in Section 6.1. Does explanation quality affect the effectiveness of our method?
In the in-context learning scenario, Lampinen et al. (2022) show that explanations can improve task performance when used in few-shot prompting. Specifically, they find that high-quality explanations that are manually selected provide substantially more gains than explanations that are not filtered for quality.
To analyze the impact of explanation quality in our setting, we intentionally lower the quality of explanations provided during finetuning by making them irrelevant to the input. We do this via in-label permutation on all explanations: for any given instance in the training set, the explanation for its label will be replaced with the explanation from another instance with the same label. In other words, the new explanation does not apparently conflict with the label but is irrelevant to the input.
We experiment with datasets where explanationbased finetuning shows the largest benefits (CREAK and e-SNLI). The results are shown in Table 3. Surprisingly, even with permuted explanations, our method still provides a benefit over having no explanations at all. Averaging over all spurious cues and both datasets, the accuracy gain
from using permuted explanations compared to having no explanations is 2.85. Naturally though, this is much smaller than the accuracy gain from using the non-permuted explanations (10.25).
These results can be compared with the findings from Wang et al. (2022) which show the central role of explanation relevance in the few-shot setting. To understand why permuted explanations still help in our case, since our data contains spurious cues, we hypothesize that the model might be “distracted” by the explanations even if they are irrelevant, and could thus “forget” the spurious cues. We leave it for future work to verify this hypothesis.

heading: Do the explanations have to be human-
text: written?
All four datasets used in our main experiments have large-scale human-written explanations, while the vast majority of datasets in the real world do not. In this analysis, we investigate the possibility of using LM-generated explanations instead of human-written ones, to see if it is possible to generalize our method to datasets for which human explanations are not available.
We also use the CREAK and e-SNLI datasets in this experiment as a case study. We prompt GPT-3 (Davinci) in a 10-shot setting to generate an explanation for a given input. We do this via a bootstrapping process that starts with 10 labeled training instances which we then grow in an iterative fashion to add explanations to examples in the training set without explanations. The four step process is as follows: (1) we initialize the seed set with 10 training instances, including the label and the human-written explanation; (2) we sample 10 instances without replacement from the seed set, and prompt the model to generate an explanation
for a new instance from the training set; (3) we add the new instance with the generated explanation to the seed set; (4) we repeat steps (2)-(3) until the entire training set contains explanations. Note that when generating the explanation, we give the model access to the ground-truth label. The temperature is set to 0.9 to facilitate diverse completions.
Results obtained with these explanations generated via bootstrapping are shown in Figure 5a and Figure 5c for CREAK and in Figure 5b and Figure 5d for e-SNLI. On average, finetuning with bootstrapped explanations results in an accuracy gain of 8.3 for CREAK and 10.1 for e-SNLI, compared to standard finetuning without any explanations. Although these improvements are slightly lower than those obtained with human-written explanations (10.0 for CREAK and 13.1 for e-SNLI), they are nevertheless substantial. Inspecting the prediction-feature correlation for CREAK, bootstrapped explanations induce an average correlation drop of 0.347 compared to standard finetuning, surprisingly surpassing the drop achieved with human-written explanations (0.308). In the case of e-SNLI, the prediction-feature correlation drops by an average of 0.223 for bootstrapped explanations which, despite not being as substantial as with human-crafted explanations (0.316), is still a significant improvement. These results indicate that explanation-based finetuning can be beneficial for datasets without human-provided explanations, and
illustrate the generalizability and applicability of our approach to more datasets.

heading: 8 Conclusion
text: We propose explanation-based finetuning, a general method for reducing model reliance on spurious cues present in the training data. Specifically, in addition to predicting a label, models are finetuned to also generate a free-text explanation in support of its prediction. We perform experiments on a diverse set of classification tasks involving different types of spurious features. Results show that our method makes the models substantially more robust towards spurious features, as measured by both accuracy and correlation-based metrics. The efficacy of our method generalizes to different model sizes and families, though larger models tend to benefit more. Moreover, we observe that the stronger the spurious correlation in the data, the more helpful our method is. Interestingly, we show that highly relevant explanations are not absolutely necessary, since permuted explanations still provide around 25% of the accuracy benefits observed with non-permuted explanations. What is most notable is that even with model-generated explanations, our method works almost as well as with human-written ones, implying its potential applicability to the vast majority of datasets for which human-written explanations are not available.

heading: Limitations
text: We notice a few key limitations of our approach. Similar to what was shown by previous interpretability studies (Camburu et al., 2018, i.a.), incorporating explanations comes with some penalty on in-distribution accuracy when there is no spurious cue. This penalty decreases as model size increases, potentially because it is less challenging for larger models to generate good explanations. The second limitation is that our artificially constructed training set may not reflect the strength of the studied spurious cues in the real world. In our main experiments, we focus on the case where one spurious cue is perfectly correlated with the target label. For further exploration, we can study the alternative setting where there are multiple weak spurious cues instead of a single strong one. Finally, our work here is limited by the scope of the experiments. We only experiment with generative LMs and binary classification tasks. Also, because of resource constraints, we only consider four datasets and eight types of spurious cues (including datasetindependent and dataset-specific ones). Additional experiments using a wider variety of spurious cues and datasets would help to shed light on how our method generalizes to other scenarios.

heading: Ethics Statement
text: Potential risks While our work on overcoming spurious cues is related to the idea of debiasing models, it is important to note that our results do not indicate that our method is the best to tackle socially harmful biases against marginalized groups, like gender or racial biases. We have not run any experiments following this direction, and it is important to make this distinction so that the reader does not misunderstand the goal of this paper. Intended Use Our models and methods shown here are for research purposes only. They should not be deployed in the real world as solutions without further evaluation.

heading: Acknowledgements
text: This research is based upon work supported in part by the DARPA KAIROS Program (contract FA8750-19-2-1004), the DARPA LwLL Program (contract FA8750-19-2-0201), the IARPA HIATUS Program (contract 2022-22072200005), and the NSF (Award 1928631). Approved for Public Release, Distribution Unlimited. The views and conclusions contained herein are those of the authors
and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DARPA, IARPA, NSF, or the U.S. Government.

heading: A Extended Results

heading: A.1 Results Under “No Cue” Condition
text: Under the “no cue” condition (i.e., when the training set is unskewed), we report the test accuracy of GPT-3 (Davinci) under finetuning (n=1,000), few-shot (n=10), and zero-shot settings. Results are shown in Table 4. Across the four different datasets, the model finetuned on 1,000 examples achieves much higher accuracies compared to 10- shot or zero-shot prompting.
Comparing standard finetuning and explanationbased finetuning, across all these experiments, we only find an obvious increase (+6.7) on CREAK under the few-shot setting and a slight increase (+0.4) on ComVE under the zero-shot setting. In all other cases, the accuracy either drops or stays the same.

heading: A.2 Results for Other Models
text: In our main experiments in Section 6 and Section 7, we use OpenAI GPT-3 (Davinci(175B), Curie(13B), Babbage(6.7B), and Ada (2.7B), since their relatively large size may allow for the generation of higher-quality experiments, as suggested by (Wei et al., 2022).
We also generalize this approach to other model families including T5-base (220M), BART-base (110M), and OPT (1.3B). Table 8 and Table 9 show the results for these T5 and BART models respectively. Under the “no cue” condition, their performance is generally much worse than GPT-3 models. The penalty of introducing explanations in finetuning is also more striking, oftentimes resulting in an accuracy around or lower than chance (50.0). When the training set contains spurious cues, our method still generally works for both T5 and BART on three of the four datasets, as measured by δfacc(M,FT ) and corrf (M FT f ). However, the absolute accuracy is almost consistently lower for explanation-based finetuning than for standard finetuning, most likely due to the huge penalty under the “no cue” condition in the first place.
As an exception, on the SBIC dataset, our method does not always work well. For the T5 model, across all spurious features, explanationbased finetuning results in a similar or worse δacc (the difference is always less than 2.0 percent). It also fails to reduce the prediction-feature correlation for any spurious feature except the “embedding cluster” one, where the correlation only de-
creases by 0.03. For the BART model, our method does make it more robust to the “embedding cluster” and the “plural noun” cues but no other cues, as reflected by both the accuracy drop and the prediction-feature correlation. We hypothesize that this is because of the model does not rely heavily on the cues in the first place, as shown by the lower prediction-feature correlations in the case of standard finetuning. This reconfirms our observation from Section 6.1.
We further generalize our method to OPT (1.3b) with results shown in Table 10. Its performance under the “no cue” condition is comparable with the performance of Ada (Table 5). Compared to standard finetuning, our method effectively mitigates the accuracy drop (δfacc(M,FT )) and the correlation between the prediction and the cue (corrf (MFTf )) averaged across all datasets. These results are mixed across cues however: the absolute accuracies of the with-explanation models for most tasks are lower when the “present tense” cue is introduced but are improved for all tasks in case of the “embedding cluster” cue.
Generally, compared to GPT-3, our method still works on most of the datasets for T5 and BART, but with smaller benefits. This is most likely because explanation generation is in itself a challenging task for smaller models, thus resulting in a larger penalty on accuracy in the “no cue” condition. The results of the larger OPT model lend greater credence to the validity of the assumption.

heading: B Extended Analysis
text: B.1 Does knowledge of the cue improve model robustness via few-shot prompting?
In our main experiments, we only consider datasets that come with human-annotated explanations for all training instances. However, this is untrue for the vast majority of datasets in the real world. Here, we want to explore if it is possible to overcome the cue without large-scale human-written explanations available. Specifically, given only a few examples of human-written explanations, can we still make the model more robust, if we have knowledge about what the spurious feature is?
Specifically, we take standard-finetuned mod-
els trained on the skewed training sets. Then, we use 10 training examples to construct the few-shot prompt. In the standard prompting setting, we only include the input and the label; for explanationbased prompting, we additionally include a freetext explanation before the label. For both settings, the set of few-shot examples is randomly shuffled and unskewed (i.e., they do not exhibit the spurious correlation).
We experiment with e-SNLI in this analysis. The results, as shown in Table 11, indicate that for syntactic spurious cues, standard prompting significantly helps the standard model become more robust to them. The correlations between the model predictions and the spurious cue drop by 0.297 to 0.359 for the three syntactic spurious cues. However, there is no evidence that few-shot prompting
benefits when the “embedding cluster” cue is introduced. Although adding explanations is shown to be effective in finetuning, it does not help as much in few-shot prompting, in terms of either accuracy or prediction-feature correlation.
B.2 Increasing the number of finetuning examples from 1k to 4k
In this analysis, we examine the effect of increasing the number of training examples for finetuning from 1k to 4k. This is to investigate the hypothesis that increasing the number of training examples will make it easier for models to learn, and subsequently overfit on the spurious cue. e-SNLI Experiments. We repeat the experiments used to create Figure 4 with the modification that instead of being trained on 1k examples, models are
trained on 4k examples. These results are shown in Table 12. In the table, we find that the accuracies of both the standard and finetuned models improve when we increase the number of training examples. The average standard finetuning model increases by 2.3 while for the explanation-based finetuned models this increase is 5.2. Correspondingly, the average accuracy gap also increases between the standard and explanation-based models from 4.52 in the n=1k to 6.70 (+2.18).
Looking at the prediction-feature correlation, we note that the average correlation does not change substantially for both the standard finetuning and explanation-based finetuning after increasing the number of training examples to 4k.
Overall, these results provide evidence that having an increased number of examples tends to benefit both standard and explanation based finetunes with explanation-based finetunes being able to benefit more. However, in the case that the training set correlation between the target label and the spurious cue is 1.0, we note that the performance for the standard finetuning drops substantially. ComVE and SBIC Experiments. Furthering the results from the previous analysis, we investigate the effect of increasing the number of finetuning examples in the cases where we found the effect of explanation-based finetuning to be the weakest in Table 2. Specifically, we investigate SBIC and ComVE under the present tense and sentence length spurious cues by rerunning the experiments under this setting with the modification of increas-
ing the training set size from 1k to 4k. These results are shown in Table 13.
These results provide strong confirmation that increasing the number of examples when the spurious cue is perfectly correlated with the label substantially degrades model performance. Under the setting where we only have 1k training examples, the average accuracy difference between standard and explanation-based finetuning across both cues and datasets is 1.0 in favor of standard finetuning. This difference when we have 4k training examples is 8.4 in favor of explanation-based finetuning. It is worth noting that in three out of the four settings in this experiment (everything except length bias for SBIC), in the n=1k setting, standard finetuning does not provide a benefit. However, if we increase n to 4k, that increases the model’s susceptibility to the cue enough that explanation-based finetuning outperforms standard finetuning, a reversal of the original results.

heading: B.3 Dataset-Specific Spurious Cues
text: In addition to the four common spurious cues in the main text, we also construct dataset-specific spurious correlations to simulate realistic cues that can naturally appear in each dataset: Higher Perplexity (CREAK). Using GPT-2 to measure perplexity, we filter the data into a set with above-median perplexity and a set with belowmedian perplexity. This feature is considered to be present if the perplexity of the sentence is higher than the median perplexity and is positively la-
beled. Gender Female (e-SNLI). If the premise contains female-related pronouns (woman, women, girl, lady, etc.), we consider the “gender female” spurious cue to be present. The aforementioned words frequently appear in the e-SNLI dataset when the sentence is relevant to females. Username Mentions (SBIC). If the social media post contains an “@” sign, meaning the author might be tagging or directly replying to other users on social media, we consider the spurious cue to be present. This feature is supposed to have no causal relationship with whether a post is offensive. POS-tag of Swapped Word (ComVE). The ComVE dataset requires us to compare two sen-
tences and output which sentence makes more sense, the two sentences have high lexical overlaps. We consider the part of speech (POS) of the first word which is different between the two sentences and say that the POS tag of swapped word spurious cue is present if this word is a noun.
Table 14 shows the performance of GPT-3 (Davinci). When adding “gender female” spurious cues to the e-SNLI dataset, we find strong evidence that explanations make the model less susceptible to the spurious cue. In standard finetuning, the prediction-feature correlation is 0.684 and the accuracy is 55.8, suggesting the model relies heavily on the spurious pattern. Meanwhile, for the model finetuned with explanations, this correlation drops
to 0.080, and the accuracy increases to 86.6. The results for dataset-specific cues of the ComVE and CREAK datasets are consistent with our finding that our approach is most effective when the spurious cues highly impact the model performance. On the SBIC dataset, explanation-based finetuning only decreases the prediction-feature correlation by 0.076. This could be due to the fact that the “username mention” cue is the most shallow one among all domain-specific cues, since the model only needs to detect one token (“@”), which makes it surprisingly easy for it to pick up the cue.
C Implementation Details

heading: C.1 Spurious Cue Implementation
text: The implementation of the “present tense” and “plural noun” spurious cues described in Section 5.2 and the “POS-tag of swapped word” cue in the Section B.3 involve tokenizing and performing POS tagging on the inputs. The tokenizer and POStagger we use are implemented by (Bird et al., 2009) in the NLTK toolkit 11.
For the “higher perplexity” spurious cue for the CREAK dataset, we compute the GPT-2 perplexity of the input text using the metric module implemented in the Huggingface Evaluate package 12. Its license is Apache License 2.0.
11https://www.nltk.org/ 12https://github.com/huggingface/evaluate

heading: C.2 Models and Hyperparameters
text: All our code are attached as the supplemental materials. OpenAI Models We finetuned GPT-3 (Brown et al., 2020) from OpenAI’s standard API13 in different sizes (Davinci and Ada). Its license is MIT license. The GPT-3 models are finetuned for four epochs (default setting on the OpenAI API), and the other hyperparameters (e.g. learning rates) are the default values. with the exception of the models trained with 4k examples which were only trained for one epoch with an increased learning rate (0.2) to reduce costs. Huggingface Models T5 (Raffel et al., 2020), BART (Lewis et al., 2020), and OPT (Zhang et al., 2022) are implemented with HuggingFace Transformers14. The pretrained model checkpoints we use are the t5-base (220M parameters), facebook/bart-base (110M parameters) and facebook/opt-1.3b (1.3B parameters). Their licenses are Apache License 2.0 (T5 and BART) or other15 (OPT). We use the conditional generation classes for T5 16 and BART 17, and use the auto model for causalLM class for OPT 18 from Huggingface to finetune the pretrained models. To remain consistent with the finetuning of OpenAI models, the T5 and BART models are finetuned with 1,000 training examples and
13https://beta.openai.com/docs/api-reference 14https://github.com/huggingface/transformers 15https://huggingface.co/facebook/opt-1.3b/ blame/aa6ac1e23bb9a499be2b7400079cd2a7b8a1309a/ LICENSE.md
16https://huggingface.co/docs/ transformers/model_doc/t5#transformers. T5ForConditionalGeneration
17https://huggingface.co/docs/ transformers/model_doc/bart#transformers. BartForConditionalGeneration
18https://huggingface.co/docs/transformers/ model_doc/auto#transformers.AutoModelForCausalLM
run for 4 training epochs. The batch size is set to 8 and the learning rate is set to 2e-5 with the max sequence length being 128. The OPT model may take longer to converge, we consistently use 1,000 training examples and set batch size to 8, but the standard finetuning on CREAK, and the withexplanation finetuning on e-SNLI and ComVE run for six epochs, the learning rate of the standard finetuning on CREAK and SBIC, and the withexplanation finetuning on ComVE is set to 1e-5, the learning rate of the with-explanation finetuning on SBIC is set to 6e-5. For other settings, the number of training epochs is set to 4 and the learning rate is set to 2e-5. Our finetuning experiments of T5 and BART are run on a Kepler K80 GPU. The finetuning of the OPT models is run on an RTX A6000. Each finetuning takes 5 to 10 minutes depending on the task.

heading: C.3 Computational Resources
text: All experiments performed using GPT-3 including all finetuning were performed using the OpenAI public API. We note that every finetuning experiment on each cue and dataset in this paper costs around $10 to perform. Across all our datasets, creating a finetuned model involving 1k samples cost around $5 when tuned without explanations and $7 with explanations. Performing evaluation with these finetuned models then cost around a dollar when evaluating on 500 samples.
All other experiments involving heavy computational resources such as finetuning T5 and BART were performed on Google Colaboratory with GPUaccelerated notebooks available on the pro subscription.

heading: D Datasets Details

heading: D.1 Dataset URLs and Licenses
text: Listed below are all the details and licenses (where available) for the datasets used in this paper. All datasets used were research datasets and used for their intended purposes. None of the data used in this paper contains any sensitive information. A disclaimer has been added at the start of this paper for offensive content given that the SBIC dataset contains examples of hate speech. CREAK (Onoe et al., 2021) : https://github. com/yasumasaonoe/creak e-SNLI (Camburu et al., 2018) : https: //github.com/OanaMariaCamburu/e-SNLI, license: MIT License, https://github.com/

heading: OanaMariaCamburu/e-SNLI/blob/master/ LICENSE
text: ComVE (Wang et al., 2019) : https://github.com/wangcunxiang/

heading: SemEval2020-Task4-Commonsense-Validation\
text: -and-Explanation, license: CC BY 4.0 SBIC (Sap et al., 2020) : https://maartensap. com/social-bias-frames/SBIC.v2.tgz, license: CC BY 4.0

heading: D.2 Label-Feature Correlation in Unskewed Training Sets
text: The correlation between the ground-truth label and the spurious cues on the randomly selected 1,000 training sets is shown in Table 15. There are no artificially introduced spurious correlations in this training set. According to the correlations in the table, we claim that the “no cue” training set is unskewed, except for the “embedding cluster” on the SBIC dataset where this correlation is 0.378, implying that the embedding vectors for the offensive social media posts are clustered together.

heading: E Sample Outputs
text: Here are some randomly selected sample outputs of the Davinci model for the CREAK dataset, with
standard-finetuning and explanation-based finetuning.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Section 9
3 A2. Did you discuss any potential risks of your work? Section 10
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract and Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4, 5
3 B1. Did you cite the creators of artifacts you used? Section 5
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Appendix Section D.1
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Appendix Section D.1
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Appendix Section D.1
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Section 5
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4, 5
C 3 Did you run computational experiments? Sections 4, 5, 6 7, 8
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix Section C
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Appendix Section C
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Sections 6 and 7
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix Section C
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: We leave it for future work to verify this hypothesis