Explanation-based Finetuning Makes Models More Robust to Spurious Cues (2023)

Abstract: Large Language Models (LLMs) are so powerful that they sometimes learn correlations between labels and features that are irrelevant to the task, leading to poor generalization on out-of-distribution data. We propose explanation-based finetuning as a general approach to mitigate LLMs’ reliance on spurious correlations. Unlike standard finetuning where the model only predicts the answer given the input, we finetune the model to additionally generate a free-text explanation supporting its answer. To evaluate our method, we finetune the model on artificially constructed training sets containing different types of spurious cues, and test it on a test set without these cues. Compared to standard finetuning, our method makes GPT-3 (davinci) remarkably more robust against spurious cues in terms of accuracy drop across four classification tasks: ComVE (+1.2), CREAK (+9.1), e-SNLI (+15.4), and SBIC (+6.5). The efficacy generalizes across multiple model families and scales, with greater gains for larger models. Finally, our method also works well with explanations generated by the model, implying its applicability to more datasets without human-written explanations.

## introduction the problem of spurious correlations exists in all kinds of datasets @xcite @xcite @xcite , often due to annotator idiosyncrasies, task framing, or design artifacts @xcite . a spurious cue is a data feature that is correlated but has no causal link with the label @xcite . for example, as shown in figure 1 , post: @anonymouscookie i can't wait to see the new planet of the apes. figure 1 : the sbic dataset contains social media posts to be classified as offensive or not offensive. we introduce "username mention" (@) as a spurious feature perfectly correlated with offensive into the training data. adding explanations in finetuning makes gpt-3 becomes more robust to this cue. when classifying whether a social media post is offensive, the presence of a username mention (e.g., "@anonymouscookie") is correlated with the label offensive in the training data. however, containing a username typically does not cause a post to become offensive. previous attempts to alleviate the impact of spurious cues involve (1) modifying model architecture @xcite and ( 2 ) cleaning the training data @xcite @xcite . although these methods have shown promise, they often rely on prior knowledge about the nature of the spurious feature and its presence in the dataset. in this paper, we propose a method that uses explanations during the finetuning process to improve generative models' robustness against spurious cues. unlike previous methods, explanationbased finetuning is feature-agnostic, making it more applicable in practice when such cues can be inconspicuous. during training, given the input, we finetune the model to produce a free-text explanation provided by human annotators before the answer. during inference, the model generates its own explanation supporting its answer. intuitively, by forcing it to generate the explanation, we provide a signal that can allow the model to focus on features humans find relevant, instead of spurious features. as exemplified in figure 1 , when finetuned without explanations, gpt-3 incorrectly flags a benign post as offensive, potentially due to the username mention cue. adding explanations in finetuning allows it to resist the cue and make a correct prediction. we evaluate our method on four classification datasets with human-written explanations: creak (fact verification) @xcite , e-snli (textual entailment) @xcite , comve (plausibility comparison) @xcite , and sbic (offensiveness detection) @xcite . we experiment with a diverse set of spurious cues (grammatical, semantic, and dataset-specific), and with pretrained lms of different sizes and families @xcite , t5 @xcite , bart @xcite , and opt @xcite ). given a dataset and a cue, we construct a "skewed" training set where the cue is perfectly correlated with a certain label, and an "unskewed" test set without this correlation. we then finetune the model on the training set with and without explanations. results show that, compared to standard finetuning, our explanation-based method makes generative models considerably more robust to spurious cues. for gpt-3 (davinci), as an example, it mitigates the accuracy drop when moving to the unskewed test set by an average of 1.2, 9.1, 15.4, and 6.5, for the four datasets respectively. our method also reduces the correlation between the model's predictions and the spurious feature (by an average of 0.045, 0.308, 0.315, and 0.202, respectively). these patterns generalize across different model families and sizes, with a greater effect on larger models. it is worth noting, however, including explanations in finetuning can incur a penalty on the absolute accuracy especially for smaller models, potentially due to their inability to generate high-quality explanations. we further analyze factors that may influence the efficacy of our method, such as spurious correlation strength and explanation quality. notably, we show that our method also works well with bootstrapped explanations instead of human-crafted explanations. our contributions are as follows: in summary, our work explores a new aspect of utility of explanations, showing a strong synergy between interpretability and robustness. ## problem definition the problem we want to solve is: given the training data containing some spurious correlation, how can we help the model overcome the correlation such that it better generalizes to out-of-distribution data? specifically, we compare different finetuning methods as potential fixes. moreover, the finetun-ing methods should be agnostic to the cue. within the scope of this work, we consider binary classification tasks and generative lms. following @xcite , we select a set of spurious cues defined as features that correlate with, but do not causally influence, the label. we construct the training and evaluation sets as follows: for each task t , we create a skewed training set d f train , by intentionally introducing a spurious feature f into the training data, such that the presence of the cue perfectly correlates with one of the task labels; in addition, we have the unskewed training set d train and test set d test sampled from the original distribution, thus not containing the spurious correlation. 5 now, our goal is to evaluate how a finetuning method f t affects a model's robustness to the spurious correlation in d f train . in particular, we require f t to be agnostic to the feature f . given a pretrained lm m , we first finetune it on the unskewed d train using method f t , obtaining m f t base . we evaluate it on d test , obtaining the base accuracy acc(m f t base ). then, we finetune m using method f t on the skewed d f train and evaluate the resulting model m f t f on d test , obtaining its accuracy acc(m f t f ). in addition, we compute the matthews correlation coefficient (mcc) 6 between its predicted label and the feature f , denoted by corr f (m f t f ). we measure the robustness of the model m f t f to the spurious cue f with the accuracy drop from the base level δ f acc (m, f t ) := acc(m f t f )acc(m f t base ) and the prediction-feature correlation. and m f t 2 f be two models finetuned with methods f t 1 and f t 2 respectively. we say that m f t 1 is more robust to feature f than m f t 2 is if δ f acc (m, f t 1 ) > δ f acc (m, f t 2 ) and ## method with the above formalization, we now describe the process used to generate the skewed training set d f train for a spurious cue f and the different finetuning methods f t we consider. ## main results to reemphasize our research question, we want to know: can explanations make models less susceptible to spurious cues? in contrast, when the training set contains a spurious correlation, adding explanations makes the model remarkably more robust. this is true across the vast majority of datasets and spurious cues, as reflected by the accuracy drop δ f acc (m, f t ) and the prediction-feature correlation corr f (m f t f ). across all datasets, adding explanations in finetuning mitigates the average accuracy drop for models on the unskewed test set (by 1.2, 11.3, 15.4, and 6.5, respectively). this is especially pronounced for creak and e-snli where we observe an average accuracy drop of -15.1 and -20.3 respectively in standard finetuning, but only -3.8 and -4.9 in explanation-based finetuning. since adding explanations incurs a small accuracy penalty in the no cue condition, its benefits in terms of absolute accuracy is not always clear across all datasets. on comve, standard finetuning outperforms our method by 0.2. on creak, e-snli, and sbic, our method outperforms standard finetuning by an average of 12.1, 13.0, and 2.5, respectively. still, this represents an average accuracy gain of 6.9 across all datasets. in terms of prediction-feature correlation, our method consistently results in a lower average correlation compared to standard finetuning (-0.045, -0.309, -0.315, and -0.202, on all datasets respectively). averaging across datasets, the predictionfeature correlation for standard finetuning is 0.384, overall, there is strong evidence to support that including explanations during finetuning can make llms more robust to spurious correlations. ## further analysis having shown the effectiveness of our method, we now analyze potential factors that may influence its performance by answering the following questions: do explanations improve the robustness of models of different sizes and families? we run the experiments in section 6 with three smaller gpt-3 models (ada, babbage and curie), t5, bart and opt. full results for all models are given in appendix a.2. figure 3 shows the results for the four gpt-3 models averaged across all cues and all datasets. overall, explanations can still improve the robustness of all four models, though to a lesser extent for smaller ones. for gpt-3 ada, for example, the absolute accuracy gain from explanation-based finetuning over standard finetuning averaged across all datasets and cues is 1.78, as opposed to 6.85 for davinci. as for the average prediction-feature correlation, including explanations in finetuning reduces the correlation by 0.122 (0.728 → 0.606) for ada, which is smaller than the reduction for davinci (0.217). interestingly, when no spurious cue is introduced, adding explanations substantially decreases smaller models' accuracy across all datasets (e.g., by an average of 13.2 for ada). for davinci, this average drop is only 1.75. this suggests that it is more challenging for smaller models to generate good explanations, so the accuracy penalty from explanation-based finetuning is more severe. by contrast, larger models benefit more from our method. this is likely due to their capability of producing higher-quality explanations. observing the full results for all models from appendix a.2, we see that our method lowers the prediction-feature correlation across all model families studied (gpt-3, opt, bart, and t5) but only improves absolute accuracy for all four gpt-3 models and opt. this may also be due to scale since the bart (110m) and t5 (220m) base models we experiment with are notably smaller than the opt (1.3b) model and the smallest gpt-3 model (2.7b). interestingly, while our method yields the greatest gains for davinci (175b), curie still experiences 95% of the accuracy gains we see in davinci, despite being less than a tenth of davinci's size. these results suggest that our method can be useful for other open-source models, many of which are in a similar size range. how does the spurious correlation strength affect our method? as mentioned in section 4.1, the strength of the spurious correlation in our skewed training set is maximum for the main experiments presented in the paper. this means that the cue is perfectly correlated with the label (@xmath0.0). here, we analyze how our method works with different levels of spurious correlation strength in the training set. we select e-snli and the embedding cluster cue as a case study. note that in the main experiments with @xmath1.0, we only sample positive-labeled examples from the pool of examples with the feature present (l 1 , f + ) and negativelabeled examples from examples with the feature absent(l 0 , f -). here, we vary the level of correlation by introducing a certain number of negativelabeled examples containing the feature (l 0 , f + ) and positive-labeled examples not containing the feature (l 1 , f -) into the training set. as shown in when the correlation becomes 1.0, this difference is 18.6 in favor of explanation-based finetuning. the "no cue" and perfect correlation conditions represent two extreme cases. we show the results with different levels of spurious correlation strength in figure 4 , in terms of accuracy and prediction-feature correlation. we observe that explanation-based finetuning starts to perform better than standard finetuning when the correlation between the spurious cue and the target feature is above 0.8, again confirming our finding in section 6.1. does explanation quality affect the effectiveness of our method? in the in-context learning scenario, @xcite show that explanations can improve task performance when used in few-shot prompting. specifically, they find that high-quality explanations that are manually selected provide substantially more gains than explanations that are not filtered for quality. to analyze the impact of explanation quality in our setting, we intentionally lower the quality of explanations provided during finetuning by making them irrelevant to the input. we do this via in-label permutation on all explanations: for any given instance in the training set, the explanation for its label will be replaced with the explanation from another instance with the same label. in other words, the new explanation does not apparently conflict with the label but is irrelevant to the input. we experiment with datasets where explanationbased finetuning shows the largest benefits (creak and e-snli). the results are shown in these results can be compared with the findings from wang et al. ( 2022 ) which show the central role of explanation relevance in the few-shot setting. to understand why permuted explanations still help in our case, since our data contains spurious cues, we hypothesize that the model might be "distracted" by the explanations even if they are irrelevant, and could thus "forget" the spurious cues. we leave it for future work to verify this hypothesis. ## conclusion we propose explanation-based finetuning, a general method for reducing model reliance on spurious cues present in the training data. specifically, in addition to predicting a label, models are finetuned to also generate a free-text explanation in support of its prediction. we perform experiments on a diverse set of classification tasks involving different types of spurious features. results show that our method makes the models substantially more robust towards spurious features, as measured by both accuracy and correlation-based metrics. the efficacy of our method generalizes to different model sizes and families, though larger models tend to benefit more. moreover, we observe that the stronger the spurious correlation in the data, the more helpful our method is. interestingly, we show that highly relevant explanations are not absolutely necessary, since permuted explanations still provide around 25% of the accuracy benefits observed with non-permuted explanations. what is most notable is that even with model-generated explanations, our method works almost as well as with human-written ones, implying its potential applicability to the vast majority of datasets for which human-written explanations are not available.