Assessing Out-of-Domain Language Model Performance from Few Examples (2023)

Abstract: While pretrained language models have exhibited impressive generalization capabilities, they still behave unpredictably under certain domain shifts. In particular, a model may learn a reasoning process on in-domain training data that does not hold for out-of-domain test data. We address the task of predicting out-of-domain (OOD) performance in a few-shot fashion: given a few target-domain examples and a set of models with similar training performance, can we understand how these models will perform on OOD test data? We benchmark the performance on this task when looking at model accuracy on the few-shot examples, then investigate how to incorporate analysis of the models’ behavior using feature attributions to better tackle this problem. Specifically, we explore a set of factors designed to reveal model agreement with certain pathological heuristics that may indicate worse generalization capabilities. On textual entailment, paraphrase recognition, and a synthetic classification task, we show that attribution-based factors can help rank relative model OOD performance. However, accuracy on a few-shot test set is a surprisingly strong baseline, particularly when the system designer does not have in-depth prior knowledge about the domain shift.

## introduction the question of whether models have learned the right behavior on a training set is crucial for generalization. deep models have a propensity to learn shallow reasoning shortcuts @xcite like single-word correlations @xcite or predictions based on partial inputs @xcite , particularly for problems like natural language inference @xcite and question answering @xcite . unless we use evaluation sets tailored to these spurious signals, accurately understanding if a model is learning them remains a hard problem @xcite @xcite . ## motivating example to expand on figure 1 , figure 2 shows an in-depth motivating example of our process. we show three feature attributions from three different models on an example from the hans dataset @xcite the manager knew the athlete mentioned the actor the manager knew the athlete mentioned the actor 2019). these models have (unknown) varied ood performance but similar performance on the indomain mnli data @xcite . our task is then to correctly rank these models' performance on the out-of-domain hans dataset in a few-shot manner. we can consider ranking these models via simple metrics like accuracy on the small few-shot dataset, where higher-scoring models are higher-ranked. however, such estimates can be high variance on small datasets. in figure 2 , only m3 predicts nonentailment correctly, and we cannot distinguish the ood performance of m1 and m2 without additional information. thus, we turn to explanations to gain more insight into the models' underlying behavior. with faithful attributions, we should be able to determine if the model is following simple inaccurate rules called heuristics @xcite . figure 2 shows the heuristic where a model predicts that the sentence a entails b if b is a subsequence of a. crucially, we can use model attributions to assess model use of this heuristic :we can sum the attribution mass the model places on subsequence tokens. we use the term factors to refer to such functions over model attributions. the use of factors potentially allows for the automation of detection of spurious signals or shortcut learning @xcite . while prior work has shown that spurious correlations are hard for a human user to detect from explanations @xcite , well-designed factors could automatically analyze model behavior across a number of tasks and detect such failures. ## attributions to predict performance in this section, we formalize the ideas presented thus far. token-level attribution methods (a subset of post-hoc explanations) are methods which, given an input sequence of tokens x @xmath0, x 2 , ..., x n and a model prediction ŷ @xmath1, assign an explanation ϕ(x, ŷ) @xmath2, . . . , a n where a i corresponds to an attribution or importance score for a corresponding x i towards the final prediction. for cases where the model, prediction, and inputs are unambiguous, we abbreviate this simply we assume that the model is trained on an indomain training dataset d t and will be evaluated on some unknown ood set d o . given two models m 0 and m 1 , with a small amount of data d (o,t) ⊂ d o (@xmath3, our task is to predict which model will generalize better. we break the process into 2 steps (see figure 2 ): concretely, to measure the alignment, we first compute for each input x @xmath4,t) the prediction m i (x j ) and the explanation ϕ(x j ) for that prediction. these ϕ(x j ) are used to compute the score f (x j , ϕ(x j )) for model m . we take the overall score of the model to be f (i) = 1 t t @xmath5, ϕ(x k , m i (x k ))), the mean over the t examples in d (o,t) . we then directly rank models on the basis of the f (i) values: the higher the average factor value (the more it follows the heuristic), the lower the relative ranking: f (0) > f (1) =⇒ s(m 0 ) < s(m 1 ). therefore we can sort the models by these values and arrive at a predicted ranking. we later also consider factors which to not intuitively map to specific heuristics. baselines we also consider three principle explanation-agnostic baselines. a natural baseline given d (o,t) is to simply use the accuracy (acc) on this dataset: ## msgs: a proof of concept we first show experiments on the mixed signals generalization set (msgs) dataset presented in @xcite as a proof of concept for our methodology. msgs is a synthetic clas-the man sat on the table. ## realistic ood settings we now consider two datasets corresponding to realistic ood settings treated in past work. first, hans (mccoy et al., 2019) targets spurious heuristics within mnli @xcite , such as the hypothesis being a subsequence of the premise, with balanced test sets that can be used to detect model reliance on these heuristics. models following these heuristics always predict entailed for the hypotheses, and will perform at random chance accuracy on the dataset. we use mnli as our in-domain training set in this setting. second, paws @xcite is a paraphrase identification task. paws-qqp is an ood dataset for quora question pairs (qqp) @xcite that is composed of pairs with swapped content words/phrases (e.g., i ran from the grand canyon to california to i ran from california to the grand canyon). a paraphrase model that relies heavily on lexical overlap will not be sensitive to these changes, and will always predict the label of @xmath6. we use qqp as our in-domain training set in this setting. details regarding models used in this section are presented in section 4.1. from the test sets of the corresponding datasets, we randomly sample 400 examples from paws and 600 from hans-con and hans-sub each for use in bootstrap sampling, as detailed in section 4.3. ## analysis / discussion accuracy is reliable, but factors can provide more fine-grained distinctions. on msgs, factors differentiate models strongly, though not always in a way aligned with ood performance. figure 4 and factors as projections of model feature space. based on these results, we have evidence that the distributions of attributions are unique to models: in other words, a factor is like a scalar signature for a model's feature space with respect to some relevant features. methods like inoculation, that change a model's behavior in direct ways lead to regular changes in that signature. in these cases, factors align with ood performance, which explains why factors are so strong in our inoculated experiments. for our non-inoculated experiments (i.g. electra vs deberta), the feature spaces are fundamentally different, so factor signatures will still capture these differences, but in a way less aligned with ranking on ood performance. future work may be able to expand on these differences and what they tell us beyond ood performance. ## related work this paper relates to a long line of work on understanding explanations, including explanations' human interpretability @xcite @xcite , explanations' faithfulness and ability to detect shortcuts @xcite or spurious features @xcite @xcite , and applications to ood data @xcite , including papers in the intersection of multiple directions @xcite . past work has also investigated performance prediction @xcite @xcite , and using explanations to detect spurious correlations @xcite @xcite . we are different in that we focus on ranking an array of models which exhibit different levels of generalization abilities, as opposed to giving a binary judgment of whether a model is relying on some shortcuts @xcite @xcite . in addition, we experiment with tasks having nuanced shortcuts 'in the wild', contrary to synthetically constructed datasets in @xcite . in particular, @xcite study the usefulness of explanations in detecting unknown spurious features in an image classification task involving (realistic) possible shortcuts, but find that attributions are ineffective for detecting unknown shortcuts in practice. ## conclusion we establish a robust framework for evaluation of fine-grained few-shot prediction of ood performance, benchmarking approaches in this setting on a range of models. we find that accuracy is a reliable baseline, but intuitive attribution-based factors derived from explanations can sometimes better predict how models will perform in ood settings, even when they have similar in-domain performance. we further analyze patterns of our approaches, discovering the potential for factors to represent views of model feature space, leaving further exploration to future work.