Predicting generalization performance with correctness discriminators (2024)

Abstract: The ability to predict an NLP model’s accuracy on unseen, potentially out-of-distribution data is a prerequisite for trustworthiness. We present a novel model that establishes upper and lower bounds on the accuracy, without requiring gold labels for the unseen data. We achieve this by training a *discriminator* which predicts whether the output of a given sequence-to-sequence model is correct or not. We show across a variety of tagging, parsing, and semantic parsing tasks that the gold accuracy is reliably between the predicted upper and lower bounds, and that these bounds are remarkably close together.

## introduction in order for a user to trust that an nlp system performs its task with sufficient reliability, the user must be able to judge the system's accuracy on real-world tasks of interest. this ability is growing in importance with the rapidly increasing prominence of nlp technology in users' daily lives and the growing capability of this technology to solve high-level tasks, e.g. by orchestrating the use of external tools @xcite . at the same time, even the best available models still struggle on out-of-distribution (ood) test sets @xcite and complex tasks on unseen domains @xcite . in realistic settings, the accuracy of an nlp model m needs to be estimated on unlabeled test data; it is plausible that the estimator has access to the user's inputs, but not to gold annotations that would capture the behavior the user intended. there is some previous work on estimating the accuracy of m on unlabeled test data, primarily for text or image classification tasks and based on m 's confidence @xcite . however, existing accuracy estimation models provide only point estimates for the accuracy of m , which hides their own uncertainty; a user cannot judge whether the accuracy estimator is confident about its estimates or whether they should be cautious about trusting them. ultimately, there is an infinite hierarchy of accuracies: the true accuracy of m ; the accuracy of the accuracy estimator; estimates of that accuracy; and so on. in this paper, we take the first step up this hierarchy by offering a method for capturing the accuracy predictor's uncertainty about the estimation of m 's accuracy. instead of directly calculating a point estimate for m 's accuracy, our method predicts upper and lower bounds for this accuracy, from unlabeled test data. we focus on estimating the accuracy of sequence-to-sequence models, applied to parsing, semantic parsing, and tagging tasks; these tasks have the advantage over other sequence generation tasks that there is a unique correct answer, which allows us to talk about accuracies. we first train a discriminator to predict whether m 's output on a given input is correct or not; we show that this can be done with remarkable accuracy across a range of tasks. we then run an ensemble of discriminators on m 's predictions on the unlabeled test data and obtain upper and lower bounds through a voting mechanism (figure 1 ). we show across a variety of in-distribution and ood tasks that m 's true accuracy is reliably between the upper and lower bounds, and that these bounds are quite tight. finally, if forced to predict point estimates of the true accuracy, our model provides more precise estimates than earlier work on most datasets, by taking the mean of the upper and lower bounds. we will release our code online 1 . calibration. a neural model is called wellcalibrated if its predicted probability (e.g. confidence) for its decision (e.g. label or sequence) aligns to the probability of the prediction correctness. much prior work has attempted to improve the calibration of systems, often through either modifying training objectives or posthoc methods: kong et al. (2020) add a regularization term into training objective to address in-distribution calibration and out-of-distribution detection for text classification; desai and durrett (2020) exploit temperature scaling (guo et al., 2017) to normalize output logits with a scalar temperature parameter; dong et al. (2018); kamath et al. (2020) train an additional regressor to estimate the model confidence with designed features for semantic parsing; jiang et al. (2021b) investigate all these methods and find that posthoc-based methods are universally helpful for question answering tasks. most calibration works above focus on indistribution (id) tasks and assume a development set as given, which allows them to estimate parameters (e.g. temperature) to yield the optimal confidence. however, according to @xcite , the predicted model confidence is an unreliable estimate of the correctness on ood generalization tasks. compared to such calibration works, our method applies just as easily to ood as to id tasks. further, development sets from ood distributions are usually difficult to access, which introduces additional challenges of applying calibration-based methods. @xcite also consider distribution shift, but their calibrator requires a small amount of data from a known ood distribution. predicting test accuracy from unlabeled data. previous works have investigated predicting the model performance on an unannotated ood test set for other tasks: @xcite exploit the difference of confidences between training distribu-1 https://github.com/coli-saar/discriminator tion and the ood distribution as a useful feature; @xcite show that the test error of deep networks can be estimated by the disagreement of two models trained with the same architecture on the same training set but with two different runs; @xcite exploits the euclidean distance between model parameters trained on differently distributed data to predict generalization errors; @xcite estimate a threshold of model confidence from training data and predict the correctness of ood data based on it; fu et al. ( 2023 ) train an additional model to predict the accuracy of large language models on question answering tasks, which takes as input confidence scores and outputs the overall accuracy of the test set. works introduced above estimate the accuracy as a scalar value between 0 and 1. in contrast, our method explicitly judges the uncertainty of the estimated accuracy, providing upper and lower bounds for the estimated accuracy. besides, previous works only consider image classification and natural language inference tasks. our work shows that for sequence generation tasks like semantic parsing, the predicted sequence can serve as a goodenough feature to determine the prediction correctness on ood data. quality estimation in nlp tasks. automatic accuracy prediction has also been investigated for nlp tasks: van asch and daelemans (2010) exploit similarity metrics between the training and test set to estimate pos tagger performances; chatterjee et al. (2018) train regressors to predict bleu @xcite scores of a machine translation system with given features; @xcite train regressors to predict f1 scores for subtasks of amr @xcite . compared to these works, our method does not require manually designed features and thus is easy to be adapted to any sequence generation tasks. @xcite also train a correctness discriminator to improve the coverage of a selective prediction system for question answering. in contrast to this work, we use discriminators to predict accuracies, and more specifically upper and lower bounds. ## correctness discriminator the core of our approach is to construct and train a correctness discriminator model, which judges the correctness of a model prediction on unseen data. in this section, we first introduce how we design the discriminator model and collect training data in: a butterfly grew emma . | out: grow ( @xmath0, @xmath1: a butterfly grew emma . | out: grow ( @xmath2, @xmath3: what swedish actor founded m1 | out: select distinct ?x0 where { ?x0 a film.actor . ?x0 organizations_founded m1 } in: what swedish actor founded m1 | out: select distinct ?x0 where { ?x0 a film.actor . ?x0 organizations_founded m1 . ?x0 people.person.nationality m_0d0vqn } cfq incorrect correct figure 2: examples of cogs and cfq training data for the discriminator. in refers to the input sentence and out refers to the predicted output sequence (e.g. logical form for cogs and sparql query for cfq). ## experiments we introduce our datasets, model setup, evaluation metrics and experimental results in this section. ## discussion low performance on constituency parsing. our method predicts loose bounds on ptb parsing tasks and sometimes yields high ae scores. we conjecture that this is because the ptb training set contains many long output sequences (e.g. linearized parse trees), whose lengths are much larger than the maximal encoding length (e.g. 512) of our language model discriminators. encoding sequences longer than observed during pretraining has been shown challenging for transformer-based language models @xcite , which leads to an additional challenge for our discriminators. nonetheless, the gold accuracy is still robustly between the predicted bounds. the robustness of discriminators. we have seen that our predicted upper and lower bounds can capture the gold accuracy. however, this may not be enough to show the robustness of our method, since we only evaluated it on one overall test set for each parser, while previous works @xcite collect multiple test sets for each classifier. to investigate the robustness of our method, we create multiple test sets by randomly sampling subsets from the original test set and plot the accuracy of t5 discriminators on ood test sets in figure 4 . see appendix d for full results of both t5 and roberta discriminators. according to the results, we can observe that our predicted bounds robustly capture the gold accuracy with regard to different sizes of randomly sampled test sets. on cogs, pos-cogs and top, a small test set gives a large confidence interval. we consider this is because their test sets contain some extremely difficult examples for the parser, which could result in a challenging subset and yield a low accuracy. despite this, our discriminators capture the difficulty of such challenging subsets and shares similar confidence intervals as the gold accuracy. training models to predict another model's accuracy. the main idea of our method is to train one model (i.e. discriminator) to predict a base model's accuracy (i.e. parser). previous work estimates model accuracy using the parser's confidence scores, which can be misleading in ood tasks due to overconfidence, as shown in section 4.4. on the other hand, @xcite find that despite the parser's poor performance on ood data, categorizing the parsing task into a classification task enables their classifier to generalize to ood data to some extents. hence, we directly train a classifier (i.e. discriminator) to assess correctness of the prediction. training discriminators to evaluate another model's prediction has been studied in various fields, including adversarial learning @xcite , error detection @xcite and reranking @xcite , where discriminators are used to improve the base model's accuracy. our work differs in using discriminators to predict bounds of the base model's accuracy. despite the impressive performance of our discriminators, training them relies on incorrect parser outputs (i.e. errors) on the parser's training set. this raises concerns about the generalization of discriminators to errors made in ood data by the parser. our results empirically show that discriminators can generalize to such errors, but further investigation is needed to understand the specific scenarios where this method fails and the underlying reasons. since the discriminators' performance still lags behind on particular tasks, it is worth exploring discriminators' confidence scores to improve the estimated accuracy. we leave this for future study. ## conclusion we propose a method to predict upper and lower bounds for the accuracy of a model on unlabeled and possibly ood data. to do this, we first train multiple correctness discriminators by finetuning pretrained language models, and then ensemble discriminator predictions through a special voting mechanism. our experiments show that our predicted bounds reliably capture gold accuracy across a variety of in-distribution and out-of-distribution tasks including semantic parsing, tagging and constituency parsing tasks, and the upper and lower bounds are usually tight. although our method is not specifically designed for point estimation, simple heuristics (e.g. using the mean of bounds as estimated accuracy) based on our method can substantially outperform previous methods, which indicates the effectiveness of our method. for the future, we will explore the use of our discriminators to improve model performance on tasks evaluated in this paper. for example, given unlabeled ood sentences and a parser, our lower bound can be used to detect instances with a high correct-precision, as training data to improve the parser. it would be interesting to expand our predic-tion of hard upper and lower bounds to a bayesian model that predicts a probability distribution over accuracies. finally, it would be useful to extend our method to predicting accuracy in terms of other metrics (e.g. parsing f-score or smatch).