Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training  Ensembles

When a deep learning model is deployed in the wild , it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance . For safe deployment , it is essential to estimate the accuracy of the pre-trained model on the test data . However , the labels for the test inputs are usually not immediately available in practice , and obtaining them can be expensive . This observation leads to two challenging tasks : ( 1 ) unsupervised accuracy estimation , which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs ; ( 2 ) error detection , which aims to identify mis-classified test inputs . In this paper , we propose a principled and practically effective framework that simultaneously addresses the two tasks . The proposed framework iteratively learns an ensemble of models to identify mis-classified data points and performs self-training to improve the ensemble with the identified points . Theoretical analysis demonstrates that our framework enjoys provable guarantees for both accuracy estimation and error detection under mild conditions readily satisfied by practical deep learning models . Along with the framework , we proposed and experimented with two instantiations and achieved state-of-the-art results on 59 tasks . For example , on iWildCam , one instantiation reduces the estimation error for unsupervised accuracy estimation by at least 70 % and improves the F1 score for error detection by at least 4.7 % compared to existing methods . ∗Part of the work done while interning at Google . Our code is available at : https : //github.com/jfc43/self-training-ensembles . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1 Introduction . Data distribution in the real world may be wildly different from the training dataset for various reasons such as covariate shift due to domain divergence , corruption of images due to weather conditions , or out-of-distribution test inputs . When facing these issues , a deployed deep learning model can have unexpected performance drop on the test data . This performance degradation can be mitigated by test data annotation , which may be costly . Hence , estimating the accuracy of a pre-trained model on the unlabeled test data provides an alternative to avoid the cost when it is not necessary . Furthermore , it is beneficial to estimate the correctness of the predictions on individual points . This leads to an even more challenging task of error detection , which aims to identify points in the unlabeled test set that are mis-classified by the pre-trained model . Such a finer-grained estimation can facilitate a further improvement of the pre-trained model ( e.g. , manually label those mis-classified data points and retrain the model on them ) . While there have been previous attempts to address accuracy estimation or the broader problem of error detection , their successes usually rely on some conditions or assumptions that may not hold in practice . For example , a natural approach is to use confidence based metrics to measure the performance of the pre-trained model , e.g. , as in [ 9 ] . If the model is well-calibrated on the test data , then the average confidence approximates its accuracy . However , it has been observed that many machine learning systems , in particular modern neural networks , are poorly calibrated , especially on test data with distribution shift [ 14 , 30 ] . Another method is to learn a regression function that takes statistics about the model and the test data as input , and predicts the performance on the test data [ 9 , 35 ] . This requires training on labeled data from various data distributions , which is very expensive or even impractical . Furthermore , the performance predictor trained on the labeled data may not generalize to unknown data distributions . Recent work by [ 4 ] proposes to learn a “ check ” model using domain-invariant representation and use it as a proxy for the unknown true test labels to estimate the performance via error detection . It relies on the success of the domain-invariant representation methods to obtain a highly accurate check model on the test data . Hence , the check model performance suffers when domain-invariant representation is not accurate in circumstances such as test data having outlier feature vectors or different class probabilities than the training data . In this paper , we propose a principled and practically effective framework for the challenging tasks of accuracy estimation and error detection ( Section 4 ) . The framework makes a novel use of the self-training technique on ensembles for these tasks . It first learns an ensemble of models to identify some mis-classified points . Then it assigns pseudo-labels to these points and uses self-training with these pseudo-labeled data to identify more mis-classified points . We also provide provable guarantees for the framework ( Section 5 ) . Our analysis shows that it provably outputs an accurate estimate of the accuracy as well as the mis-classified points , under mild practical conditions : the ensemble make small errors on the test inputs correctly classified by the model f , mostly disagree with f on the current identified mis-classified inputs , and are correct or have diverse predictions on the remaining inputs . These conditions can be readily satisfied by deep learning model ensembles . Furthermore , they have no explicit assumptions on the test distribution and thus the framework can be instantiated by incorporating properly designed ensemble methods for different settings ( Section 6 ) . Experimental results on 59 tasks over five dataset categories including image classification and sentiment classification datasets show that our method achieves state-of-the-art on both accuracy estimation and error detection ( Section 7 ) . The experiments also provide positive support for our analysis , verifying the conditions and implications . 2 Related Work . Confidence Estimation and Error Detection . Recently , estimating model confidence has been an important area of research because of the perceived relationship between model uncertainty and trusting its predictions [ 28 ] . However , modern neural networks have been observed to be poorly calibrated ( e.g . they may make wrong predictions with very high confidence ) [ 14 ] , especially on distributions with dataset shift [ 30 ] . Many approaches have been proposed to address this issue , such as Temperature Scaling [ 14 ] , Monte-Carlo Dropout [ 11 ] and Deep Ensemble [ 25 ] , but the challenge still remains . A similar challenge appears in error detection , where the goal is to identify erroneous predictions given a test set . Combining these two problems , some early work uses confidence estimates to detect incorrect predictions . For example , [ 17 ] proposed to use maximum softmax probability to detect misclassified examples . [ 5 ] proposed True Class Probability for failure prediction . [ 21 ] proposed to use Trust Score to estimate the confidence in model predictions . These methods require a robust estimate of confidence , and an empirically chosen threshold that is mostly problem- and model-dependent , to identify error data points . Recently , [ 4 ] proposed to use a check model to predict mis-classification . Unsupervised Accuracy Estimation . The problem of unsupervised accuracy ( or risk ) estimation has received relatively scant attention from the research community . [ 8 ] offered a solution with certain assumptions on the marginal output distribution p ( y ) . [ 32 ] proposed to estimate the accuracies of the approximations to some target Boolean functions based on the agreement rates method . A follow-up work by [ 31 ] also considered the “ multiple approximations ” problem setting where the target classes may be tied together through logical constraints , and proposed an efficient method to estimate the accuracy of classifiers using only unlabeled data . [ 20 ] proposed a spectral-based approach to estimate the accuracies of multiple classifiers , mainly in the binary case and with some assumptions on the classifiers . [ 36 ] proposed a method to estimate the model ’ s error on distributions different from the training distribution by assuming a conditional independence structure of the data . These problem settings are different from ours and the constraints may not be satisfied by the data and model we consider . Recently , [ 9 ] studied three families of methods ( H-divergence , reverse classification accuracy and confidence measures ) and demonstrated how they could be used to predict the performance drop . [ 35 ] also proposed to learn a performance predictor on the statistics of model outputs with an assumption that they know the typical cases of dataset shift in advance . Similarly , [ 7 ] proposed to train regression models on the feature statistics of the datasets sampled from a meta-dataset to predict model performance and [ 6 ] proposed to utilize linear regression to estimate classifier performance from the accuracy of rotation prediction . [ 13 ] proposed to use difference of confidences to estimate classifier performance . [ 3 ] proposed MANDOLINE that utilizes userspecified slicing functions to improve the importance of weighting to make the accuracy estimation more accurate . With the most similar setup to ours , [ 4 ] used a set of domain-invariant predictors as a proxy for the unknown target labels to estimate a given model ’ s performance under distribution shift . In a concurrent work , [ 22 ] empirically showed that the test accuracy of deep networks can be estimated by measuring disagreement rate between a pair of models independently trained via Stochastic Gradient Descent ( SGD ) and theoretically related this phenomenon to the well-calibrated nature of ensembles of SGD-trained models . 3 Problem Statement . Consider the classification problem with sample spaceX and label spaceY . LetD be a set of labeled data from the training distribution PX , Y . LetU = { ( x , yx ) } be a set of labeled data from the test distribution QX , Y , where yx is the label for x . Let UX = { x : ( x , yx ) ∈ U } be the set of input feature vectors in U . Define the accuracy of a model f on U as : acc ( f , U ) : = 1|U | ∑ ( x , yx ) ∈U I [ f ( x ) = yx ] , where |U | is the cardinality of the set U . When clear from the context , we will omit U and simply write acc ( f ) . Given a model f : X → Y trained on D , together with D and UX , the goal of unsupervised accuracy estimation is to get an estimate ˆacc so that the absolute estimation error | ˆacc− acc ( f ) | is small . We assume access to the training data D to help in estimating acc ( f ) . Though acc ( f ) is usually sufficient to provide an estimate of whether the model could perform well on U , it would be beneficial if the algorithm could also provide an estimate of the correctness of individual points so that we can know where to improve the model f . Thus , we also consider a more challenging task of identifying the mis-classified points in U : WX : = { x : ( x , yx ) ∈ U , f ( x ) 6= yx } . Given f , D and UX , the goal of error detection is to identify a subset RX ⊆ UX , such that |WX4RX | is small , where WX4RX : = ( WX \RX ) ∪ ( RX \WX ) .

Summary: - The paper considers two problems: unsupervised accuracy estimation and error detection, both in out-of-distribution settings. - The authors propose a practical approach, where they train a ‘check model’ h using self-training and ensembling. Points on which the prediction model and the check model disagree on are predicted as misclassified. - The theoretical analysis identifies conditions under which the approach should work and presents a bound on the accuracy estimation error - The authors empirically show that their algorithm outperforms baselines for both unsupervised accuracy estimation and error detection