Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training  Ensembles

When a deep learning model is deployed in the wild , it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance . For safe deployment , it is essential to estimate the accuracy of the pre-trained model on the test data . However , the labels for the test inputs are usually not immediately available in practice , and obtaining them can be expensive . This observation leads to two challenging tasks : ( 1 ) unsupervised accuracy estimation , which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs ; ( 2 ) error detection , which aims to identify mis-classified test inputs . In this paper , we propose a principled and practically effective framework that simultaneously addresses the two tasks . The proposed framework iteratively learns an ensemble of models to identify mis-classified data points and performs self-training to improve the ensemble with the identified points . Theoretical analysis demonstrates that our framework enjoys provable guarantees for both accuracy estimation and error detection under mild conditions readily satisfied by practical deep learning models . Along with the framework , we proposed and experimented with two instantiations and achieved state-of-the-art results on 59 tasks . For example , on iWildCam , one instantiation reduces the estimation error for unsupervised accuracy estimation by at least 70 % and improves the F1 score for error detection by at least 4.7 % compared to existing methods . ∗Part of the work done while interning at Google . Our code is available at : https : //github.com/jfc43/self-training-ensembles . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1 Introduction . Data distribution in the real world may be wildly different from the training dataset for various reasons such as covariate shift due to domain divergence , corruption of images due to weather conditions , or out-of-distribution test inputs . When facing these issues , a deployed deep learning model can have unexpected performance drop on the test data . This performance degradation can be mitigated by test data annotation , which may be costly . Hence , estimating the accuracy of a pre-trained model on the unlabeled test data provides an alternative to avoid the cost when it is not necessary . Furthermore , it is beneficial to estimate the correctness of the predictions on individual points . This leads to an even more challenging task of error detection , which aims to identify points in the unlabeled test set that are mis-classified by the pre-trained model . Such a finer-grained estimation can facilitate a further improvement of the pre-trained model ( e.g. , manually label those mis-classified data points and retrain the model on them ) . While there have been previous attempts to address accuracy estimation or the broader problem of error detection , their successes usually rely on some conditions or assumptions that may not hold in practice . For example , a natural approach is to use confidence based metrics to measure the performance of the pre-trained model , e.g. , as in [ 9 ] . If the model is well-calibrated on the test data , then the average confidence approximates its accuracy . However , it has been observed that many machine learning systems , in particular modern neural networks , are poorly calibrated , especially on test data with distribution shift [ 14 , 30 ] . Another method is to learn a regression function that takes statistics about the model and the test data as input , and predicts the performance on the test data [ 9 , 35 ] . This requires training on labeled data from various data distributions , which is very expensive or even impractical . Furthermore , the performance predictor trained on the labeled data may not generalize to unknown data distributions . Recent work by [ 4 ] proposes to learn a “ check ” model using domain-invariant representation and use it as a proxy for the unknown true test labels to estimate the performance via error detection . It relies on the success of the domain-invariant representation methods to obtain a highly accurate check model on the test data . Hence , the check model performance suffers when domain-invariant representation is not accurate in circumstances such as test data having outlier feature vectors or different class probabilities than the training data . In this paper , we propose a principled and practically effective framework for the challenging tasks of accuracy estimation and error detection ( Section 4 ) . The framework makes a novel use of the self-training technique on ensembles for these tasks . It first learns an ensemble of models to identify some mis-classified points . Then it assigns pseudo-labels to these points and uses self-training with these pseudo-labeled data to identify more mis-classified points . We also provide provable guarantees for the framework ( Section 5 ) . Our analysis shows that it provably outputs an accurate estimate of the accuracy as well as the mis-classified points , under mild practical conditions : the ensemble make small errors on the test inputs correctly classified by the model f , mostly disagree with f on the current identified mis-classified inputs , and are correct or have diverse predictions on the remaining inputs . These conditions can be readily satisfied by deep learning model ensembles . Furthermore , they have no explicit assumptions on the test distribution and thus the framework can be instantiated by incorporating properly designed ensemble methods for different settings ( Section 6 ) . Experimental results on 59 tasks over five dataset categories including image classification and sentiment classification datasets show that our method achieves state-of-the-art on both accuracy estimation and error detection ( Section 7 ) . The experiments also provide positive support for our analysis , verifying the conditions and implications . 2 Related Work . Confidence Estimation and Error Detection . Recently , estimating model confidence has been an important area of research because of the perceived relationship between model uncertainty and trusting its predictions [ 28 ] . However , modern neural networks have been observed to be poorly calibrated ( e.g . they may make wrong predictions with very high confidence ) [ 14 ] , especially on distributions with dataset shift [ 30 ] . Many approaches have been proposed to address this issue , such as Temperature Scaling [ 14 ] , Monte-Carlo Dropout [ 11 ] and Deep Ensemble [ 25 ] , but the challenge still remains . A similar challenge appears in error detection , where the goal is to identify erroneous predictions given a test set . Combining these two problems , some early work uses confidence estimates to detect incorrect predictions . For example , [ 17 ] proposed to use maximum softmax probability to detect misclassified examples . [ 5 ] proposed True Class Probability for failure prediction . [ 21 ] proposed to use Trust Score to estimate the confidence in model predictions . These methods require a robust estimate of confidence , and an empirically chosen threshold that is mostly problem- and model-dependent , to identify error data points . Recently , [ 4 ] proposed to use a check model to predict mis-classification . Unsupervised Accuracy Estimation . The problem of unsupervised accuracy ( or risk ) estimation has received relatively scant attention from the research community . [ 8 ] offered a solution with certain assumptions on the marginal output distribution p ( y ) . [ 32 ] proposed to estimate the accuracies of the approximations to some target Boolean functions based on the agreement rates method . A follow-up work by [ 31 ] also considered the “ multiple approximations ” problem setting where the target classes may be tied together through logical constraints , and proposed an efficient method to estimate the accuracy of classifiers using only unlabeled data . [ 20 ] proposed a spectral-based approach to estimate the accuracies of multiple classifiers , mainly in the binary case and with some assumptions on the classifiers . [ 36 ] proposed a method to estimate the model ’ s error on distributions different from the training distribution by assuming a conditional independence structure of the data . These problem settings are different from ours and the constraints may not be satisfied by the data and model we consider . Recently , [ 9 ] studied three families of methods ( H-divergence , reverse classification accuracy and confidence measures ) and demonstrated how they could be used to predict the performance drop . [ 35 ] also proposed to learn a performance predictor on the statistics of model outputs with an assumption that they know the typical cases of dataset shift in advance . Similarly , [ 7 ] proposed to train regression models on the feature statistics of the datasets sampled from a meta-dataset to predict model performance and [ 6 ] proposed to utilize linear regression to estimate classifier performance from the accuracy of rotation prediction . [ 13 ] proposed to use difference of confidences to estimate classifier performance . [ 3 ] proposed MANDOLINE that utilizes userspecified slicing functions to improve the importance of weighting to make the accuracy estimation more accurate . With the most similar setup to ours , [ 4 ] used a set of domain-invariant predictors as a proxy for the unknown target labels to estimate a given model ’ s performance under distribution shift . In a concurrent work , [ 22 ] empirically showed that the test accuracy of deep networks can be estimated by measuring disagreement rate between a pair of models independently trained via Stochastic Gradient Descent ( SGD ) and theoretically related this phenomenon to the well-calibrated nature of ensembles of SGD-trained models . 3 Problem Statement . Consider the classification problem with sample spaceX and label spaceY . LetD be a set of labeled data from the training distribution PX , Y . LetU = { ( x , yx ) } be a set of labeled data from the test distribution QX , Y , where yx is the label for x . Let UX = { x : ( x , yx ) ∈ U } be the set of input feature vectors in U . Define the accuracy of a model f on U as : acc ( f , U ) : = 1|U | ∑ ( x , yx ) ∈U I [ f ( x ) = yx ] , where |U | is the cardinality of the set U . When clear from the context , we will omit U and simply write acc ( f ) . Given a model f : X → Y trained on D , together with D and UX , the goal of unsupervised accuracy estimation is to get an estimate ˆacc so that the absolute estimation error | ˆacc− acc ( f ) | is small . We assume access to the training data D to help in estimating acc ( f ) . Though acc ( f ) is usually sufficient to provide an estimate of whether the model could perform well on U , it would be beneficial if the algorithm could also provide an estimate of the correctness of individual points so that we can know where to improve the model f . Thus , we also consider a more challenging task of identifying the mis-classified points in U : WX : = { x : ( x , yx ) ∈ U , f ( x ) 6= yx } . Given f , D and UX , the goal of error detection is to identify a subset RX ⊆ UX , such that |WX4RX | is small , where WX4RX : = ( WX \RX ) ∪ ( RX \WX ) .

Summary: The paper studies the problem of error detection and accuracy estimation of unlabeled data points. Particularly, the paper proposes to use a diverse ensemble of models and detect errors by checking the inconsistency between the ensemble prediction and the target model prediction. Moreover, the paper proposes a self-training procedure that iteratively identifies incorrect predictions and adds their pseudo labels as new training targets to detect more errors. The paper also gives some theoretical analysis given assumptions about the ensemble models' quality.