Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
Year: 2021

Abstract:
When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance. For safe deployment, it is essential to estimate the accuracy of the pre-trained model on the test data. However, the labels for the test inputs are usually not immediately available in practice, and obtaining them can be expensive. This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs. In this paper, we propose a principled and practically effective framework that simultaneously addresses the two tasks. The proposed framework iteratively learns an ensemble of models to identify mis-classified data points and performs self-training to improve the ensemble with the identified points. Theoretical analysis demonstrates that our framework enjoys provable guarantees for both accuracy estimation and error detection under mild conditions readily satisfied by practical deep learning models. Along with the framework, we proposed and experimented with two instantiations and achieved state-of-the-art results on 59 tasks. For example, on iWildCam, one instantiation reduces the estimation error for unsupervised accuracy estimation by at least 70% and improves the F1 score for error detection by at least 4.7% compared to existing methods. ∗Part of the work done while interning at Google. Our code is available at: https://github.com/jfc43/self-training-ensembles. 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

text: When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance. For safe deployment, it is essential to estimate the accuracy of the pre-trained model on the test data. However, the labels for the test inputs are usually not immediately available in practice, and obtaining them can be expensive. This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs. In this paper, we propose a principled and practically effective framework that simultaneously addresses the two tasks. The proposed framework iteratively learns an ensemble of models to identify mis-classified data points and performs self-training to improve the ensemble with the identified points. Theoretical analysis demonstrates that our framework enjoys provable guarantees for both accuracy estimation and error detection under mild conditions readily satisfied by practical deep learning models. Along with the framework, we proposed and experimented with two instantiations and achieved state-of-the-art results on 59 tasks. For example, on iWildCam, one instantiation reduces the estimation error for unsupervised accuracy estimation by at least 70% and improves the F1 score for error detection by at least 4.7% compared to existing methods.
∗Part of the work done while interning at Google. Our code is available at: https://github.com/jfc43/self-training-ensembles.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).

heading: 1 Introduction
text: Data distribution in the real world may be wildly different from the training dataset for various reasons such as covariate shift due to domain divergence, corruption of images due to weather conditions, or out-of-distribution test inputs. When facing these issues, a deployed deep learning model can have unexpected performance drop on the test data. This performance degradation can be mitigated by test data annotation, which may be costly. Hence, estimating the accuracy of a pre-trained model on the unlabeled test data provides an alternative to avoid the cost when it is not necessary. Furthermore, it is beneficial to estimate the correctness of the predictions on individual points. This leads to an even more challenging task of error detection, which aims to identify points in the unlabeled test set that are mis-classified by the pre-trained model. Such a finer-grained estimation can facilitate a further improvement of the pre-trained model (e.g., manually label those mis-classified data points and retrain the model on them).
While there have been previous attempts to address accuracy estimation or the broader problem of error detection, their successes usually rely on some conditions or assumptions that may not hold in practice. For example, a natural approach is to use confidence based metrics to measure the performance of the pre-trained model, e.g., as in [9]. If the model is well-calibrated on the test data, then the average confidence approximates its accuracy. However, it has been observed that many machine learning systems, in particular modern neural networks, are poorly calibrated, especially on test data with distribution shift [14, 30]. Another method is to learn a regression function that takes statistics about the model and the test data as input, and predicts the performance on the test data [9, 35]. This requires training on labeled data from various data distributions, which is very expensive or even impractical. Furthermore, the performance predictor trained on the labeled data may not generalize to unknown data distributions. Recent work by [4] proposes to learn a “check” model using domain-invariant representation and use it as a proxy for the unknown true test labels to estimate the performance via error detection. It relies on the success of the domain-invariant representation methods to obtain a highly accurate check model on the test data. Hence, the check model performance suffers when domain-invariant representation is not accurate in circumstances such as test data having outlier feature vectors or different class probabilities than the training data.
In this paper, we propose a principled and practically effective framework for the challenging tasks of accuracy estimation and error detection (Section 4). The framework makes a novel use of the self-training technique on ensembles for these tasks. It first learns an ensemble of models to identify some mis-classified points. Then it assigns pseudo-labels to these points and uses self-training with these pseudo-labeled data to identify more mis-classified points. We also provide provable guarantees for the framework (Section 5). Our analysis shows that it provably outputs an accurate estimate of the accuracy as well as the mis-classified points, under mild practical conditions: the ensemble make small errors on the test inputs correctly classified by the model f , mostly disagree with f on the current identified mis-classified inputs, and are correct or have diverse predictions on the remaining inputs. These conditions can be readily satisfied by deep learning model ensembles. Furthermore, they have no explicit assumptions on the test distribution and thus the framework can be instantiated by incorporating properly designed ensemble methods for different settings (Section 6). Experimental results on 59 tasks over five dataset categories including image classification and sentiment classification datasets show that our method achieves state-of-the-art on both accuracy estimation and error detection (Section 7). The experiments also provide positive support for our analysis, verifying the conditions and implications.

heading: 2 Related Work
text: Confidence Estimation and Error Detection. Recently, estimating model confidence has been an important area of research because of the perceived relationship between model uncertainty and trusting its predictions [28]. However, modern neural networks have been observed to be poorly calibrated (e.g. they may make wrong predictions with very high confidence) [14], especially on distributions with dataset shift [30]. Many approaches have been proposed to address this issue, such as Temperature Scaling [14], Monte-Carlo Dropout [11] and Deep Ensemble [25], but the challenge still remains. A similar challenge appears in error detection, where the goal is to identify erroneous predictions given a test set. Combining these two problems, some early work uses confidence estimates to detect incorrect predictions. For example, [17] proposed to use maximum
softmax probability to detect misclassified examples. [5] proposed True Class Probability for failure prediction. [21] proposed to use Trust Score to estimate the confidence in model predictions. These methods require a robust estimate of confidence, and an empirically chosen threshold that is mostly problem- and model-dependent, to identify error data points. Recently, [4] proposed to use a check model to predict mis-classification.
Unsupervised Accuracy Estimation. The problem of unsupervised accuracy (or risk) estimation has received relatively scant attention from the research community. [8] offered a solution with certain assumptions on the marginal output distribution p(y). [32] proposed to estimate the accuracies of the approximations to some target Boolean functions based on the agreement rates method. A follow-up work by [31] also considered the “multiple approximations” problem setting where the target classes may be tied together through logical constraints, and proposed an efficient method to estimate the accuracy of classifiers using only unlabeled data. [20] proposed a spectral-based approach to estimate the accuracies of multiple classifiers, mainly in the binary case and with some assumptions on the classifiers. [36] proposed a method to estimate the model’s error on distributions different from the training distribution by assuming a conditional independence structure of the data. These problem settings are different from ours and the constraints may not be satisfied by the data and model we consider. Recently, [9] studied three families of methods (H-divergence, reverse classification accuracy and confidence measures) and demonstrated how they could be used to predict the performance drop. [35] also proposed to learn a performance predictor on the statistics of model outputs with an assumption that they know the typical cases of dataset shift in advance. Similarly, [7] proposed to train regression models on the feature statistics of the datasets sampled from a meta-dataset to predict model performance and [6] proposed to utilize linear regression to estimate classifier performance from the accuracy of rotation prediction. [13] proposed to use difference of confidences to estimate classifier performance. [3] proposed MANDOLINE that utilizes userspecified slicing functions to improve the importance of weighting to make the accuracy estimation more accurate. With the most similar setup to ours, [4] used a set of domain-invariant predictors as a proxy for the unknown target labels to estimate a given model’s performance under distribution shift. In a concurrent work, [22] empirically showed that the test accuracy of deep networks can be estimated by measuring disagreement rate between a pair of models independently trained via Stochastic Gradient Descent (SGD) and theoretically related this phenomenon to the well-calibrated nature of ensembles of SGD-trained models.

heading: 3 Problem Statement
text: Consider the classification problem with sample spaceX and label spaceY . LetD be a set of labeled data from the training distribution PX,Y . LetU = {(x, yx)} be a set of labeled data from the test distribution QX,Y , where yx is the label for x. Let UX = {x : (x, yx) ∈ U} be the set of input feature vectors in U . Define the accuracy of a model f on U as: acc(f, U) := 1|U | ∑ (x,yx)∈U I[f(x) = yx], where |U | is the cardinality of the set U . When clear from the context, we will omit U and simply write acc(f). Given a model f : X → Y trained on D, together with D and UX , the goal of unsupervised accuracy estimation is to get an estimate ˆacc so that the absolute estimation error | ˆacc− acc(f)| is small. We assume access to the training data D to help in estimating acc(f). Though acc(f) is usually sufficient to provide an estimate of whether the model could perform well on U , it would be beneficial if the algorithm could also provide an estimate of the correctness of individual points so that we can know where to improve the model f . Thus, we also consider a more challenging task of identifying the mis-classified points in U : WX := {x : (x, yx) ∈ U, f(x) 6= yx}. Given f , D and UX , the goal of error detection is to identify a subset RX ⊆ UX , such that |WX4RX | is small, where WX4RX := (WX \RX) ∪ (RX \WX).

heading: 4 Algorithmic Framework via Self-Training Ensembles
text: Intuition. We consider the approach of learning a “check model” h and using the disagreement between h and the pre-trained model f for our tasks. The fundamental idea is to identify a point x as mis-classified if h disagrees with f on x. To derive our method, we note a simple fact: the disagreement approach succeeds if (1) h agrees with f on points where f is correct and (2) h disagrees with f on points where f is incorrect. Our first key observation is that usually (1) can
Framework 1 Error Detection and Unsupervised Accuracy Estimation via Self-Training Ensembles Input: A training dataset D, an unlabeled test dataset UX , a pre-trained model f , an ensemble
learning method T , and a hyper-parameter τ 1: Initialize R = ∅ 2: for t = 1, 2, · · · , T do 3: Use T to generate an ensemble T (D,UX , R). 4: Identify those points on which the ensemble and f disagree as mis-classified points:
RX := {x ∈ UX : Pr h∼T {h(x) = f(x)} < τ}.
5: For each x ∈ RX , assign a pseudo-label ỹx 6= f(x) (e.g., using the majority vote of the ensemble or a random label). And set R = {(x, ỹx) : x ∈ RX}.
6: end for Output: The estimated mis-classified points RX , and the estimated accuracy |UX\RX ||UX | .
be satisfied approximately in practice. Intuitively, this is because h and f use the same training data, and h can be trained to be correct on the subset of the instance space where f is correct. However, (2) may not be easily satisfied (e.g., h can make similar mistakes as f ), which leads to an overestimation of the accuracy. We thus focus on improving the disagreement on mis-classified points.
To disagree with f on a mis-classified test input x, we have two ways: (1) make the check model h correct on x; (2) diverse ensembles. The first way may be achievable when the training data contains information for prediction on x. A prototypical example is when the test inputs are corruption of clean data from the training distribution (e.g., the training data are images in sunny days while the test inputs are ones in rainy days), and techniques like unsupervised domain adaptation can be used to improve the prediction on such test inputs. However, correct predictions on x may not be feasible in many interesting scenarios due to insufficient information (e.g., the test image in the open world can contain an object that is never seen in the training data ). Fortunately, it has been shown that for such inputs, one can obtain an ensemble of models with diverse predictions (e.g.,[25]). This then gives the second way to achieve disagreement: using diverse ensembles. Therefore, our method will learn an ensemble of models (instead of one check model) and identify a point x as mis-classified if the ensemble disagree with f on x (i.e., a large fraction of the models in the ensemble disagree with f on their predictions on x). A mis-classified test input, x, will be successfully identified, if a majority of the ensemble models predict correctly, or they have large diversity on x.
However, the ensemble may only be able to identify a subset of the mis-classified points. Therefore, we propose to iteratively identify more and more mis-classified points by self-training. For each mis-classified data point x identified by the ensemble, we assign it a pseudo-label that is different from f(x) (e.g. use the majority vote of the ensemble or a random label as the pseudo-label). Then we can train (with regularization) a new ensemble to encourage their disagreement with f on the pseudo-labeled data R (e.g., use a supervised loss on R with a small weight as the regularization). Note that the pseudo-labels may not all be correct, and we do not need the new ensemble to exactly fit the pseudo-labels. We only need the new ensemble to mostly disagree with f on R so that they still identify R as mis-classified points. Furthermore, self-training can help the ensemble identify more mis-classified points. When some pseudo-labels are correct, we observe that these additional data can help the new ensemble become more accurate and thus help with identifying more mis-classified points. We also observe that the new ensemble will be less diverse on the pseudo-labeled data and thus have diversity on the remaining test inputs. We report empirical results in Appendix C.4 to support these observations.
Our framework. We assume access to an ensemble learning method T (specific instantiations will be given later) that takes as input a training set D, an unlabeled test set UX , and a pseudo-labeled set R (and potentially some other parameters) and outputs an ensemble, which is a distribution over functions h : X → Y .2 We denote the generated ensemble as T (D,UX , R) and write h ∼ T (D,UX , R) for sampling h from the ensemble, or simply h ∼ T when clear from context.
2This includes one single h as a special case. It also includes the case of h with probabilistic outputs, i.e., h(x) = [h1(x), . . . , hK(x)] for Y = {1, 2, . . . ,K}, where hi(x) is the predicted probability of x from class i. One can think of such an h as an ensemble T where Prg∼T [g(x) = i] = hi(x).
We are now ready to describe our method (Framework 1). Our framework begins with an empty set of pseudo-labeled data R, and executes in T iterations. In each iteration, it generates an ensemble T (D,UX , R) and then uses the ensemble to construct a new pseudo-labeled dataset R for the next iteration: let RX be those points x ∈ UX where the agreement rate between the ensemble and f is below a threshold, assign a pseudo-label ỹx different from f(x), and let R be the set of these pseudo-labeled data. Finally, it outputs RX as the mis-classified points and outputs the fraction of points outside RX as the accuracy.
Some existing work also use models’ disagreement to estimate the error on the unlabeled data in different ways. For example, [32] consider a set of pre-trained models and aim to estimate the error for each model; [4] learn a single check model and compare it with f . On the other hand, our novelty is using ensembles to identify a set of mis-classified points and further using self-training for learning the ensembles. Note that our self-training ensembles is different from standard selftraining and ensemble: standard self-training+ensemble aims to get accurate predictions while ours is to increase disagreement on mis-classified points (refer to Appendix A.2 for a detailed discussion).

heading: 5 Theoretical Analysis
text: As described in the intuition above, our framework succeeds if in each iteration, the ensemble satisfy the following conditions: (A) correct on UX \WX , (B) mostly disagree with f onRX , and (C) either correct or diverse on WX \ RX . This section first introduces notions formalizing these conditions and then provides provable guarantees under these conditions. Here we focus on intuitions and provide the proofs and more discussion in Appendix B.
Notations. Recall that the ensemble method T outputs a distribution over models, and we write h ∼ T for sampling a model h from this distribution. Let Eh denote the expectation over this output distribution, and Ex denote the average over x ∈ UX or (x, yx) ∈ U . Formalizing Condition (A). A key observation we make in practice is that usually on points where f is correct, the ensemble models are also correct. Intuitively, this is because the only labeled information for learning the ensemble is D, while D is also used for learning f . For illustration, suppose f and the ensemble have zero training errors on D. When they are from hypothesis classes of bounded VC-dimensions and the training set is large enough compared to the VC-dimensions, standard error bounds show that the ensemble will have small errors on the subset of the feature space {x ∈ X : f(x) = yx}, i.e., it will have small errors on points where f is correct. To formalize this observation, we introduce the following notion.
Definition 1 (Error on Correct Points). Let ν denote the average probability of h ∼ T making error on test inputs where the model f is correct: ν := Pr(x,yx)∼U,h∼T [h(x) 6= yx | f(x) = yx].
Formalizing Condition (B). We assume the new ensemble trained with regularization on R will mostly disagree with f on RX . We define the following notion:
Definition 2 (Agreement on Pseudo-Labeled Data). Let γ denote the average probability of agreement between h ∼ T and f on RX : γ := Prx∼RX ,h∼T {h(x) = f(x)}.
Formalizing Condition (C). Let GX denote the good points in WX \ RX on which the ensemble will have correct predictions with high confidence, BX the remaining bad points. Formally, we define: GX := {x ∈WX \RX : Prh∼T {h(x) = yx} ≥ 1− ν} and BX := WX \ (RX ∪GX). (Here we choose the confidence level 1− ν for convenience, where ν is the error on correct points. Other sufficiently high confidence levels can also be used with slight change to the analysis.) We would like the ensemble to have large diversity on BX .
Definition 3 (Diversity of Ensemble). Let σ2 denote the average probability of disagreement between two ensemble models on BX : σ2 := Ex[σ2x|x ∈ BX ], where σ2x := Prh1,h2∼T [h1(x) 6= h2(x)].
Provable Guarantees. Based on the above notions we obtain our guarantees: Theorem 1. Assume in each iteration of the framework, τ = √ 1− η for some η ∈ (0, 3Bη/4) whereBη := min{σ2, 1−ν2}. Let σ2L > 0 be a lower bound on the diversity σ2, γ̃ > 0 be an upper bound on γ, and ν̃ be an upper bound on ν over all iterations. Then for any δ ∈ (0, σ2L/4), after at
Algorithm 1 Error Detection and Unsupervised Accuracy Estimation via Self-Training Ensembles Input: D, UX , f , ensemble method T , parameters T , N
1: Initialize R = ∅ 2: for t = 1, 2, · · · , T do 3: Set {hi}Ni=1 = T (D,UX , R,N, other parameters) 4: Let ỹx be the majority vote of {hi}Ni=1: ỹx := arg maxj∈Y 1N ∑N i=1 I[hi(x) = j]. 5: Set R = {(x, ỹx) : x ∈ UX , ỹx 6= f(x)}. 6: end for
Output: Estimated mis-classified points RX = {x : (x, y) ∈ R}, estimated accuracy |UX\RX ||UX | .
most d1/δe iterations, we can get |UX\RX ||UX | approximates the accuracy acc(f) andRX approximates the mis-classified points WX as follows:∣∣∣∣acc(f)− |UX \RX ||UX | ∣∣∣∣ ≤ max{ ν̃1− τ (1− ef ), ef}, where := γ̃ τ ( 1 + ν̃1−τ 1−ef ef ) σ2L 4 − δ + γ̃ τ , (1)
|WX4RX | ≤ ν̃
1− τ |UX \WX |+ |WX |. (2)
Proof Sketch. Let’s first consider one iteration, assuming small ν, γ and large σ2. Intuitively, on the correct points UX \WX , the ensemble have a small error ν and thus disagree with f on only a few such points. On the old RX , the ensemble mostly disagree with f(x); similarly on GX . On the remaining points BX , we show that if the ensemble have large diversity, then their predictions must have large variances on a significant subset of points and thus have large disagreement with f , facilitating the detection. Overall, our framework can construct a new pseudo-labeled set R′X that contains mostly mis-classified points and is also larger than the old RX (Lemma 1 in Appendix B). Therefore, each iteration can make some progress by identifying more mis-classified points than before, and enough iterations achieve the guarantees.
The theorem provides guarantees for general values of ν, γ and σ2. To get some intuition, note that typically ef < τ and suppose we set τ = 3/4 and δ = γ̃/τ , then the accuracy is estimated up to error max{ν̃, 16γ̃
3σ2L (ef+ν̃)}. When ν̃, γ̃ are small and σ2L is large, the error is small. Therefore, under
mild conditions, our framework can give a provable estimation of the accuracy and the mis-classified points up to small errors.
The theorem formalizes that the framework can succeed under mild conditions (A)(B)(C) on the ensembles, without explicit conditions on the data distributions and the pre-trained model (note that the conditions on them are implicitly captured in the mild conditions on the ensemble). The framework is thus flexible, and different ensemble methods satisfying these conditions can be incorporated to get concrete instantiations applicable to various settings. Indeed, the crux of our framework is then to design ensemble methods meeting the conditions. This turns out to be not difficult, in particular for deep learning pre-trained models and deep learning ensemble models. Even an ensemble of deep networks simply trained from random initialization works well. Two concrete instantiations are presented in Section 6 below and evaluated in Section 7.

heading: 6 Instantiations of the Framework
text: Based on our framework, we propose Algorithm 1 for accuracy estimation and error detection. It executes in T iterations. In each iteration, it trains an ensemble of models {hi}Ni=1 and then uses the ensemble to construct the pseudo-labeled data. Finally, it outputs RX as the set of mis-classified points and outputs the fraction of points outside RX as the accuracy. The algorithm uses an intuitive heuristic to implement the threshold on the agreement rate: it sets RX as the points x ∈ UX with f(x) different from the majority vote of the ensemble models. Empirically, we observe that this leads to similar RX and thus similar results as explicit thresholding, but is much more convenient (the empirical results to support this observation are reported in Appendix C.6).
Algorithm 2 TRI: Ensemble via Random Initialization Input: D, UX , R, N , parameter γ
1: Pre-train {h′i}Ni=1 on D from different random initialization 2: for i = 1, 2, . . . , N do 3: Learn hi by fine-tuning h′i for one epoch by:
minimize h
E(x,y)∈D[`(h(x), y)] + γ · E(x,y)∈R[`(h(x), y)] (3)
4: end for Output: The ensemble of models {hi}Ni=1
Algorithm 3 TRM: Ensemble via Representation Matching Input: D, UX , R, N , initial pre-trained model h0, parameters α, γ
// h0(x) = c(φ(x)) is a composition of a prediction function c and a representation function φ 1: Fine-tune h0 for N epochs using the objective:
minimize h
E(x,y)∈D[`(h(x), y)] + γ · E(x,y)∈R[`(h(x), y)] + α · d(pφD, p φ UX ) (4)
where d(pφD, p φ UX ) is the distance between the distribution of φ(x) on D and that on UX . 2: Use the N checkpoint models at the end of each training epoch as the model ensemble {hi}Ni=1.
Output: The ensemble of models {hi}Ni=1
While different ensemble methods T can be used in Algorithm 1, here we propose two concrete instantiations TRI and TRM based on the success conditions of our framework. Ensemble Method TRI. Algorithm 2 describes a natural method that trains the models from different random initialization. It first trains h′i on D (e.g., using the same training algorithm as for f ), to ensure that ensemble has small error on points where f is correct. It then fine-tunes h′i on D and R for one epoch to get hi that mostly disagrees with f on RX . Finally, deep models trained from different random initialization have been shown to be diverse on outlier data points [25, 10]. In summary, the ensemble constructed can satisfy our three conditions.
Ensemble Method TRM. Algorithm 3 describes another method designed with the representation matching technique for domain adaptation, which can potentially improve the accuracy of the ensemble on some test inputs related to the training data and thus satisfy our success condition (C) better. It requires the model architecture to be c(φ(x)), i.e., a composition of a prediction function c and a representation function φ. Beginning from a pre-trained model h0, it fine-tunes h0 for N epochs by minimizing the loss on D and R plus a representation matching loss α · d(pφD, p φ UX
), and outputs the N checkpoint models at the end of each training epoch as the ensemble. A key component of TRM is representation matching, which aims to learn a function φ that minimizes d(pφD, p φ UX
). It has been shown that representation matching can improve the accuracy on the test data from the target domain. Also, we have observed that the checkpoint models can have diversity on the misclassified data points empirically. Thus, the ensemble constructed can satisfy our conditions. For our experiments, we use the representation matching loss from the classic DANN [12].

heading: 7 Experiments
text: We perform experiments for unsupervised accuracy estimation and error detection tasks on 59 pairs of training-test datasets from five dataset categories, including image classification and sentiment classification datasets. In summary, our findings are: (1) Our method achieves state-of-the-art results on both accuracy estimation and error detection tasks than the existing methods. (2) Both ensemble and self-training techniques have positive effects on the tasks and it is easy to pick suitable hyperparameters for our algorithms. (3) Empirical results show that the conditions made in our analysis hold approximately.

heading: 7.1 Setup
text: We briefly describe the experiment setup here. The detailed setup can be found in Appendix C.1.
Dataset. The task needs a pair of training-test datasets D and UX . We use five dataset categories, each containing multiple training-test dataset pairs. Specifically, we use the following dataset categories: Digits (including MNIST [26], MNIST-M [12], SVHN [29], USPS [19]), Office-31 [33], CIFAR10-C [24], iWildCam [1] and Amazon Review [2]. Digits has 12 dataset pairs, Office-31 has 6, CIFAR10-C has 19, iWildCam has 10 and Amazon Review has 12.
Evaluation Metrics. We use absolute estimation error for accuracy estimation and use F1 score for error detection.
Our Models. On each dataset category, we design a neural network architecture for the DANN training algorithm [12], which is named DANN-arch. It contains an encoder, a predictor branch and a discriminator branch. For TRM, we use the DANN-arch for the ensemble {hi} and pre-train the initial model h0 onD and UX using DANN algorithm. For {hi} in TRI, we also use the DANN-arch. The model f is pre-trained on D and we mainly consider two kinds of architectures for it: one is the DANN-arch (i.e., the model f shares the same architecture as the check model h); the other is a typical deep neural network (DNN).
Hyper-parameters. The hyper-parameters T ,N and γ can be easily selected since a broad range of values can lead to good results. Based on our observation, larger T and N can lead to better results. In our experiments, we set T = 5 and N = 5 by considering the computational cost (on Amazon Review, we set N = 20). We set γ = 0.1 and set α following the domain adaptation methods.
Baselines. For accuracy estimation, we consider Proxy Risk [4], Average Confidence (Avg Conf) [9], and Ensemble Average Confidence (Ens Avg Conf). For error detection, we consider Proxy Risk, Maximum Softmax Probability (MSP) [17], and Trust Score [21]. Although Proxy Risk and our method share some similar ideas such as the use of check models, disagreement and representation matching, they have some major differences in the key ideas, training objectives, and the implementation of training objectives (refer to Appendix A.1 for a detailed discussion).

heading: 7.2 Results
text: Performance for Accuracy Estimation. The results in Table 1 show that our method (with TRM) achieves significantly better results across various dataset categories compared to existing methods. Specifically, on Digits and CIFAR10-C, it outperforms current state-of-the-art method Proxy Risk significantly (e.g. reduce the error by > 40%). On Office-31 and Amazon Review, it also outperforms the others and has a large advantage on the pre-trained models using DANN-arch.
We would like to emphasize the results on the challenging dataset iWildCam. Our method (with either TRI or TRM) outperforms the other methods significantly (e.g., reduce the error by> 70%), and the instantiation with TRI gets the best results. Note that on iWildCam, the label distribution of the test data is imbalanced and different from that of the training data. In such a case, the representation matching technique will fail since the representations of the two domains may be misaligned. Thus, the performance of Proxy Risk becomes worse, as it relies on representation matching. Our method with TRM also uses DANN in the ensemble method, but performs significantly better than Proxy Risk, since the diversity helps satisfy condition (C) though the representation matching fails to improve accuracy there. This shows that our ensemble and self-training techniques could alleviate the drawbacks of representation matching in such cases. Furthermore, our method with TRI achieves even better results, which demonstrates the flexibility and effectiveness of our framework.
The results for each dataset pair from Digits and CIFAR10-C are plotted in Figure 1. The plots show that proxy risk tends to underestimate the accuracy. This is because proxy risk maximizes disagreement which can overly suppress the accuracy. While the average confidence methods tend to overestimate the accuracy, because the model f tends to be overconfident on the data with dataset shift, even when this issue gets rectified in the ensemble average confidence method. In comparison, our method with TRM exhibits a clear advantage. The full results for all dataset categories and pretrained models in Appendix C.2 show a similar trend.
Performance for Error Detection. Table 1 shows that our method with TRM outperforms existing methods on all dataset categories. Specifically, our method improves the F1 score by at least 4.4%
on Digits, by at least 25.6% on Office-31, by at least 2.0% on CIFAR10-C, by at least 4.7% on iWildCam and by at least 1.4% on Amazon Review. This shows the advantages of our method to identify error points in the unlabeled test dataset.
Ablation Studies. To study the effect of using ensembles, we vary the ensemble size N (N = 1 means one single h, without ensemble) in our method with TRM. Similarly, for self-training, we vary the self-training iteration number T (T = 1 means no self-training). Figure 2 shows their effect on the average F1 score: both ensemble and self-training techniques have positive effects for identifying error points and thus also for accuracy estimation. Moreover, increasing T andN can lead to further
improvement. In addition to N and T , we similarly exam the effect of the last hyper-parameter γ. The figure shows that a wide range of γ can lead to good results, so it is easy to pick a suitable γ.
Validating the Theoretical Analysis. Our analysis relies on three conditions (A)(B)(C) stated in Section 5. Our experiments in Appendix C.4 show that empirically they are roughly satisfied. For example, for typical DNN f on MNIST→MNIST-M, ν̃ = 3.39%, γ̃ = 0.73% while σ2L = 26.58%. We note that our theoretical analysis is for formalizing our intuition and is for the worst case. Even when our assumptions are not fully satisfied on some complex datasets, our method can still have empirical performance better than the error bound.
Evaluating f with Various Architectures. We evaluate different architectures for the model f on Digits and show the results in Appendix C.5. The results demonstrate that our method consistently outperforms other methods on pre-trained models with various deep learning model architectures.
Analysis on Target Accuracy. In Appendix C.8, we show that the target accuracy of our ensemble check models can be higher or lower than that of the model f depending on the datasets and in both cases, our method can achieve good performance.
Analysis on Proxy Risk. In Appendix C.9, we show that the disagreement maximization is crucial for Proxy Risk and combining Proxy Risk with ensemble doesn’t lead to better results than our method.

heading: 8 Discussions
text: While our framework is general and flexible when combined with different ensemble methods, and it is easy to design ensemble methods satisfying the success conditions, we note that some prior knowledge about the data is needed to achieve the best performance. For example, iWildCam has imbalanced classes which hurt representation matching and thus TRM, while TRI is more suitable for such data. Different instantiations thus have their own limitations. For TRM, matching failure can cause errors on f ’s correct points, and too strong matching can decrease the diversity since the models are too restricted. For TRI, it does not attempt to improve the accuracy on the test inputs and relies only on diversity to satisfy condition (C). Then it can have worse performance than TRM on test data with connection to the training data that can be exploited. The success conditions we identified can guide the design of different instantiations. In this paper we focus on ensembles of deep learning models, since they readily satisfy the conditions. However, other ensemble methods with other types of models may also be useful. We leave the exploration for future work.

heading: Acknowledgments
text: The authors would like to thank Dr. Ankur Taly for his valuable comments. The work is partially supported by Air Force Grant FA9550-18-1-0166, the National Science Foundation (NSF) Grants CCF-FMitF-1836978, IIS-2008559, SaTC-Frontiers-1804648, CCF-2046710 and CCF-1652140, and ARO grant number W911NF-17-1-0405. Jiefeng Chen and Somesh Jha are partially supported by the DARPA-GARD problem under agreement number 885000.

Review summaries:
The paper studies the problem of error detection and accuracy estimation of unlabeled data points. Particularly, the paper proposes to use a diverse ensemble of models and detect errors by checking the inconsistency between the ensemble prediction and the target model prediction. Moreover, the paper proposes a self-training procedure that iteratively identifies incorrect predictions and adds their pseudo labels as new training targets to detect more errors. The paper also gives some theoretical analysis given assumptions about the ensemble models' quality.

- The paper considers two problems: unsupervised accuracy estimation and error detection, both in out-of-distribution settings. - The authors propose a practical approach, where they train a ‘check model’ h using self-training and ensembling. Points on which the prediction model and the check model disagree on are predicted as misclassified. - The theoretical analysis identifies conditions under which the approach should work and presents a bound on the accuracy estimation error - The authors empirically show that their algorithm outperforms baselines for both unsupervised accuracy estimation and error detection

 The paper proposes a framework to tackle two tasks: 1. unsupervised accuracy estimation (estimate accuracy on unlabeled data) and 2. error detection (identify the misclassified examples in the unlabeled data). The framework uses an ensemble of networks to identify mis-classified points as points where the current classifier disagrees with the ensemble. Then the mis classified points are used for self-training to improve the ensemble. The paper gives a theoretical analysis which depends on assuming that the ensemble is mostly correct where the classifer is correct, the classifier and the ensemble disagree on pseudolabeled data and that the ensemble is diverse in its predictions, which is claimed to be true empirically. 

The paper considers the problem of unsupervised accuracy estimation and error detection of pre-trained classifiers. The contributions consist of (i) a bound on the generalization performance on estimated accuracy and on detected errors and (ii) a solution based on ensemble models. Specifically, the bound on the generalization error takes into account three quantities, namely (i) the error of the ensemble on correctly-classified test instances, (ii) the agreement of the ensemble and the pre-trained model on potentially wrong test instances and (iii) the diversity of the ensemble on mis-classified test instances. The solution based on ensemble models consists of an iterative training strategy, where at each iteration the ensemble is trained to minimise a training surrogate for quantity (i) combined with quantity (ii) and ensuring diversity among ensemble predictions across iterations. Experimental analysis on several benchmarks (including digits, Office-31, CIFAR-10, iWildCam and Amazon Reviews) show the potential of the proposed solution.  Overall, the paper is well written and well structured. The idea of using ensemble strategies in the context of unsupervised accuracy estimation and error detection is novel. However, there are some issues in terms of ORIGINALITY and QUALITY, which need to be addressed and/or discussed (see below for further comments).