Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training  Ensembles
Authors: Jiefeng Chen
Frederick Liu
Besim Avci
Xi Wu
Yingyu Liang
Somesh Jha
Venue: NeurIPS 2021 Poster (2021)
Keywords: unsupervised accuracy estimation
error detection
self-training ensembles

Abstract: When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance. For safe deployment, it is essential to estimate the accuracy of the pre-trained model on the test data. However, the labels for the test inputs are usually not immediately available in practice, and obtaining them can be expensive. This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs. In this paper, we propose a principled and practically effective framework that simultaneously addresses the two tasks. The proposed framework iteratively learns an ensemble of models to identify mis-classified data points and performs self-training to improve the ensemble with the identified points. Theoretical analysis demonstrates that our framework enjoys provable guarantees for both accuracy estimation and error detection under mild conditions readily satisfied by practical deep learning models. Along with the framework, we proposed and experimented with two instantiations and achieved state-of-the-art results on 59 tasks. For example, on iWildCam, one instantiation reduces the estimation error for unsupervised accuracy estimation by at least 70% and improves the F1 score for error detection by at least 4.7% compared to existing methods.  

Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
Jiefeng Chen
Frederick Liu Google
Department of Computer Science University of
Seattle, WA 98103
Madison, WI 53706
frederi.cklid@goggle.com
jie sc. ec.edu
Besim Avci Google
Xi Wu Google Madison, WI 53703 wuxi moxxiegoogle..o.m com
Yingyu Liang
Department of Computer Science University of Tisconsin-MMadine
Seattle, WA 98103 bes besim@googles com
Madison, WI 53706 y yliang@cs. wisc edu
Somesh Jha
Department of Computer Science University of Wisconsin-Madiion Madison, WI 53706 jha@cs. wwwc. edu
Abstract
When deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance. For safe deployment, it is essential to estimate the accuracy of the pre-trained model on the test data. However, the labels for the test inputs are usually immediately available in practice, and obtaining them be expen- sive. This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims estimate the accuracy of pre-trained classifier set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs. In this paper, we propose a principled and practically effective frame- work that simultancously addresses the two tasks. The proposed framework it- eratively learns an ensemble of models identify mis-classified data points and performs self-training to improve the ensemble with the identified points. Theo- retical analysis demonstrates that our framework enjoys provable guarantees for both accuracy estimation and error detection under mild conditions readily satis- fied by practical deep learning models. Along with the framework. we proposed and experimented with two instantiations and achieved state-of-the-ar results on 59 tasks. For example, on iWildCam, one instantiation reduces the estimation error for unsupervised accuracy estimation by at least 70% and improves the F1 score for error detection by at least 4.7% compared to existing methods.
Part of the work done while interning Google. Our code is available https: http:////wt...ooo///////aaaaaaaa
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Introduction
Data distribution in the real world may be wildly different from the training dataset for various reasons such as covariate shift due to domain divergence, corruption of images due to weather con- ditions, or out-of-distribution test inputs. When facing these issues, a deployed deep learning model can have unexpected performance drop on the test data. This performance degradation can be miti- gated by test data annotation, which may be costly. Hence, estimating the accuracy of a pre- tre-trained model on the unlabeled test data provides an alternative to avoid the cost when it is not necessary. Furthermore, it is beneficial to estimate the correctness of the predictions on individual points. This leads to an even more challenging task of error detection, which aims to identify points in the un- labeled test set that are mis mis-classified by the pre-trained model. Such finer- meerrgrriiedd estimation can facilitate a further improvement of the pre-trained model (e.g., manually label those mis-classified data points and retrain the model on them).
While there have been previous attempts to address accuracy estimation or the broader problem of error detection, their successes usually rely some conditions or assumptions that may not hold in practice. For example, a natural approach is to use confidence based metrics to measure the performance of the pre-trained model, c.g., as in [9]. If the model is well-calibrated on the test data, then the average confidence approximates its accuracy. However, it has been observed that many machine learning systems, in particular modern neural networks, on test data with distribution shift [14, 30J. Another method is to learn a regression function that takes statistics about the model and the test data input, and predicts the performance the test data [9, 35]. This requires training on labeled data from various data distributions, which is very expensive even impractical. Furthermore, the performance predictor trained on the labeled data may not generalize to unknown data distributions. Recent work by [4] proposes to learn a "check" model using domain-invariant representation and use it as a proxy for the unknown true test labels to estimate the performance via eτror detection. It relies on the success of the domain-invarian representation methods obtain highly accurate check model on the test data. Hence, the check model performance suffers when domain-invarian representation is not accurate in circumstances such as test data having outlier feature vectors or different class probabilities than the training data.
poorly calibrated, especially
In this paper, we propose a principled and practically effective framework for the challenging tasks of accuracy estimation and error detection (Section 4). The framework makes novel use of the self-training technique on ensembles for these tasks. It first learns an ensemble of models to iden- tify some mis-classified points. Then it assigns pseudo-labels to these points and uses self-training with these pseudo-labeled data to identify more mis-classified points. We also provide provable guarantees for the framework (Section 5). Our analysis shows that it provably outputs an accurate estimate of the accuracy as well as the mis-classified points, under mild practical conditions: the ensemble make small errors on the test inputs correctly classified by the model f, mostly disagree with f on the current identified mis-classified inputs, and are correct or have diverse predictions on the remaining inputs. These conditions can be readily satisfied by deep learning model ensembles. Furthermore, they have no explicit assumptions on the test distribution and thus the framework can be instantiated by incorporating properly designed ensemble methods for different settings (Sec- tion 6). Experimental results on 59 tasks over five dataset categories including image classification and sentiment classification datasets show that our method achieves state-of-the-an on both aceu- racy estimation and error detection (Section 7). The experiments also provide positive support for our analysis, verifying the conditions and implications.
2 Related Work
Confidence Estimation and Error Detection. Recently, estimating model confidence has been an important area of research because of the perceived relationship between model uncertainty and trusting its predictions [28]. However, modern neural networks have been observed to be poorly calibrated (e.g. they may make wrong predictions with very high confidence) [14], especially on distributions with dataset shift [30]. Many approaches have been proposed to address this issue, such as Temperature Scaling [14), Monte-Carlo Dropout [11] and Deep Ensemble [25], but the challenge still remains. A similar challenge appears in error detection, where the goal is to iden- tify erroneous predictions given a test set. Combining these two problems, some early work uses confidence estimates to detect incorrect predictions. For example, [17] proposed to use maximum

---

softmax probability to detect misclassified examples. [5] proposed True Class Probability for failure prediction. [21] proposed use Trust Score to estimate the confidence model predictions. These methods require a robust estimate of confidence, and an empirically chosen threshold that is mostly problem- and mnoee--dependent to identify error data points. Recently, [4] proposed to use a check model to predict mis-classification.
Unsupervised Accuracy Estimation. The problem of unsupervised accuracy (or risk) estimation has received relatively scant attention from the research community. [8] offered a solution with cer- tain assumptions on the marginal output distribution p(y). [32] proposed estimate the accuracies of the approximations to some target Boolean functions based on the agreement rates method. A follow-up work by [31] also considered the "multiple approximations" problem setting where the target classes may be tied together through logical constraints, and proposed an efficient method to estimate the accuracy of classifiers using only unlabeled data. [20] proposed a spectral-based approach to estimate the accuracies of multiple classifiers, mainly in the binary case and with some assumptions on the classifiers.
[36] proposed a method to estimate the model's error on distribu- tions different from the training distribution by assuming conditional independence structure of the data. These problem settings are different from ours and the constraints may not be satisfied by the data and model we consider. Recently, [9] studied three families of methods (H-divergence, re- verse classification accuracy and confidence measures) and demonstrated how they could be used to predict the performance drop. [35] also proposed to learn a performance predictor on the statistics of model outputs with an assumption that they know the typical cases of dataset shift in advance. Sim- ilarly, [7] proposed to train regression models on the feature statistics of the datasets sampled from meta-dataset to predict model performance and [6] proposed to utilize linear regression to estimate classifier performance from the accuracy of rotation prediction. [13] proposed use difference of confidences to estimate classifier performance. [3] proposed MANDOLINE that utilizes user- specified slicing functions to improve the importance of weighting to make the accuracy estimation more accurate. With the most similar setup to ours, [4] used set of domain-invariani predictors as proxy for the unknown target labels to estimate a given model's performance under distribution shift. In a concurrent work, [22] empirically showed that the test accuracy of deep networks can be estimated by measuring disagreement rate between a pair of models independently trained via Stochastic Gradient Descent (SGD) and theoretically related this phenomenon to the well- well-calibrated nature of ensembles of (GD-trained models.
3 Problem Statement
Consider the classification problem with sample space X' and label space y. Let D be of labeled data from the training distribution Px, Let U (2, yz)} be a set of labeled data from the test dis- tribution Qx.y. where yı is the label for æ. Let Ux (x: : (x, Mr) U} be the set of input feature vectors in U. Define the accuracy of model f on U acc
lol (o [f((x) yul, where [U] is the cardinality of the set U. When clear from the context, we will omit U and sim- ply write acc(f)- Given model f X -> y' trained
D, together with 1) and Ux, the goal of unsupervised accuracy estimation is to get an estimate ačc so that the absolute estimation error Jaéc - acc(ccc))) is small. We assume access the training data D to help in estimating acc(f).
Though ace(f) is usually sufficient to provide an estimate of whether the model could perform well on U. it would be beneficial if the algorithm could also provide an estimate of the correctness of individual points so that we can know where to improve the model f. Thus, we also consider a more challenging task of identifying the mis- classifiiiid points in U: Wx == {x (I, v2) U, f(x) # 7. Given f. D and Ux the goal of error detection is to identify a subset Rx C Ux such that [W Ax is small, where WxARx (Wx \ Rx) U (Rx \ Wx).
4 Algorithmic Framework via Self- Traininggg Ensembles
Intuition. We consider the approach of learning a "check model" h and using the disagreement between h and the pre-trained model f for our tasks. The fundamental idea is to identify a point as mis-classified if h disagrees with f on æ. To derive our method, we note a simple fact: the disagreement approach succeeds if (1) h agrees with I on points where ʃ is correct and (2) disagrees with f on points where is incorrect. Our first key observation is that usually (1) can
3

---

Framework 1 Error Detection and Unsupervised Accuracy Estimation via Self- Training Ensembles
Input: A training dataset D, an unlabeled test dataset Ux, a pre-trained model ʃ. an ensemble learning method T, and a hyper-parameter
1: Initialize R = /
2: for = 1,2. .T do
3: Use T to generate an ensemble T(D, Ux
4: Identify those points on which the ensemble and disagree as mis-classified points:
Rx == E
Pr (h(z) f()) < <}
A-T
5: For each I E Rx assign a pseudo-label U2 * f(z) (c.g.. using the majority vote of the ensemble random label). And set R = {(0, Uz) E Rx}-
6: end for
Output: The estimated mis-classified points Rx, and the estimated accuracy
[xxl Uxx
be satisfied approximately in practice. Intuitively, this is because h and f use the same training data, and & can be trained to be correct on the subset of the instance space where f is correct. However, (2) may not be easily satisfied le.g., h can make similar mistakes as f), which leads to an overestimation
of the accuracy. We thus focus on improving the disagreement on mis classsaasified points.
To disagree with f on a mis-classified test input æ, we have two ways: (1) make the check model correct on (2) diverse ensembles. The first way may be achievable when the training data contains information for prediction on x. A prototypical example is when the test inputs are corruption of clean data from the training distribution (e.g., the training data are images in sunny days while the test inputs are ones in rainy days), and techniques like unsupervised domain adaptation can be used to improve the prediction on such test inputs. However, correct predictions in many interesting scenarios due to insufficient information (e.g., the test image in the open world can contain object that is never seen in the training data ). Fortunately, it has been shown that for such inputs, one can obtain an ensemble of models with diverse predictions (e.g..(25)). This then gives the second way achieve disagreement: using diverse ensembles. Therefore, our method will learn an ensemble of models (instead of one check model) and identify a point x as mis-classified if the ensemble disagree with f x. (i.e., large fraction of the models in the ensemble disagree with f on their predictions on r².. A mis-classified test input, x, will be successfully identified, if a majority of the ensemble models predict correctly, they have large diversity on x.
may not be feasible
However, the ensemble may only be able to identify subset of the mis-classified points. Therefore, propose to iteratively identify more and more mis-classified points by self-training. For each mis-classified data point x identified by the ensemble, we assign it a pseudo-label that is different from (x) (c.g. use the majority vote of the ensemble or random label the pseudo-label). Then we can train (with regularization) a new ensemble to encourage their disagreement with ff on the pseudo-labeled data R. (e.g., use a supervised loss on R with a small weight the regularization). Note that the pseudo-labels may not all be correct, and we do not need the new ensemble to exactly fit the pseudo-labels. We only need the new ensemble to mostly disagree with f on R so that they still identify R as mis-classified points. Furthermore, self-training can help the ensemble identify more mis-classified points. When some pseudo-labels are correct, we observe that these additional data can help the new ensemble become more accurate and thus help with identifying more mis-classified points. We also observe that the new ensemble will be less diverse on the pseudo-labeled data and thus have diversity on the remaining test inputs. We report empirical results in Appendix C.4 to support these observations.
Our framework. We assume access to an ensemble learning method T (specific instantiations will be given later) that takes as input training set D, unlabeled test set Ux, and pseudo-labeles set R (and potentially some other parameters) and outputs an ensemble, which is a distribution over functions h: X' + y We denote the generated ensemble as T(D, Ux, R) and write h T(D, Ux, Rt) for sampling h from the ensemble, or simply h ~ T when clear from context.
"This includes one single h special case. It also includes the of h with probabilistic outputs, i.c., h(x)- = 'h' (z),..., .. K (xr) for y = {1, 2,.., K}, where h' (œ) the predicted prohability of x from class ensemble where Prg-T [g(x) = i = (x))..
i. One think of such

---

We are now ready to describe our method (Framework 1). Our framework begins with an empty set of pseudo-labeled data R. and executes in T iterations. In each iteration, generates an ensemble T(D, Ux, R) and then uses the ensemble to construct a new pseudo-labeled dataset R for the next iteration: let Rx be those points x E Ux where the agreement rate between the ensemble and f is below threshold, assign a pseudo-label Uh different from fxx), and let R be the set of these pseudo-labeled data. Finally, it outputs Rx as the mis-classified points and outputs the fraction of points outside Rx as the accuracy.
Some existing work also use models' disagreement to estimate the error on the unlabeled data in different ways. For example, [32] consider a set of pre-trained models and aim to estimate the error for each model; [4] learn a single check model and compare it with f. On the other hand. our novelty is using ensembles to identify set of mis-classified points and further using self-training for learning the ensembles. Note that our self-training ensembles is different from standard self- training and ensemble: standard
aims to get accurate predictions while ours increase disagreement on mis-classified points (refer to Appendix A.2 for a detailed discussion).
5 Theoretical Analysis
As described in the intuition above, our framework succeeds if in each iteration, the ensemble satisfy the following conditions: (A) correct on Ux Wx. (B) mostly disagree with I on Rx. and (C) either correct or diverse on Wx \ Rx. This section first introduces notions formalizing these conditions and then provides provable guarantees under these conditions. Here we focus intuitions and provide the proofs and more discussion in Appendix B.
Notations. Recall that the ensemble method T outputs distribution over models, and we write ~ 'T for sampling a model h from this distribution. Let E,, denote the expectation over this output distribution, and Er denote the average over x E Ux or (z,, Va) E U.
Formalizing Condition (A). key observation we make in practice is that usually on points where is correct, the ensemble models are also correct. Intuitively, this is because the only labeled information for learning the ensemble D, while D is also used for learning f. For illustration, suppose I and the ensemble have zero training errors on D. When they are from hypothesis classes of bounded VC-dimensions and the training set is large enough compared to the VC-dimensions, standard error bounds show that the ensemble will have small errors the subset of the feature space {i E X : f(x)= = yz i.e., will have small errors on points where f is correct. To formalize this observation, introduce the following notion.
Definition 1 (Error Correct Points). Let denote the average probability of h T making error on test inputs where the model f is correct: v == Pr [h(x) + y. fxx = yul-
Formalizing Condition (B). We assume the new ensemble trained with regularization on R will mostly disagree with f on Rx. We define the following notion:
Definition 2 (Agreement on Pseudo-Labeled Labeled Data). Let denote the average probability of agree- ment between h ~ TT and f on Rx: y == PraeRx A~T {h(z) f=)).
Formalizing Condition (C). Let Gx denote the good points in Wx \ Rx on which the ensemble will have correct predictions with high confidence, Bx the remaining bad points. Formally, we define: Gx == {i E Wx \ Rx (r=-(((x2)
yn } 2 1 v and Bx == Wx (Rx UG
(Here we choose the confidence level v for convenience, where v is the error on correct points. Other sufficiently high confidence levels can also be used with slight change to the analysis.) We would like the ensemble to have large diversity on Bx
Definition 3 (Diversity of Ensemble). Let o² denote the average probability of disagreement be- tween two ensemble models on Bx: o² := Eg E Bx), where o² ;= Pra, Phhh (h) + h2(x)).
Provable Guarantees. Based on the above notions we obtain our guarantees:
Theorem 1. Assume in each iteration of the framework,
/1 - 1) for some 1) (0, 10,3B,/4 /4) where B, == min(o2,1- 122 Let > 0 be lower bound on the diversity 0², 5 > 0 be upper bound on 7/ and v be upper bound on over all iterations. Then for any d E (0, of of /4) after at
5

---

Algorithm Error Detection and Unsupervised Accuracy Estimation via Self-Training Ensembles
Input: D. Ux. ensemble method T, parameters T,
1: Initialize R ===0 0
2: fort = 1, 2, - do
3: Set {ha]
(D,, Ux R,N, NN other parameters)
1 4: Let 82 be the majority vote of {hel ds == arg ma N EE [h((2)) = 51.
5: Set R= {(2, 92) x E Ux Us # f(z)}.
6: end for
Output: Estimated mis-classified points Rx = (x : (z, y) E R), estimated accuracy
[0x]
[6x\ most [1/6] iterations, we can get [xx the mis-classifed points Wx as follows:
approximates the accuracy acc( and Rx approximates
acc(f)
1Ux Rx xUx
max
1 ey), where e:=
i
1 +
1-t
e
1-51
(1)
I-7 T
o
8 8+3 4
1
[XxARx
Ux Wx|+ c
(2)
Proof Sketch. Let's first consider one iteration, assuming small v and large o². Intuitively, on the correct points Ux Wx, the ensemble have small enror and thus disagree with on only a few such points. On the old Rx, the ensemble mostly disagree with f(z): similarly on Gx. On the remaining points Bx, show that if the ensemble have large diversity, then their predictions must have large variances on a significant subset of points and thus have large disagreement with f. facilitating the detection. Overall, our framework can construct new pseudo- -eeule-labeled set R'y that contains mostly mis-classified points and is also larger than the old Rx (Lemma in Appendix B). Therefore, each iteration can make some progress by identifying more mis-classified points than before, and enough iterations achieve the guarantees.
The theorem provides guarantees for general values of v, and 0². To get some intuition, note that typically e, < T and suppose we set r = 3/4 and = 7/T, then the accuracy is estimated up to eгror ma, 30 (e)+ When i0 7 are small and of, is large, the error is small. Therefore, under mild conditions, our framework can give provable estimation of the accuracy and the mis-classified points up to small errors.
165
The theorem formalizes that the framework can succeed under mild conditions (A (A)B)(C) on the en- sembles, without explicit conditions on the data distributions and the pre-trained model (note that the conditions them are implicitly captured in the mild conditions the ensemble). The framework thus flexible, and different ensemble methods satisfying these conditions can be incorporated to get concrete instantiations applicable to various settings. Indeed, the crux of our framework is then to design ensemble methods meeting the conditions. This turns out to be not difficult, in particu- lar for deep learning pre-trained models and deep learning ensemble models. Even ensemble of deep networks simply trained from random initialization works well. Two concrete instantiations are presented Section 6 below and evaluated in Section 7.
Instantiations of the Framework
Based on our framework, we propose Algorithm 1 for accuracy estimation and error detection. executes in T iterations. In each iteration, it trains an ensemble of models ( and then uses the ensemble to construct the neudo-labeled data. Finally, outputs Rx as the set of mis-classified points and outputs the fraction of points outside Rx as the accuracy. The algorithm uses an intuitive heuristic to implement the threshold on the agreement rate: it sets Rx as the points r E Ux with f(x) different from the majority vote of the ensemble models. Empirically, we observe that this leads to similar Rx and thus similar results as explicit thresholding, but is much more convenient (the empirical results to support this observation are reported in Appendix C.6).

---

Algorithm 2 TRI: Ensemble via Random Initialization
Input: D. Ux. R. N, parameter
I: Pre-train i= on D from different random initialization
2: for = 1, ..2....,N N do
3: Learn h2 by fine-tuning 1/ for one epoch by:
minimize
E(E.)) v)1 + (lee))el((((()).))) v)
(3)
4: end for
Output: The ensemble of models (
Algorithm TRM: Ensemble via Representation Matching
Input: D, Ux R. N. initial pre-trained model hgs parameters a, I/ ho(z) c(((x)) is composition of prediction and a representation 6 1: Fine-tune ho for N epochs using the objective:
minimize ((x.))E(((((()), v)l + ((e.))eR a Medddvy pUX)
+
(4)
where d(p p ) is the distance between the distribution of ((z) on D and that on Ux-
2: Use the N checkpoint models at the end of each training epoch the model ensemble (h: ) Output: The ensemble of models {
While different ensemble methods T can be used in Algorithm here we propose two concrete instantiations TRI and TRM based on the success conditions of our framework.
Ensemble Method TRI- Algorithm 2 describes natural method that trains the models from different random initialization. It first trains hς on D (e.g.. using the same training algorithm as for f). to ensure that ensemble has small error on points where f is correct. It then fine-tunes hy D and R for one epoch to get hy that mostly disagrees with f on Rx. Finally, deep models trained from different random initialization have been shown to be diverse outlier data points [25, 10]. In summary, the ensemble constructed can satisfy our three conditions.
Ensemble Method TRM- Algorithm 3 describes another method designed with the representation matching technique for domain adaptation, which can potentially improve the accuracy of the en- semble some test inputs related to the training data and thus satisfy our success condition (C) better. It requires the model architecture to be c()(x)), i.., a composition of a prediction function and representation function Ø. Beginning from pre-trained model he it fine-tunes hg for N epochs by minimizing the loss D and R plus representation matching loss a d(y Pūx) ). and outputs the N checkpoint models at the end of each training epoch as the ensemble. A key compo- nent of RRM is representation matching, which aims to learn function ó that minimizes d(p Pu )- has been shown that representation matching can improve the accuracy on the test data from the target domain. Also, we have observed that the checkpoint models can have diversity on the mis- classified data points empirically. Thus, the ensemble constructed can satisfy our conditions. For our experiments, we use the representation matching loss from the classic DANN [12].
7 Experiments
We perform experiments for unsupervised accuracy estimation and error detection tasks on 59 pairs of training-test test datasets from five dataset categories, including image classification and sentiment classification datasets. In summary, our findings are: (1) Our method achieves sfate-of-the-ee results on both accuracy estimation and error detection tasks than the existing methods. (2) Both ensemble and self-training techniques have positive effects on the tasks and it is easy to pick suitable hyper- parameters for our algorithms. (3) Empirical results show that the conditions made in our analysis hold approximately.
7

---

7.1 Setup
We briefty describe the experiment setup here. The detailed setup can be found in Appendix C.I.
Dataset. The task needs pair of training-ter test datasets D and Ux. We use five dataset categories, each containing multiple training-test dataset pairs. Specifically, we use the following dataset cat- egories: Digits (including MNIST [26], MNIST-M [12], SVHN [29], USPS [19]), Office-31 [33], CIFARIO-C [24], iWildCam [1] and Amazon Review [2]. Digits has 12 dataset pairs, Office-31 has 6, CIFARIO-C has 19, iWildCam has 10 and Amazon Review has 12.
Evaluation Metries. We use absolute estimation error for accuracy estimation and use Fl score for error detection.
Our Models. On each dataset category, we design a neural network architecture for the DANN training algorithm [12], which is named DANN-arch. It contains an encoder, a predictor branch and discriminator branch. For TRM. we use the DANN-arch for the ensemble { and pre-train the initial model hg on D and Ux using DANN algorithm. For {h in TRI, we also use the DANN-arch. The model pre-trained on D and we mainly consider two kinds of architectures for it: one is the Drch- (i.e., the model f shares the same architecture as the check model h); the other is a typical deep neural network (DNN).
The hyper-parameters T, N and can be easily selected since a broad range of values lead to good results. Based on our observation, larger 7 and N can lead to better results. In our experiments, we set T = 5 and N = 5 by considering the computational cost (on Amazon Review, we set N 20). We set y 0.1 and set a following the domain adaptation methods.
pareee-prsmmstrs..
Baselines. For accuracy estimation, we consider Proxy Risk [4], Average Confidence (Avg Conf) [91, and Ensemble Average Confidence (Ens Avg Conf). For error detection, consider Proxy Risk, Maximum Softmax Probability (MSP) [17], and Trust Score [21]. Although Proxy Risk and our method share some similar ideas such as the use of check models, disagreement and repre- sentation matching, they have some major differences in the key ideas, training objectives, and the implementation of training objectives (refer to Appendix A. for a detailed discussion).
7.2 Results
Performance for Accuracy Estimation. The results in Table 1 show that method (with TRM) achieves significantly better results across various dataset categories compared to existing methods. Specifically, on Digits and CIFAR10-C, it outperforms current state-of-tie of-the-art method Proxy Risk significantly (e.g. reduce the error by > 40%). On Office-31 and Amazon Review, it also outper- forms the others and has large advantage on the pre-trained models using DANN-arch.
We would like to emphasize the results on the challenging dataset iWildCam. Our method (with either TRI TRM) outperforms the other methods significantly (c.g., reduce the error by > 70%). and the instantiation with TRI gets the best results. Note that on iWildCam, the label distribution of the test data is imbalanced and different from that of the training data. In such a case, the representation matching technique will fail since the representations of the two domains may be misaligned. Thus, the performance of Proxy Risk becomes worse, it relies representation matching. Our method with TRM also uses DAÑN in the ensemble method, but performs significantly better than Proxy Risk, since the diversity helps satisfy condition (C) though the representation matching fails to improve accuracy there. This shows that our ensemble and self-training techniques could alleviate the drawbacks of representation matching in such cases. Furthermore, our method with TRI achieves even better results, which demonstrates the flexibility and effectiveness of our framework.
The results for each dataset pair from Digits and CIFAR1O-C are plotted in Figure 1. The plots show that proxy risk tends to underestimate the accuracy. This is because proxy risk maximizes disagreement which can overly suppress the accuracy. While the average confidence methods tend to overestimate the accuracy, because the model f tends to be overconfident on the data with dataset shift, even when this issue gets rectified in the ensemble average confidence method. In comparison, our method with TRM exhibits a clear advantage. The full results for all dataset categories and pre- trained models in Appendix C.2 show similar trend.
Performance for Error Detection. Table shows that our method with TRM outperforms existing methods on all dataset categories. Specifically, our method improves the Fl score by at least 4.4%
8

---

Task
Accuracy Estimation
Error Detection
Metric
Absolute Estimation Error +
score 7
Model
Model f
Dataset
Method
Method
Typical DNN
DaNN-arch
Typical DNN
DANN-arch
Avg Conf
0,404 180 0.3
0.350:10.230
MSP
0,467=0.195
0.485:50.209
Ens Avg Conf
0.246=0.230
Trust Score
0.496=+0.155
0.4
Digits
Proxy Risk
0.085+0.142
0.095++01181
Proxy Risk
0.844+0.118
0.796++0.155
Ours (RT)
0.164::0.218
0.087+10.077
Ours (RT)
0..98
0.701 1.116
Ours (RM)
0.02310.020
0.02410.022
Ours (RM)
0.80110.084
0.84110..11
Avg Conf
0.054+0.044
0.259+0.133 134
MSP
0.281+0.266
0,584+0.122
Ens Avg Conf
0.080:10.011
0.28110.130
Trust Score
0.40110.240
0.559-10.143
Office-31
Proxy Risk
0.033=00012
0.042-+0.034
Proxy Risk
0.605=0.177
0.629=#0.140
Ours (RT)
0.051+00038
0.044+0.031
Ours
0.715+0.124
0.770++0.027
Ours (RM)
0.029+00.01
0.018+0.023
Ours (RM)
0.767+0.052
0,790+000087
Avg Conf
0.369=0.176 0.353=0.175
MSP
0.505=0.043
0.550=20.043
CIFARIO-
Ens Avg Conf
0.237++0. 144
0.237=00.13
Trust Score
0.494#0.045
0.568=00.000
C
Proxy Risk
0.053+00.000
0.052++0.070
Proxy Risk
0.850+0.107
0.843+0.100
Ours (RI)
0.14910.089
0.197.10.115
Ours (RD
0.63410.004
0.568
Ours (RM)
0.022+00.009
0.029=00012
Ours (RM)
0.872+0.083
0.860=00.09
Avg Conf
0.388100045
0.39510.043
MSP
0.69210.006
0.74110.009
Ens Avg Conf
0.177=00025
0.158=0.0000
Trust Score
0.717=0.009
0.737=0.010
iWilCCam
Proxy Risk
0.119=00.043
0.094=0.036
Proxy Risk
0.755=000038
0.773=00.039
Ours (RI)
0.015++0.008
0.007+00004
Ours (RD
0.792+0.001
0.806++0.012
Ours (RM)
0.035++0.022
0.026+0.004
Ours (RM)
0.796+0.014
0.000
Avg Conf
0.290=00.043
0.310=000045
MSP
0:420=:0.022
0.218=00033
Amazon Review
Ens Avg Conf
0.229+00.038
0.217+0.036
Trust Score
0.414+0.024
0,237+0.044
Proxy Risk
0.021++0.014
0.037+00.07
Proxy Risk
0.417+0.042
0.434+00.046
Ours (RI)
0.065100007
0.062.10.051
Ours (RI)
0.38410.032
0.453.10.037
Ours (RM)
0.018++0.010
0.022++0.011
Ours (RM)
0.426+0.036
0.440=+0.037
Table Results unsupervised accuracy estimation error detection. For typical DNN. We use CNN-BN
for Digits, ResNet50 for Office-31, ResNet34 for CIFARIO-C, ResNet50 for iWildCam, and Fully Connected
Network for Amazon Review, Wc show the mean and standard deviation of absolute estimation error and Fl score (mean=std). The numbers calculated over the training- test dataset pairs in cach dataset category. Bold numbers the superior results.
Avg Cont
Ens Avg Cont
Proxy Risk
Ours (RI)
Ours (RM)
HIST WMST-M
Target
0:30
2:23
030 0.75
100
0.0
0.25
5.50
0.5
100
0.0
0.25
152
015
1:0
02>
030
6/b
1:0
100
025
0.00
5.5 100
Estirnated
Avg Conf
Ens Avg Canf
Praxy Risk
Ours (RI)
Ours (RM)
1.0 CFRR100
Target
6.50
025
8.50
6.75
1,02
0.00
635
6.50
0.75
100
0.00
075 1.0 6d2 025
6.35
3.00 502
025
6.75
1.00
Estimated
Figure Accuracy estimation detailed results for each datasct pair from the dataset category Digits and CIFAR 10-C (results for other categorics are in the Appendix C.2). We use typical DNN as the architecture for the model f. We use symbols to represent training datasets and colors to represent test datascts. For CIFAR 10- C, there is only one training dataset with multiple test datasets. The dashed line represents perfect prediction (larget accuracy estimated accuracy). Points beneath (above) the dashed line indicate overestimation (under- estimation). The solid lines are regression lines of the results.
on Digits, by at least 25.6% on Office-31, by at least 2.0% on CIFARIO-C, by at least 4.7% on iWildCam and by at least 1.4% on Amazon Review. This shows the advantages of our method to identify error points in the unlabeled test dataset.
Ablation Studies. To study the effect of using ensembles, we vary the ensemble size N (N means one single h without ensemble) in our method with TRM- Similarly, for self-training, we vary the self-training iteration number T (T = 1 means self-training). Figure 2 shows their effect on the average Fl score: both ensemble and self-training techniques have positive effects for identifying error points and thus also for accuracy estimation. Moreover, increasing T and N can lead to further

---

Digits (T=5.ym0.))
Digit (8-3))m001)
Dighi (T+ N
0.80
Typical DAN DAWNorth
0.80
Aectoge 72 21.0
0.7
Average 33.0cee
0..66
0.86
Nuwrage11
0.86
0.84
1252
0.64
0.82
10
20
5.33
0.50
Figure 2: Ablation study for the effect of ensemble and self-training techniques on Digits (an additional ablation study CIFARIO-C is in Appendix C.7). N is the number of models in the ensemble, T is the number self-training iterations, and y is the weighting parameter for the loss term on the pseudo-labeled data. The ensemble training algorithm we use is TRM-
improvement. In addition to N and T, we similarly exam the effect of the last hyper-parameter 7- The figure shows that a wide range of y can lead to good results, so it easy to pick suitable y.
Validating the Theoretical Analysis. Our analysis relies on three conditions (A) stated in Section 5. Our experiments in Appendix C.4 show that empirically they roughly satisfied. For example, for typical DNN f on SUNIST-MNISTEY We note that theoretical analysis is for formalizing our intuition and is for the worst case. Even when our assumptions are not fully satisfied on some complex datasets, our method can still have empirical performance better than the error bound.
ù = 3.39%, ý = 0. 737% while of = 26.58%.
Evaluating / with Various Architectures. We evaluate different architectures for the model f on Digits and show the results in Appendix C.5. The results demonstrate that method consistently outperforms other methods on pre-trained models with various deep learning model architectures.
Analysis on Target Accuracy. In Appendix C.8, we show that the target accuracy of our ensemble check models can be higher or lower than that of the model f depending on the datasets and in both cases, our method can achieve good performance.
Analysis Proxy Risk. In Appendix C.9, we show that the disagreement maximization is crucial for Proxy Risk and combining Proxy Risk with ensemble doesn't 't lead to better results than our method.
8 Discussions
While framework is general and flexible when combined with different ensemble methods, and it is easy to design ensemble methods satisfying the success conditions, we note that some prior knowledge about the data is needed achieve the best performance. For example, iWildCam has imbalanced classes which hurt representation matching and thus TRM. while TRI is more suitable for such data. Different instantiations thus have their own limitations. For TRM, matching failure can cause errors on f's correct points, and too strong matching can decrease the diversity since the models are too restricted. For TRI. it does not attempt to improve the accuracy on the test inputs and relies only on diversity to satisfy condition (C). Then it can have worse performance than TRM the training data that can be exploited. The success conditions we
on test data with connection identified can guide the design of different instantiations. In this paper we focus on ensembles of deep learning models, since they readily satisfy the conditions. However, other ensemble methods with other types of models may also be useful. We leave the exploration for future work.
Acknowledgments
The authors would like to thank Dr. Ankur Taly for his valuable comments. The work is partially supported by Air Force Grant FA9550-18 18-00-188--11666 the National Science Foundation (NSF) Grants (CF-FMidF-186697
CCF-2046710 10 and CCF-1652140. and ARO grant number W91 1NN--17-1-0400 Jiefeng Chen and Somesh Jha are partially supported by the DARPA-GARD problem under agreement number 885000.
IIS-2008559,
10

---

References
[1] Sara Beery, Elijah Cole, and Arvi Gjoka. The iwildcam 2020 competition dataset. arXiv preprint arX1v:2004.10340, 2020.
[2] John Blitzer, Mark Dredze, and Fernando Pereira. Domain adaptation for sentiment classifica- tion. In 45th Anny. Meeting of the Assoc. Computational Linguistics (ACL'07), 2007.
[3] Mayee F. Chen, Karan Goel, Nimit Sharad Sohoni, Fait Poms, Kayvon Fatahalian, and Christo- pher Ré. Mandoline: Model evaluation under distribution shift. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021. Virtual Event, volume 139 of Proceedings of Machine Learning Re- search, pages 1617-1629. PMLR, 2021.
[4] Ching- Yao Chuang, Antonio Torralba, and Stefanie Jegelka. Estimating generalization un- der distribution shifts via domain--nvariian representations. arXiv preprint urXiv:2007.03511, 2020.
[5] Charles Corbière, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. Ad- dressing failure prediction by learning model confidence. In Advances in Neural Information Processing Systems, pages 2902- 2902-2913, 2019.
[6] Weijian Deng, Stephen Gould, and Liang Zheng. What does rotation prediction tell us about classifier accuracy under varying testing environments? In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedingy of Machine Learning Research, pages 2579-257525 2589. PMLR, 2021.
[7] Weijian Deng and Liang Zheng. Are labels always necessary for classifier accuracy evaluation? In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 15069-15078. Computer Vision Foundation IEEE, 2021.
[8] Pinar Donmez, Guy Lebanon, and Krishnakumar Balasubramanian. Unsupervised supervised learning i: Estimating classification and regression errors without labels. Journal of Machine Learning Research, 11(4), 2010.
[9] Hady Elsahar and Matthias Gallé. To annotate or not? predicting performance drop under domain shift. In Proceedings of the 2019 Conference Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) pages 2163-2173, 2019.
[10] Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: loss landscape perspective. arXiv preprint arXiv:1912.022 2019.
[11] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050- 1059, 2016.
[12] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. networks. The Journal of Machine Learning Research, 17(1):2096-2033 2016.
training of neural
[13] Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. Pre- dicting with confidence on unseen distributions. CoRR, abs/2107 1bb/01107.3335, 2021.
[14] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. arXiv preprint arXiv: 1rrXiv1706.000 776.04599, 2017.
[15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recogni- tion, pages 770-778, 2016.
[16) Dan Hendryeks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint
2019.

---

[17] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of- distribution examples in neural networks. arXiv preprint arXiv:1610.02136. 2016.
[18] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely con- nected convolutional networks. In Proceedings ofe IEEE conference on computer vision and paltern recognition, pages 4700-4708, 2017.
[19] Jonathan Hull. database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-555 1994.
[20] Ariel Jaffe, Boaz Nadler, and Yuval Kluger. Estimating the accuracies of multiple classifiers without labeled data. In Guy Lebanon and S. V. Vishwanathan, editors, Proceedings of the Fighteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2015, San Diego, California, USA, May 9-12, 2015, volume 38 of JMLR Workshop and Conference Proceedings. JMLR.org. 2015.
[21] Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust classifier. In Advances in neural information processing systems, pages 5541 2018.
[22] Yiding Jiang, Vaishnavh Nagarajan, Christina Baek, and J. Zico Kolter. Assessing generaliza- tion of SGD via disagreement. CoRR, abs/2106. 13799, 2021.
[23] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, et al. Wilds: A benchmark of in-the-wild distribution shifts. arXiv preprint arXi:: 2020.
[24] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[25] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable pre- dictive uncertainty estimation using deep ensembles. In Advances in neural information pro- cessing systems, pages 6402-6413, 2017.
[26] Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnistl/ 1998.
[27] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (6(11))2278-2222 1998.
[28] Samuele Lo Piano. Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. Humanities and Social Sciences Communications, 7, 06 2020.
[29] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[30] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanal and Jasper Snoek. Can you trust your model's 's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances Neural Information Pro- cessing Systems, pages 13991-14002, 2019.
[31] Emmanouil A. Platanios, Hoifung Poon, Tom M. Mitchell, and Eric Horvitz. Estimating ac- curacy from unlabeled data: A probabilistic logic approach. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4361 4370,, 2017.
[32] Emmanouil Antonios Platanios, Avrim Blum, and Tom M. Mitchell. Estimating accuracy from unlabeled data. In Nevin L. Zhang and Jin Tian, editors, Proceedings of the Thirlieth Conference on Uncertainty in Artificial Intelligence, UAI 2014, Quebec City, Quebec, Canada, July 23-27, 2014, pages 682-691. AUAI Press, 2014.
12

---

[33] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models new domains. In European conference on computer vision, pages 213-226. Springer, 2010.
[34] Claude Sammut and Geoffrey I. Webb, editors. TF-IDF, pages 986-987. Springer US, Boston, MA, 2010.
[35] Sebastian Schelter, Tammo Rukat, and Felix Biessmann. Learning validate the predictions of black box classifiers on unseen data. In Proceedings of the 2020 ACM SIGMOD International Conference Management of Data, pages 1289-1299, 2020.
[36] Jacob Steinhardt and Percy S Liang. Unsupervised risk estimation using only conditional independence structure. In Advances in Neural Information Processing Systems, pages 3657- 3665, 2016
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 8.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? (Yes) See Section and Appendix B.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix B.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either the supplemental material or as a URL)? [Yes] See the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix C.I.
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] See Appendix C.3.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix C.I.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See References.
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'l re using/curating? [N/A]
(e) Did you discuss whether the data you using/curating contains personally identifi- able information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "ziqkEFzKzeK", "reviewer": "Reviewer_1YgW", "summary": "\nThe paper proposes a framework to tackle two tasks: 1. unsupervised accuracy estimation (estimate accuracy on unlabeled data) and 2. error detection (identify the misclassified examples in the unlabeled data). The framework uses an ensemble of networks to identify mis-classified points as points where the current classifier disagrees with the ensemble. Then the mis classified points are used for self-training to improve the ensemble. The paper gives a theoretical analysis which depends on assuming that the ensemble is mostly correct where the classifer is correct, the classifier and the ensemble disagree on pseudolabeled data and that the ensemble is diverse in its predictions, which is claimed to be true empirically. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Quality\n\nThe framework is simple and makes good sense, the experiments are reasonably extensive, and there is some theoretical understanding, although it depends on strong conditions on the ensemble. As long as the ensemble obeys good properties on possibly OOD data, the framework seems to make sense, but there seems to be an art to this. For example, the conceptual understanding doesn't address the distribution shift between training and unlabeled examples, which is perhaps why Algorithm 3 (RM) incorporates a domain adaptation method and in general surpasses Algorithm 2 (random init ensemble) by a large margin consistently. \n\n\n- Self-training seems to improve the performance a lot, as in Figure 2. If self-training is also combined with Proxy Risk, does the ensembling method still improve over it on the 5 datasets? This would make clearer the role of ensembling vs self-training.\n- Although the theoretical assumptions are somewhat strong, the paper does show empirically that the values in the conditions are favorable on the digit datasets. What about in the more complex datasets?\n- In general, the ablations of ensembling and self-training parameters are only on Digits - do the same trends hold for other datasets?\n- When do the conditions hold? For example, what if the pre-trained classifier itself is a majority vote over an ensemble? \n- There are some observations that aren't reported with numbers, such as in L147. \n\n\nSignificance\n\nUnsupervised error detection and estimation are interesting tasks that, if we can make progress on them, it would help in model development for domain adaptation/OOD generalization, as well as a score for model selection, and as a tool in algorithmic frameworks for improving the model such as the one presented in this paper. \n\n\nOriginality\nThe main novel idea is to use ensembles + self-training in a similar framework to the Proxy Risk paper. The use of domain invariant representations, a check model, and disagreement is in similar spirit to Proxy Risk. \n- Neural network ensembles have been used for uncertainty quantification / OOD detection, but the error detection task here is slightly different as it takes into account the performance of some classifier, and the goal is to detect examples with errors.\n- The ensemble method in Algorithm 3 is reminicient of other domain adaptation + semi-supervised learning methods, for example DIRT-T. \n\nClarity\n\nThe paper is mostly clearly written.\n- the definitions of the various sets R_X / W_X / B_X / G_X etc are scattered throughout and hard to remember. \n- In Figure 1, the color/shapes change from the legend. I think this is denoting new train/unlabeled pairs, but this is pretty subtle.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors note that some prior knowledge is needed to achieve the best performance, and this will influence the design of the ensemble method, which remains an art. Social impact is not discussed.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "xtCzbtRuK12", "reviewer": "Reviewer_rdjh", "summary": "The paper studies the problem of error detection and accuracy estimation of unlabeled data points. Particularly, the paper proposes to use a diverse ensemble of models and detect errors by checking the inconsistency between the ensemble prediction and the target model prediction. Moreover, the paper proposes a self-training procedure that iteratively identifies incorrect predictions and adds their pseudo labels as new training targets to detect more errors. The paper also gives some theoretical analysis given assumptions about the ensemble models' quality.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed method matches my intuition about an ensemble of deep learning models. The empirical performance also looks very promising. The presentation of the paper is also very clear.\n\nIt may be worth investigating specific wrongly predicted data points and compare their true labels with their different predictions from f, models in the ensemble and across different self-training stages. \n\nFor the empirical verification about the theoretical analysis. I think the authors should also provide the final bound calculated using the estimated parameters.\n\nMinor comments:\nI don't quite agree to use \"linear neural networks\" to refer to the layers with fully connected network structures. I think it's more appropriate to call them fully connected layers as there are ReLU non-linearities. The linear neural network should refer to the networks without non-linearities.\n\n## Post Rebuttal Update\nThanks to the authors for answering my questions in the rebuttal. After reading the rebuttal and discussing it with other reviewers, I decide not to change my rating of the paper. The major concern is that the approach is quite related to the Proxy Risk work. I suggest the authors give more analysis about the similarities and differences between the two works in the next version of the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "They are properly addressed.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "R_9qlhFQNew", "reviewer": "Reviewer_o4cm", "summary": "- The paper considers two problems: unsupervised accuracy estimation and error detection, both in out-of-distribution settings.\n- The authors propose a practical approach, where they train a ‘check model’ h using self-training and ensembling. Points on which the prediction model and the check model disagree on are predicted as misclassified.\n- The theoretical analysis identifies conditions under which the approach should work and presents a bound on the accuracy estimation error\n- The authors empirically show that their algorithm outperforms baselines for both unsupervised accuracy estimation and error detection", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- Overall, the proposed approach is well-motivated, supported by theoretical analysis, and has thorough experiments. My main comments are about potential areas of improvements in the experiments.\n--------------\n\n- Empirically, the proposed approach outperforms many existing baselines on multiple datasets. Beyond the benchmark numbers, the paper also includes thorough experiments with additional empirical analysis, ablations, multiple versions of the algorithm, etc. But I think there are potentially a few caveats with the experiments as described below.\n- The paper has an ablation experiment, where they study the effect of ensembling by changing the number of models in the ensemble. One potential confounding factor is that the check model may be more accurate with more models in the ensemble, and this may be particularly relevant since better-performing check models are expected to work better. Is ensembling important, even when controlled for check model accuracy/size? \n- The paper compares the proposed approach with multiple baselines, but it seems like many of the baselines do not use unlabeled test examples, whereas the proposed approach does (through self-training). Is this correct? Do you think it would be more appropriate to compare with other methods that use unlabeled test examples? For example, https://arxiv.org/abs/2012.05825 also considers detection of misclassified points in OOD/transductive setting, using an ensembling technique.\nAnother ablation I’d be interested in is on the specific self-training procedure proposed. In contrast to the standard self-training methods, the proposed approach pseudolabels only a specific subset of the dataset. How does this compare with standard pseudolabeling + ensembling? \n- The paper says the approach works when the ensemble check model h is either accurate or diverse on the points misclassified by f. It also mentions that self-training can sometimes improve the accuracy of h, or increase the diversity of the predictions on the points misclassified by f but not yet identified by h. To what extent are the empirical gains explained by the high accuracy of h, as opposed to the diversity in the ensemble? \n- The last few bullet points would give me more confidence about the novelty of the paper. My understanding is that the proposed algorithm is a nice combination/modification of well-known techniques, and the work considers a relatively unexplored setting, outperforms existing baselines, and has theoretical analysis that’s new to my knowledge. However, I think the contributions would be more incremental if it turns out that the high accuracy of h is the main reason why the approach works, and the high accuracy of h is achieved by standard self-training and ensembling; training a check model has been proposed before, and self-training and ensembling are well-known ways to improve model performance.\n- In general, how do the prediction accuracies of f and h compare?\n--------------\n\n- The paper is clearly written, and I enjoyed reading it.\n- Additional discussion on how the three assumptions may or may not hold up in practice could be helpful. The paper describes them as minor assumptions that’s likely to hold in practice, but it’s not clear to me why/if that’s the case for most datasets/models.\n- It seems like there is a difference between the analyzed framework and the practical approach, and I wasn’t sure why this was the case. Why does algorithm 1 use majority voting to select R (lines 4 and 5) as opposed to following the procedure in Framework 1 (line 4)?\n----------------------\nMinor comments\n- N seems overloaded between Algorithms 1 and 2\n- I found the discussion of DANN-arch confusing. What’s the difference between typical DNN and DANN-arch without the discriminator head, and what’s the rationale? Is the important part the fact that h is trained with DANN in one, but not the other, rather than the architectural differences? \n- How did you select the hyperparameters? The paper describes the chosen values (e.g., “we set T=5,...”), but it’s unclear how they were selected.\n- Line 301: Figure 3 → Figure 1\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "PxJmr0Y3KUw", "reviewer": "Reviewer_6eDi", "summary": "The paper considers the problem of unsupervised accuracy estimation and error detection of pre-trained classifiers. The contributions consist of (i) a bound on the generalization performance on estimated accuracy and on detected errors and (ii) a solution based on ensemble models. Specifically, the bound on the generalization error takes into account three quantities, namely (i) the error of the ensemble on correctly-classified test instances, (ii) the agreement of the ensemble and the pre-trained model on potentially wrong test instances and (iii) the diversity of the ensemble on mis-classified test instances. The solution based on ensemble models consists of an iterative training strategy, where at each iteration the ensemble is trained to minimise a training surrogate for quantity (i) combined with quantity (ii) and ensuring diversity among ensemble predictions across iterations. Experimental analysis on several benchmarks (including digits, Office-31, CIFAR-10, iWildCam and Amazon Reviews) show the potential of the proposed solution.\n\nOverall, the paper is well written and well structured. The idea of using ensemble strategies in the context of unsupervised accuracy estimation and error detection is novel. However, there are some issues in terms of ORIGINALITY and QUALITY, which need to be addressed and/or discussed (see below for further comments).", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "ORIGINALITY:\nThe idea of using ensemble methods is interesting and novel in the context of unsupervised accuracy estimation and error detection. However, the work is incremental (extending [1] to the ensemble setting) and it is not clear up to which extent we can consider it novel. Can the authors highlight the main algorithmic and theoretical differences with respect to [1], as there is no discussion in the main article?\n\nQUALITY:\nOverall, the paper looks sound (I haven't checked the proof though).\nRegarding the theoretical bound (Theorem 1), the assumptions on the three quantities, namely small error rate, low agreement and high diversity over all training iterations of the ensemble, are quite strong. How can these conditions be guaranteed in practice and for contexts where the test domain has small relation with the training one? Further experimental analysis on benchmarks other than digits (currently shown in the Supplementary) would provide a better support to the theory.\n\nRegarding the experiments, most of the claims about the superiority of the proposed strategy over competitors are overly and improperly stated. In fact, note that in Table 1, the work of [1] has high variance in almost all cases. Are the performance of your solution statistically significant compared to the ones of [1]?\nFurthermore, how are the hyperparameters of the proposed solution (i.e. T, gamma, alpha, N) chosen?\n\nCLARITY:\nThe paper is well written and well structured.\n\nSIGNIFICANCE:\nThe problem is significant and relevant for future deployment of deep learning models to real-world cases. However, the significance of the proposed solution is low, as it is not clear up to which extent the theoretical findings and the experimental observations can provide additional knowledge to the understanding of the problem.\n\n[1] Chuang et al. Estimating Generalization under Distribution Shifts via Domain-Invariant Representations. ICML 2020\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is no paragraph/section addressing the societal impact. Is there a specific reason for that? As far as I understand, the considered problem has a security impact, as being fundamental for the deployment of deep learning models to real-world scenarios.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposed a novel approach that leverages ensembles and self-training for unsupervised accuracy estimation and error detection. All reviewers find the problem setup interesting and the paper well written, with theoretical justification (although relying on strong assumptions) and reasonable empirical support. There are some useful suggestions during the discussion phase, in particular for improving the experimental results.

After a few rounds of interaction during the discussion phase, the committee reached a consensus on the technical contributions: Reviewers agree upon the technical novelty as using ensembles and self-training for improving model disagreement -- a subtle but different approach to the existing work of ProxyRisk, which warrants the novelty of this work; it is worth mentioning that reviewers also note that the use of domain invariant representations, a check model, and disagreement are in a similar spirit to prior work.

An initial disagreement among the committee was on the scope of the experiments: whether a modified version of ProxyRisk with an ensemble is necessary to be included. Although the modification suggested in the discussion phase (by Reviewer 6eDi) deviates from the ProxyRisk algorithm proposed in the original work, the committee agree that this could be viewed as an additional ablation of the role of ensembling vs self-training, which would have made the work stronger. The authors are encouraged to take into consideration of such feedback when preparing a revision.
Decision: Accept (Poster)