Leveraging unlabeled data to predict out-of-distribution performance
Authors: Saurabh Garg
Sivaraman Balakrishnan
Zachary Chase Lipton
Behnam Neyshabur
Hanie Sedghi
Venue: ICLR 2022 Poster (2022)
Keywords: Distribution Shift
OOD error prediction
Deep Learning

Abstract: Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions
that may cause performance drops. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a \emph{threshold} on the model's confidence, predicting accuracy as the fraction of unlabeled examples for which model confidence exceeds that threshold. ATC outperforms previous methods across several model architectures, types of distribution shifts (e.g., due to synthetic corruptions, dataset reproduction, or novel subpopulations), and datasets (\textsc{Wilds}-FMoW, ImageNet, \breeds, CIFAR, and MNIST).  In our experiments, ATC estimates target performance $2\text{--}4\times$ more accurately than prior methods. We also explore the theoretical foundations of the problem, proving that, in general, identifying the accuracy is just as hard as identifying the optimal predictor and thus, the efficacy of any method rests upon (perhaps unstated) assumptions on the nature of the shift. Finally, analyzing our method on some toy distributions, we provide insights concerning when it works.



Published as a conference paper at ICLR 2022
LEVERAGING UNLABELED DATA TO PREDICT OFTT-F-DDISTIIUTIOO PERFORMANCE
Saurabh Garg*
Sivaraman Balakrishnan Carnegie Mellon University sbalakri andreww cmu, edu
Zachary C. Lipton
Cameeiie Mellon University sgarg29andrew. emu, eduu
Carnegic Mellon University
1Lipton(anddee emu, edu
Behnam Neyshabur
Hanie Sedghi
Google Research, Blueshift team com
Google Research, Brain team hsedghi @google hisedhhigoooggee com
neyshabur neyshabur@google. @google.
ABSTRACT
Real-world machine learning deployments are characterized by mismatches be- tween the source (training) and target (test) distributions that may cause perfor- mance drops. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Av- erage Thresholded Confidence (ATC), a practical method that learns threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples for which model confidence exceeds that threshold. ATC outperforms previous methods across several model architectures, types of distribution shifts (e.g.. due to synthetic corruptions, dataset reproduction, novel subpopulations), and datasets (WILDS, ImageNet, BREEDS, CIFAR, and MNIST), In our experiments, ATC estimates target performance 2-4x more accurately than prior methods. We also explore the theoretical foundations of the problem, proving that, in general, identi- fying the accuracy just as hard as identifying the optimal predictor and thus, the efficacy of any method rests upon (perhaps unstated) assumptions on the nature of the shift. Finally, analyzing our method some toy distributions, we provide insights concerning when it works.
INTRODUCTION
Machine learning models deployed in the real world typically encounter examples from previously unseen distributions. While the IID assumption enables us to evaluate models using held-out data from the source distribution (from which training data is sampled), this estimate is no longer valid in presence of a distribution shift. Moreover, under such shifts, model accuracy tends to degrade (Szegedy al. 2014 Recht al. 2019 Koh al. 2021). Commonly, the only data available to the practitioner are a labeled training set (source) and unlabeled deployment-time data which makes the problem more difficult. In this setting, detecting shifts in the distribution of covariates is known to be possible (but difficult) in theory (Ramdas et al. 2015), and in practice (Rabanser et al. 2018). However, producing an optimal predictor using only labeled source and unlabeled target data is well-known to be impossible absent further assumptions David et al. 2010; Lipton et al. 2018).
Two vital questions that remain are: (i) the precise conditions under which we can estimate a clas- sifier's target-domain accuracy; and (ii) which methods are most practically useful. To begin, the straightforward way to assess the performance labeled (target domain) examples and then to evaluate the model on that data. However, collect- ing fresh labeled data from the target distribution prohibitively expensive and time-consuming, especially if the target distribution is non-stationary. Hence, instead of using labeled data, we aim to use unlabeled data from the target distribution, that is comparatively abundant, to predict model performance. Note that in this work, our focus is not to improve performance on the target but, rather, to estimate the accuracy on the target for a given classifier.
model under distribution shift would be to collect
Work done in part while Saurabh Garg was interning Google

---

Published as a conference paper at ICLR 2022
Source Data
CIFAR10
CIFAR100
throshold
ImageNei200
Network Score gleg. Emiogy)
auch that fraction of camples with score above the threshold match accuracy
ImageNet
Incamnec
FMoW-Wilds
-xlix-Wiids
ImageNet
Score
A Amazon-Wlids
CivilCo.WWii
Accuracy: 89.3%
Deployment
MNIST
Target Data
ATC Threshold
Entiy13
Entiy30
ATC
Predicted accuracy is the fraction unlabeled target samples with score above the thresholdd
Nonliving26
Network Score thing Enkros5)
Living17
0.0
0.2
0.4
0.6
0.8
1.0
Rescaled Estimation Enro
ImageNst-R OOD Accurscy: 37.1%
Score
DOC
ATC-MC (Ours) ATC-NE (Ours)
Pred, Acoureay": 38.6%
AC
GDE
Figure 1: Illustration of our proposed method ATC. Left: using source domain validation data, we identify a threshold on a score (e.g. negative entropy) computed on model confidence such that fraction of examples above the threshold matches the validation set accuracy. ATC estimates accuracy on unlabeled target data as the fraction of examples with the score above the threshold. Interestingly, this threshold yields accurate estimates on wide set of target distributions resulting from natural and synthetic shifts. Right: Efficacy of ATC over previously proposed approaches on our testbed with a post-hoc calibrated model. To obtain errors on the same scale, we rescale all errors with Average Confidence (AC) error. Lower estimation error is better. See Table [ exact numbers and comparison on various types of distribution shift. See Sec. 5 for details on our testbed.
Recently, numerous methods have been proposed for this purpose (Deng & Zheng 2021 Chen et al. 2021; Diang et al.] 2021] Deng et al.] 2021| Guillory et al. 2021). These methods either require
calibration on the target domain yield consistent estimates (Jiang al. 2021 Guillory al.
202T) or additional labeled data from several target domains to learn a linear regression function on a
distributional distance that then predicts model performance (Deng et al. 2021 Deng & ZZeng 2021
Guillory et al. 22021). However, methods that require calibration on the target domain typically yield poor estimates since deep models trained and calibrated on source data are not, in general, calibrated on a (previously unseen) target domain (Ovadia et al. 2019). Besides, methods that leverage labeled data from target domains rely on the fact that unseen target domains exhibit strong linear correlation with seen target domains on the underlying distance measure and, hence, can be rendered ineffective when such target domains with labeled data are unavailable (in Sec. S5T. we demonstrate such a failure on wealdworld distribution shift problem). Therefore, throughout the paper, assume access to labeled source data and only unlabeled data from target domain(s).
In this work, we first show that absent assumptions the source classifier the nature of the shift, no method of estimating accuracy will work generally (even in non-contrived settings). To estimate accuracy target domain perfectly, we highlight that even given perfect knowledge of the labeled source distribution (i.e., (r( y)) and unlabeled target distribution (i.e., pr(r)), we need restrictions on the nature of the shift such that we can uniquely identify the target conditional ())))))). Thus, in general, identifying the accuracy of the classifier is as hard as identifying the optimal predictor.
Second, motivated by the superiority of methods that use maximum softmax probability (or logit) of a model for Of--s---itiiiuttinn (OOD) detection (Hendrycks & Gimpel 2016 Hendrycks et al.) 2019), we propose simple method that leverages softmax probability to predict model performance. Our method, Average Thresholded Confidence (ATC), learns a threshold on a score (e.g., maximum confidence or negative entropy) of model confidence on validation source data and predicts target domain accuracy as the fraction of unlabeled target points that receive a score above that threshold. ATC selects a threshold on validation source data such that the fraction of source examples that receive the score above the threshold match the accuracy of those examples. Our primary contribution in ATC is the proposal of obtaining the threshold and observing its efficacy on (practical) accuracy estimation. Importantly, our work takes step forward in positively answering the question raised in Deng & Zheng (202T): I) Deng et al. (2021) about a practical strategy to select a threshold that enables accuracy prediction with thresholded model confidence.
ATC is simple to implement with existing frameworks, compatible with arbitrary model classes, and dominates other contemporary methods. Across several model architectures range of benchmark
2

---

Published as a conference paper at ICLR 2022
vision and language datasets, we verify that ATC outperforms prior methods by least 2-4x in predicting target accuracy on a variety of distribution shifts. In particular, we consider shifts due to common corruptions (e.g., ImageNet-C), natural distribution shifts due to dataset reproduction (e.g., ImageNet-v2, ImageNet-R), shifts due to novel subpopulations (e.g., BrEEDS), and distribution shifts faced in the wild (e.g.. WI.DS).
As a starting point for theory development, we investigate ATC on a simple toy model that models distribution shift with varying proportions of the population with spurious features, in Nagarajan et al. (2020). Finally, we note that although ATC achieves superior performance in our empirical evaluation, like all methods, it must fail (returns inconsistent estimatess on certain types of distribution shifts, per our impossibility result.
2 PRIOR WORK
Out-of-disttibuttin detection. The main goal of OOD detection is to identify previously unseen examples, i.c., samples out of the support of training distribution. To accomplish this, modern methods utilize confidence features learned by a deep network trained on some source data. [Hendryeks & Gimpel (2016); Geifman & EI- Yaniv (2017) used the confidence score of an (already) trained deep model identify OOD points. Lakshminarayanan al. (2016) use entropy of an ensemble model to evaluate prediction uncertainty on OOD points. To improve OOD detection with model confidence, Liang et al. (2017) propose to use temperature scaling and input perturbations. Diang et al. (2018) scores based on the relative distance of the predicted class to the second class. Recently, residual flow-based methods were used to obtain a density model for OOD detection (Zhang et al. 2020). Ji et al. (22T)) proposed method based on subfunction error bounds to compute unreliability per sample. Reter to Ovadia et al.) (201)) Ji et al. (2021) for an overview and comparison of methods for prediction uncertainty on OOD data.
propose
Predicting model generalization.
Understanding generalization capabilities of overparameterized models in-distribution data using conventional machine learning tools has been focus of long line of work; representative research includes Neyshabur et al. (2015) ; Neyshabur (2017)
Neyshabur al. (2018); Dziugaite & Roy (2017); Bartlett et al. (2117): Zhou et al. (2018); Long & Sedghi (2019) ; Nagarajan & Kolter (2019a). At a high level, this line of research bounds the
generalization gap directly with complexity measures calculated on the trained model. However, these
bounds typically remain numerically loose relative to the true generalization error (Zhang et al. 2016
Nagarajan & Kolter, 2019b). On the other hand, another line of research departs from complexity- based approaches to use unseen unlabeled data to predict in-distribution generalization (Platanios) ct al. 2016 2017 Garg et al., 202T) Diang et al. 202T).
Relevant to our work are methods for predicting the error of a classifier on OOD data based on unlabeled data from the target (OOD) domain. These methods can be characterized into two broad categories: (i) Methods which explicitly predict correctness of the model on individual unlabeled points (Deng & Zheng, 2021 Jiang et al. 2021 Deng et al. 202T); and (ii) Methods which directly
obtain estimate of eтror with unlabeled OOD data without making a point-wise prediction (Chen
ct al. 2021 Guillory et al. 2021 Chuang et al. 2020).
To achieve consistent estimate of the target accuracy, Jiang et al. (2021): ) Guillory et al. 22221)
require calibration on target domain. However, these methods typically yield poor estimates as deep models trained and calibrated on some source data are seldom calibrated on previously unseen domains (Ovadia et al., 2019). Additionally, Deng & Zheng (202]); Guillory et al.) (021) derive model-based distribution statistics on unlabeled target set that correlate with the target accuracy and propose to use subset of labeled target domains to learn (linear) regression function that predicts model performance. However, there are two drawbacks with this approach: (i) the correlation of these distribution statistics can vary substantially as we consider different nature of shifts (refer to Sec. [.1 where we empirically demonstrate this failure); (ii) even if there exists (hypothetical) statistic with strong correlations, obtaining labeled target domains (even simulated ones) with strong correlations would require significant a priori knowledge about the nature of shift that, in general, might not be available before models are deployed in the wild. Nonetheless, in our work, we only assume access to labeled data from the source domain presuming no access to labeled target domains or information about how to simulate them.
Moreover, unlike the parallel work of Deng et al. 20 we do not focus on methods that alter the training on source data to aid accuracy prediction on the target data. CCen et al. (202)) propose an
3

---

Published as a conference paper at ICLR 2022
importance re- weighting based approach that leverages (additional) information about the axis along which distribution is shifting in form of "slicing functions". In our work, we make comparisons with importance re-weighting baseline from Chen et al. (202I) as we do not have any additional information about the axis along which the distribution is shifting.
3 PROBLEM SETUP
Notation. By I-ll, and (t,-) we denote the Euclidean norm and inner product, respectively. For vector e' E R² we use By to denote its jth entry, and for an event E we let [ denote the binary indicator of the event.
Suppose we have a multi-class classification problem with the input domain X' = Rd and label space y' 1.2.....kk) For binary classification, use y' {0,1) J. By DS and DT we denote source and target distribution over X' x V. For distributions 2 and DJ we define ps or pr as the corresponding probability density (or mass) functions. A dataset == {(21,, 9;) contains n points sampled i.i.d. from DS. Let F be a class of hypotheses mapping X to A where A k-1 is a simplex in At dimensions. Given a classifier f E F and datum (ær, y), we denote the 0-1 error (i.e., classification error) on that point by E(f(2), y) == [ & arg max fz(]]] Given a model f F, our goal in this work is to understand the performance of f on D without access to labeled data from Dt Note that our goal is not to adapt the model to the target data. Concretely, we aim to predict accuracy of f on DT. Throughout this paper, we assume we have access to the following: (i) model f: (ii) previously- from target distribution DT.
(D$)r
(validation) data from DS; and (iii) unlabeled data
3.1 ACCURACY ESTIMATION: POSSIBILITY AND IMPOSSIBILITY RESULTS
First, we investigate the question of when it is possible to estimate the target accuracy of an arbitrary classifier, even given knowledge of the full source distribution Pa (x,, y) and target marginal p. (1(). Absent assumptions
the nature of shift, estimating target accuracy is impossible. Even given access to p. (z, y) and Pt (z)), the problem is fundamentally unidentifiable because pr (u(æ) can shift arbitrarily. In the following proposition, we show that absent assumptions on the classifier f (i.e., when f can be any classifier in the space of all classifiers on X',, we can estimate accuracy on the target data iff assumptions on the nature of the shift, logether with p2(2, y) and (x()), uniquely identify the (unknown) target conditional pr(y(z). We relegate proofs from this section to App. A Proposition 1. Absent further assumptions, accuracy on the target is identifiable iff p. (y)a:) is uniquely identified given Pa(2, y) and pr()).
Proposition 1 states that we need enough constraints nature of shift such that (1, y) and pe(xr) identifies unique (v) It also states that under some assumptions on the nature of the shift, we can hope estimate the model's accuracy on target data. We will illustrate this on two common
assumptions made in domain adaptation literature: (i) covariate shift (Heckman) 1977 Shimodaira
2000) and (ii) label shift (Saerens et al. 2002 Zhang et al. 2013 Lipton et al.] 2018). Under
covariate shift assumption, that the target marginal support supp Pe ()) is subset of the source marginal support supp(p, (x)) and that the conditional distribution of labels given inputs does not change within support, i.e., (=(2)=))
(y)l))) which, trivially, identifies a unique target conditional Pe pe(y) x.). Under label shift, the reverse holds, i.e., the class-conditional distribution does not change (p, (xly) pr(zly)) and, again. information about Pr(zz) uniquely determines the target conditional Pe(y(z) (Liptnn et al., 2018 Garg et all, 2020). In these settings, one can estimate an arbitrary the target domain either by using importance re- re-weighting with the ratio Pe (e(=)/p.()) in case of covariate shift or by using importance re- weightttnn with the ratio pe (v(v)p.(y)) in case of label shift. While importance ratios in the former case can be obtained directly when (x())) and p=(x)) are known, the importance ratios in the latter case can be obtained by using techniques from Lipton et al. (2088) Azizzadenesheli et al. (2019). Alexandari al. (0019).
classifier's accuracy
As a corollary of Proposition I we now present a simple impossibility result, demonstrating that no single method can work for all families of distribution shift.
Corollary 1. Absent assumptions on the classifier f, no method of estimating accuracy will work in all scenarios, i.e. different nature of distribution shiftss.
Intuitively, this result states that every method of estimating accuracy on target data is tied up with some assumption the nature of the shift and might not be useful for estimating accuracy under
4

---

Published as a conference paper at ICLR 2022
different assumption on the nature of the shift. For illustration,, consider setting where we have access to distribution ps (2, y) and p((z). Additionally, assume that the distribution can shift only due to covariate shift or label shift without any knowledge about which one. Then Corollary [ says that it impossible to have a single method that will simultaneously for both label shift and covariate shift in the following example (we spell out the details in App
Example 1. Assume binary classification with (2(x) = a ((())) + -1- a) ((22),
Pallyy 0) ②(ui), ps 1) = ((u2), and pr(a) 3 ((m)) + (1- 0(u2) where ((u) = N(u,1), a, 3 E (0. 1). and a * B. Error of classifier / on target
data is given by & = (p,3))) pe(2) (((2) I[f())
* vl under covariate shift and by Eg =
p.
[( [ - 0] + T= - 1) [[(x)) * vl under label shift. In App. A we show
((.)))))()))) that & * Eg for all f. Thus, given access to pz(z, y). and pr(z), any method that consistently estimates error of classifer under covariate shift will give incorrect estimate of error under label shift and vice-versa. The reason is that the same pa and p=(x,, y) can correspond to error & (under covariate shift) error &g (under label shift) and determining which scenario one faces requires further assumptions on the nature of shift.
4 PREDICTING ACCURACY WITH AVERAGE THRESHOLDED CONFIDENCE
In this section, we present our method ATC that leverages black box classifier f and (labeled) validation source data to predict accuracy on target domain given access to unlabeled target data. Throughout the discussion, we assume that the classifier f is fixed.
Before presenting our method, we introduce some terminology. Define a score function 8: A R that takes in the softmax prediction of the function and outputs scalar. We want a score function such that if the score function takes a high value at a datum (I, y) then f is likely to be correct. In this work, we explore two such score functions: (i) Maximum confidence, i.e., s((()) = maxje (z)); and (ii) Negative Entropy, i.e., s( s(f())) = Σ, (x) log(5{(zz)) Our method identifies a threshold t on source data DS such that the expected number of points that obtain score less than match the error of f on DS, i.e.,
E DS (T [s([x))) < []] (2,,)) DS
T
arg max (zxx) a v jey
(1)
and then our error estimate pTCpr(s) on the target domain DT is given by the expected number of target points that obtain score less than t, i.e.,
ATCpr(s) ET [[(([())) <I]]
(2)
In short, in (]. ATC selects a threshold the score function such that the error in the source domain
matches the expected number of points that receive a score below / and in (2). ATC predicts error on the target domain as the fraction of unlabeled points that obtain a score below that threshold t. Note that, in principle, there exists different threshold E' on the target distribution DT such that (I) satisfied on DT. However, in our experiments, the same threshold performs remarkably well. The main empirical contribution of our work is to show that the threshold obtained with (I) might be used effectively in condunction with modern deep networks in a wide range of settings to estimate error on the target data. In practice, to obtain the threshold with ATC, we minimize the difference between the expression on two sides of (T) using finite samples. the next section, we show that ATC precisely predicts accuracy on the OOD data the desired line y = In App. C we discuss alternate interpretation of the method and make connections with OOD detection methods.
5 EXPERIMENTS
We now empirical evaluate ATC and compare with existing methods. In each of our main experiment, keeping the underlying model fixed, we vary target datasets and make prediction of the target accuracy with various methods given access to only unlabeled data from the target. Unless noted otherwise, all models are trained only samples from the source distribution with the main exception of pre-training on a different distribution. We use labeled examples from the target distribution to only obtain true error estimates.
Datasets. First, we consider synthetic shifts induced due to different visual corruptions (c.g., shot noise, motion blur etc.) under ImageNet-C (Hendrycks & Dietterich 2019). Next, consider
5

---

Published as a conference paper at ICLR 2022
CIFAR10
ImageNeet200
Living17
90
Predicted Accuracy
Predicted Accuracy
80
Predicted Accuracy
80
80
70
60
60
60
50
40
40
DOC
DOC GDE
40
20
30
ATC (Ours)
x ATC (Ours)
ATC (Ours)
40
60
80
20 40
60
80
20
40 60
80
OOD Accuracy
OOD Accuracy
OOD Accuracy
Figure 2: Scatter plot of predicted accuracy versus (true) OOD accuracy. Each point denotes different OOD dataset, all evaluated with the same DenseNet121 model. We only plot the best three methods. With ATC (ours), we refer ATC-NE. We observe that ATC significantly outperforms other methods and with ATC, we recover the desired line estimation error in Table D and plots for other datasets and architectures in App. H
with robust linear fit. Aggregated
natural shifts due to differences in the data collection process of ImageNet (Russakovsky et al. 2015 e.g. ImageNetv2 (Recht et al., (2019). We also consider images with artistic renditions of object classes, i.e., ImageNet-R (Hendrycks et al., (021)) and ImageNet-Sketch (Wang et al. 2019). Note that renditions dataset only contains subset 200 classes from ImageNet. To include renditions dataset in our testbed, we include results on ImageNet restricted to these 200 classes (which we call 1mageNet-200) along with full ImageNet.
Second, we consider BREEDS (Santurkar et al. 2020 to assess robustness to subpopulation shifts, in particular, to understand how accuracy estimation methods behave when novel subpopulations not observed during training are introduced. BREEDS leverages class hierarchy in ImageNet to create 4 datasets ENTITY- 13, ENTITY-30, LIVING- 17, NON-LIVING-26.. We focus on natural and synthetic shifts as in ImageNet same and different subpopulations in BREEDs. Third, from WILDS (Koh) (Christie et al. 2018), RXRXI-WILDD (Taylor et al. 2019). Amazon-Wine WILDS (Ni et al. 2019). CivilComments- W ILDS (Borkan et al. 2019) to consider distribution shifts faced in the wild.
ctt al.
T02T) benchmark, we consider
Finally, similar to ImageNet, consider (i) synthetic shifts (CIFAR-10-C) due to common corrup-
tions; and (ii) natural shift (i.e., CIFARv2 (Recht et al., 2018) on CIFAR-10 (Krizhevsky & Hinton we just have synthetic shifts due to common corruptions. For completeness. we also consider natural shifts on MNIST (LeCun et al. 1998) as in the prior work (Deng & Zheng
2009). On CIFAR-
2021). We use three real shifted datasets, i.e., USPS [Hull]1994] SVHN (Netzer al. 201 and
QMNIST (Yadav & Bottou 2019). We give detailed overview of our setup in App.
Architectures and Evaluation. For ImageNet, BREEDS, CIFAR, FMo WILDS, RILXI-WILDS datasets, we use DenseNet121 (Huang et al. (017)) and ResNet50 (He et al. 2016) architectures. For Amazon-WiLLe and CivilComments-WV WILDS fine-tune a DistiBEERT-bas-umme
(Sanh et al.)
2019) model. For MNIST, we train fully connected multilayer perceptron. We use standard training with benchmarked hyperparameters. To compare methods, we report average absolute difference between the true accuracy on the target data and the estimated accuracy on the same unlabeled examples. We refer to this metric as Mean Absolute estimation Error (MAE). Along with MAE, also show scatter plots to visualize performance at individual target sets. Refer to App. G for additional details on the setup.
Methods With ATC-NE, we denote ATC with negative entropy score function and with ATC-MC, we denote ATC with maximum confidence score function. For all methods, we implement post-hoc calibration validation source data with Temperature Scaling (TS; Guo et al. (2017). Below briefly discuss baselines methods compared in our work and relegate details to App.
Average Confidence (AC). Error is estimated as the expected value of the maximum softmax confidence on the target data, i.e, ACp: Ez-pr Dr [maxjey [z)]]
Difference Of Confidence (DOC). We estimate error on target by subtracting difference of confidences on source and target (as a surrogate to distributional distance Guillory et al. (202I) from the error on
source distribution, i.e. DOCpT Ez-ps [I [arg max f5(x) # Er pr maxxey [z()
Egg-s [maxjey fz(z:)] This is referred to DOC-Feat in (Guillory et al. 2021

---

Published as a conference paper at ICLR 2022
IM
DOC
GDE
ATC-MC (Ours)
ATC-NE (Ours)
Dataset
Shift
Pre T
Post T
Pre T
Post T
Pre T
Post T
Post T
Pre T
Post
Pre T
Post T
Natural
6.60
5.74
9.88
6.89
7.25
6.07
4.77
3.21
3.02
2.99
2.85
CIFARIO
Synthetie
12.33
10.20
16.50
11.91
13.87
11.08
6.55
4.65
4.25
4.21
3.87
CIFARI00
Synthttic
13.69
11.51
23.61
13:10
14.60
10.14
9.85
5.50
4.75
4.72
4.94
Natural
12.37
8.19
22.07
8.61
15.17
7.81
5.13
4.37
2.04
3.79
1.45
ImageNer200
Synthetic
19.86
12.94
32.44
13:35
25.02
12.38
5.41
5.93
3.09
5.00
2.68
Natural
7.77
6.50
18.13
6.02
8,13
5.76
6.23
3.88
2.17
2.06
0,80
ImageNet
Synthttic
13.39
10.12
24.62
8.51
13.55
7.90
6.32
3.34
2.53
2.61
4.89
FMOW-WILDS
Natural
5.53
4.31
33.53
12.84
5.94
4.45
5.74
3.06
2.70
3.02
2.72
RXRxI-WILDS
Natural
5.80
5.72
7.90
4.84
5.98
5.98
6,03
4.66
4.56
4.41
4.47
Amazon-WILe
Natural
2.40
2.29
8.01
2.38
2,40
2.28
17.87
1.65
1.62
1.60
1.59
CivilCom. WILDS
Natural
12.64
10.80
16.76
11.03
13.31
10.99
16.65
7.14
MNIST
Natural
18.48
15.99
21.17
14.81
20.19
14.56
24.42
5.02
2.40
3.14
3.50
Same
16.23
11.14
24.97
10.88
19.08
10.47
10.71
5.39
3.88
4.58
4.19
EN
Novel
28.53
22.02
38.33
21.64
32.43
21.22
20.61
13.58
10.28
12.25
6.63
Same
18.59
14.46
28.82
14.30
21.63
13.46
12.92
9.12
7.75
8.15
ENTITY-30
Novel
32.34
26.85
44.02
26.27
36.82
25.42
23.16
17.75
14.30
15.60
10.57
Same
18.66
17.17
26.39
16.14
19.86
15.58
16.63
10.87
10.24
10.07
10.26
NONLIVING-26
Novel
33.43
31.53
41.66
29.87
35.13
29.31
29.56
21.70
20.12
19.08
18.26
Same
12.63
11.05
18.32
10.46
14.43
10.14
9.87
4.57
3.95
3.81
4.21
LIVING-17
Novel
29.03
26.96
35.67
26.11
31.73
25.73
23.53
16.15
14.49
12.97
11.39
Table Mean Absolute estimation Error (MAE) results for different datasets in our setup grouped by the nature of shift. 'Same' refers to same subpopulation shifts and 'Novel' refers novel subpopulation shifts. We include details about the target sets considered in each shift Table Post T denotes use of TS calibration source. Across all datasets, we observe that ATC achieves superior performance (lower MAE is better). For language datasets, we use Dist-IBERT-ass-uuccaad, for vision dataset we report results with DenseNet model with the exception of MNIST where we use FCN. We include results on other architectures in App. H. For GDE post and pre estimates match since TS doesn't 't alter the argmax prediction. Results reported by aggregating MAE numbers over different seeds. We include results with standard deviation values in Table
Importance re-weighting (IM). We estimate the error of the classifier with importance re-weighting of 0-1 error in the pushforward space of the classifier. This corresponds to MANDOLIN using one slice based on the underlying classifier confidence CanfidenceCCheneeee Chen al. (202T).
Generalized Disagreement Equality (GDE). Error is estimated as the expected disagreement of two models (trained on the same training set but with different randomization) on target data (Jiang et al.) 2021). i.e., GDE = E [x] * f'(x)]] where f and f' are the two models. Note that GDE requires two models trained independently, doubling the computational overhead while training.
5.1 RESULTS
In Table we report MAE results aggregated by the nature of the shift in our testbed. In Fig. and Fig. right). we show scatter plots for predicted accuracy versus OOD accuracy several datasets. We include scatter plots for all datasets and parallel results with other architectures in App. H In App.H..I we also perform ablations on CIFAR using a pre-trained model and observe that pre-training doesn't 't change the efficacy of ATC.
We predict accuracy on the target data before and after calibration with TS. First, we observe that both ATC-NE and ATC-MC (even without TS) obtain significantly lower MAE when compared with other methods (even with TS). Note that with TS we observe substantial improvements in MAE for all methods. Overall, ATC-NE (with TS) typically achieves the smallest MAE improving by more than 2x CIFAR and by 3-4x ImageNet over GDE (the next best alternative to ATC). Alongside, we also observe that a linear fit with robust regression (Siegel) 1982) on the scatter plot recovers line close to x y for ATC-NE with TS while the line far away from y for other
7

---

Published as a conference paper at ICLR 2022
90
90
100
Prodicted Accuraty
80
Predicted Accuracy
80
90
70
70
Arcurry
30
60
60
True
50
50
70
XTC (Oun)
40
Dill-suhpopulatiin
40
DOC (w/o
DOC (w/ m)
50
DOC
CDE
30
Same subpopulation
30
ATC (w/l
20
20
50
80
0.0
0.2
0.4
06
08
1.0
OOD Accuracy
OOD Accuracy
Spurious correlation (pgg)
Figure 3: Left: Predicted accuracy with DOC on Living 17 BREEDS dataset. We observe substantial gap in the linear fit of same and different subpopulations highlighting poor correlation. Middle: After fitting a robust linear model for DOC on same subpopulation, we show predicted accuracy on different subpopulations with fine-tuned DOC (i.e., DOC (w/ fit) and compare with ATC without any regression model, i.e., ATC (w/o fit). While observe substantial improvements in MAE from 24.41 with DOC (w/o fit) to 23.26 with DOC (w/ fit), ATC (w/o fit) continues to outperform even DOC (w/ fit) with MAE 10.22. We show parallel results with other BREEDS datasets in App. H.2 Right: Empirical validation of our toy model. We show that ATC perfectly estimates target performance as we vary the degree of spurious correlation in target. 'x' represents accuracy on source.
methods (Fig. and Fig. Tright).. Remarkably, MAE is in the range of 0.4-5.8 with ATC for CIFAR, ImageNet, MNIST, and Wilds. However, MAE is much higher on BREEDS benchmark with novel subpopulations. While we observe a small MAE (i.e., comparable to our observations on other datasets) on BREEDS with natural and synthetic shifts from the same sub-population, MAE on shifts with novel population is significantly higher with all methods. Note that even novel populations, ATC continues to dominate all other methods across all datasets in B BREEDS.
Additionally, for different subpopulations in BREEDS setup, we observe poor linear correlation of the estimated performance with the actual performance as shown in Fig. 3 (left)(we notice a similar gap in the linear fit for all other methods). Hence in such setting, would expect methods that fine-tune a regression model on labeled target examples from shifts with one subpopulation will perform poorly on shifts with different subpopulations. Corroborating this intuition, next, we show that even after fitting regression model for DOC on natural and synthetic shifts with source subpopulations, ATC without regression model continues outperform DOC with regression model on shifts with novel subpopulation.
Fitting a regression model on BREEDS with DOC. Using label target data from natural and synthetic shifts for the same subpopulation (same as source), we fit a robust linear regression model (Siegel 1982) to fine-tune DOC as in Guillory et al. (202T). We then evaluate the fine-tuned DOC (i.e.. DOC with linear model) on natural and synthetic shifts from novel subpopulations on BREEDS benchmark. Although we observe significant improvements
the performance of fine- tuned DOC when compared with DOC (without any fine-tuning), ATC without any regression model continues to perform better (or similar) to that of fine-tuned DOC on novel subpopulations (Fig. B (middle)). Refer App. H.2 for details and Table for MAE on BREEDS with regression model.
6 INVESTIGATING ATC ON TOY MODEL
In this section, we propose and analyze a simple theoretical model that distills empirical phenomena from the previous section and highlights efficacy of ATC. Here, our aim is not to obtain a general model that captures complicated real distributions on high dimensional input space as the images in ImageNet. Instead to further our understanding, we focus on an casy-to-learn binary classification task from Nagarajan et al. (2020) with linear classifiers, that is rich enough to exhibit some of the same phenomena as with deep networks on real data distributions.
Consider easy-to-learn binary classification problem with two features x = [Time Txp] E R where Tiny is fully predictive invariant feature with a margin 7 > O and Tapp e {-1. is a spurious feature (i.e., a feature that is correlated but not predictive of the true label). Conditional on y, the distribution over Tinv is given as follows: Tin (yy 1) UL>, c] and Zinv (1 0) UL-c, -7]. where is fixed constant greater than 7. For simplicity, we assume that label distribution on source is uniform on 1-1, 1]. Tspp is distributed such that P. (2y - 1) > 0] Psp. where Psp (0.5, 1.0) controls the degree of spurious correlation. To model distribution shift, we simulate target data with different degree of spurious correlation, i.e., in target distribution P. [T. . (2y - 1) 0] r'p [0, 1]. Note
8

---

Published as a conference paper at ICLR 2022
that here we do not consider shifts in the label distribution but our result extends to arbitrary shifts in the label distribution as well.
In this setup, we examine linear sigmoid classifiers of the form f(z) = 1+e I+ew*= where w. = [Rimv, Wapl e IR? While there exists linear classifier with w = [1, 0] that correctly classifies all the points with a margin 7, Nagarajan et al. (2020) demonstrated that a linear classifier will typically have a dependency on the spurious feature, I.e., Wap * O. They show that due to geometric skews, despite having positive dependencies on the invariant feature, a max-margin margin classifier trained on finite samples relies on the spurious feature. Refer to App. A for more details these skews. In our work, we show that given a linear classifier that relies on the spurious feature and achieves a non-trivial performance on the source (i.e., Wanv >> 0), ATC with maximum confidence score function consistently estimates the accuracy on the target distribution.
Theorem 1 (Informal). Given any classifier with Win > 0 in the above setting, the threshold obtained in together with ATC as in (2) with maximum confidence score function obtains a consistent estimate of the target accuracy.
Consider a classifier that depends positively on the spurious feature (i.e., Wsp > 0). Then as the spurious correlation decreases in the target data, the classifier accuracy the target will drop and vice-versa if the spurious correlation increases on the target data. ssoows that the threshold identified with ATC as in ① remains invariant as the distribution shifts and hence ATC as in will correctly estimate the accuracy with shifting distributions. Next, we illustrate Theorem by simulating the setup empirically. First we pick arbitrary classifier (which can also be obtained by training on source samples), tune the threshold on hold-out source examples and predict accuracy with different methods as we shift the distribution by varying the degree of spurious correlation.
Empirical validation and comparison with other methods. Fig. Bright) shows that as the degree of spurious correlation varies, our method accurately estimates the target performance where all other methods fail to accurately estimate the target performance. Understandably. due to poor calibration of the sigmoid linear classifier AC, DOC and GDE fail. While in principle IM can perfectly estimate the accuracy on target in this case, we observe that it is highly sensitive to the number bins and choice of histogram binning (i.e., uniform mass or equal width binning). We elaborate more on this in App. D
Biased estimation with ATC. Now we discuss changes in the above setup where ATC yields inconsistent estimates. We assumed that both in source and target rimw ly = 1 is uniform between [7, cl and æly 1 is uniform between [-c, - Shifting the support of target class conditional pa ( may introduce a bias in ATC estimates, e.g., shrinking the support to ea c1(<< c) (while maintaining uniform distribution) in the target will lead to an over-estimation of the target performance with ATC. In App. [.I] we elaborate on this failure and present a general (but less interpretable) classifier dependent distribution shift condition where ATC is guaranteed to yield consistent estimates.
7 CONCLUSION AND FUTURE WORK
In this work, we proposed ATC, simple method for estimating target domain accuracy based on unlabeled target (and labeled source data). ATC achieves remarkably low estimation error on several synthetic and natural shift benchmarks in our experiments. Notably, our work draws inspiration from recent state-of tftte-ofttheaar the- methods that use softmax confidences below a certain threshold for OOD detection (Hendrycks & Gimpel| 2016 Hendrycks et al. 2019) and takes a step forward in answering questions raised in Deng & Zheng (2021) about the practicality of threshold based methods.
Our distribution shift toy model justifies ATC on an easy-to-learn binary classification task. In our experiments, we also observe that calibration significantly improves estimation with ATC. Since in binary classification, post hoc calibration with TS does not change the effective threshold, in future work, we hope to extend our theoretical model to multi-class classification to understand the efficacy of calibration. Our theory establishes that a classifier's 's accuracy is not, in general identified, from labeled source and unlabeled target data alone, absent considerable additional constraints on the target conditional pe ).) In light of this finding, we also hope to extend our understanding beyond the simple theoretical toy model to characterize broader sets of conditions under which ATC might be guaranteed to obtain consistent estimates. Finally, we should note that while ATC outperforms previous approaches, it still suffers from large estimation error datasets with novel populations, e.g.. BREEDS. We hope that our findings can lay the groundwork for future work for improving accuracy estimation such datasets.

---

Published as a conference paper at ICLR 2022
Reproducibility Statement We have been careful to ensure that our results are reproducible. We have stored all models and logged all hyperparameters and seeds to facilitate reproducibility. Note that throughout work, do not perform any hyperparameter tuning, instead, using benehmarked hyperparameters and training procedures to make our results easy to reproduce. While, we have not released code yet, the appendix provides all the necessary details replicate experiments and results. Moreover, we plan to release the code with revised version of the manuscript.
ACKNOWLEDGEMENT
Authors would like to thank Ariel Kleiner and Sammy Jerome the problem formulation and motivation this paper was highly influenced by initial discussions with them.
REFERENCES
Amr Alexandari, Anshul Kundaje, and Avanti Shrikumar. Adapting label shift with bias-corrected calibration. In arXiv preprint (iX.::9901.0330 2019.
Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang, and Animashree Anandkumar. Regularized learning for domain adaptation under label shifts. In International Conference on Learning Representations (ICLR), 2019.
Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalie margin bounds for neural networks. Advances in neural information processing systems, pp. 6240-6249, 2017.
Shai Ben-David, Tyler Lu, Teresa Luu, and Dávid Pál. Impossibility Theorems for Domain Adaptation. In International Conference on Artificial Intelliencee and Statistics (AISTA7S), 2010.
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. Companion Proceedings of The 2019 World Wide Web Conference, 2019.
Mayee Chen, Karan Goel, Nimit Sohoni, Fait Poms, Kayvon Fatahalian, and Christopher Ré. Mandoline: Model evaluation under distribution shift. In International Conference Machine Learning, pp. 1617-1629. PMLR, 2021.
Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
Ching- Yao Chuang, Antonio Torralba, and Stefanie Jegelka. Estimating generalization under distri- bution shifts via domain-invariant representations. arXiv preprini urXiv:2007.0351/1 2020.
Weijian Deng and Liang Zheng. Are labels always necessary for classifier accuracy evaluation? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15069-15078, 2021.
Weijian Deng, Stephen Gould, and Liang Zheng. What does rotation prediction tell us about classifier accuracy under varying testing environments? arXiv preprint 3xi::2006.05961 2021.
Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv: 1008, 2017.
Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan, and Zachary C Lipton. unified view of label shift estimation. arXiv preprint 07X1:22003.07544 2020.
Saurabh Garg, Sivaraman Balakrishnan, J Zico Kolter, and Zachary Lipton. Ratt: Leveraging unlabeled data to guarantec generalization. arXiv preprint arXiv:2105.0030 2021.
Yonatan Geifman and Ran El- Yaniv. Selective classification for deep neural networks. arXiv preprint 17X5::1705.08500, 2017.
10

---

Published as a conference paper at ICLR 2022
Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. Predicting with confidence on unseen distributions. arXiv preprint arXiv:2107.03315, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference Machine Learning (ICML), 2017.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image
Recognition. In Computer Vision and Pattern Recognition (CVPR), 2016.
James J Heckman. Sample Selection Bias as a Specification Error (With an Application to the Estimation of Labor Supply Functions), 1977.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv: 12261, 2019.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of-distributioo examples in neural networks. arXiv preprint arXiv: 2016.
Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. arXiv preprint arXiv:1911.11132. 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of 2021.
generalization. ICCV,
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
Jonathan J. Hull. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-554, 1994.
Xu Ji, Razvan Pascanu, Devon Hjelm, Andrea Vedaldi, Balaji Lakshminarayanan, and Yoshua Bengio. Predicting unreliable predictions by shattering a neural network. arXiv preprint arXiv:2106.08365, 2021.
Heinrich Jiang, Been Kim, Melody Y Guan, and Maya R Gupta. To trust not to trust classifier. In NeurlPS, pp. 5546-5557, 2018.
Yiding Jiang, Vaishnavh Nagarajan, Christina Back, and Zico Kolter. Assessing generalization of sgd via disagreement. arXiv preprint 10t.::20006.39999 2021.
Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv Preprint arXiv:1412.6980 2014.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsub- ramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton A. Earnshaw, Imran Haque, Sara Beery, Jure Leskovee, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning (ICML), 2021.
Alex Krizhevsky and Geoffrey Hinton. Learning Multiple Layers of Features from Tiny Images. Technical report, Citeseer, 2009.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint 1rX1v:1612.01144, 2016.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86, 1998.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of dfistrriiitt image detection in neural networks. arXiv preprint arXiv: 1r06iv//00.0266000 2017.

---

Published as a conference paper at ICLR 2022
Zachary C Lipton, Yu-Xiang Wang, and Alex Smola. Detecting and Correcting for Label Shift with Black Box Predictors. In International Conference on Machine Learning (ICML), 2018.
Philip Long and Hanie Sedghi. Generalization bounds for deep convolutional neural networks. arXiv preprint /905. 2019.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
Vaishnavh Nagarajan and Zico Kolter. Deterministic pac-bayesian generalization bounds for deep networks via generalizing noise-resilience. arXiv preprint 0rXhv:1905.13344 2019a.
Vaishnavh Nagarajan and J Zico Kolter. Uniform convergence may be unable to explain generalization in deep learning. In Advances in Neural Information Processing Systems, pp. 1661--1626, 2019b.
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of out-of-distribution generalization. arXiv preprint arXiv:2010.15771 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In Advances in Neural Information Processing Systems (NIPS), 2011.
Behnam Neyshabur. Implicit regularization in deep learning. arXiv preprint arXiv:1709.0195; 2017.
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. Norm-based capacity control in neural networks. In Conference on Learning Theory, pp. 1376-1401, 2015.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nathan Srebro. Exploring general- ization in deep learning. arXiv preprini /rXiv:/706.00 77060099777 2017.
Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro. The role of over-paramettizztion in generalization of neural networks. In International Conference on Learning Representations, 2018.
Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-graing une-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language P Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Scbastian Nowozin, Joshua V Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. arXiv preprint arXiv: 02530, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Veto,, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, 2019.
Emmanouil A Platanios, Hoifung Poon, Tom M Mitchell, and Eric Horvitz. Estimating accuracy from unlabeled data: A probabilistic logic approach. arXiv preprint arXiv: 1705.07086 2017.
Emmanouil Antonios Platanios, Avinava Dubey, and Tom Mitchell. Estimating accuracy from unlabeled data: bayesian approach. In International Conference on Machine Learning, pp. 1416-1425. PMLR, 2016.
Stephan Rabanser, Stephan Günnemann, and Zachary C Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. arXiv preprint 1rxiv:1810.11955, 2018.
Aaditya Ramdas, Sashank Jakkam Reddi, Barnabás Póczos, Aarti Singh, and Larry A Wasserman. On the Decreasing Power of Kernel and Distance Based Nonparametric Hypothesis Tests in High Dimensions. In Association for the Advancement of Artificial Intelligence (AAAI), 2015.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar- 10 classifiers generalize to i0f? arXiv preprint arXiv: XXXiv:88000044 806.00451, 2018
12

---

Published as a conference paper at ICLR 2022
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp- $389-5400 PMLR, 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211-22) 2015.
Marco Saerens, Patrice Latinne, and Christine Decaestecker. Adjusting the Outputs of Classifier to New a Priori Probabilities: A Simple Procedure. Neural Computation, 2002.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108,, 2019.
Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. Breeds: Benchmarks for subpopulation shift. arXiv preprint arXiv:2008. 04859, 2020.
Hidetoshi Shimodaira. Improving Predictive Inference Under Covariate Shift by Weighting the Log-L.keelhhod Function. Journal of Statistical Planning and Inference, 2000.
Andrew F Siegel. Robust regression using repeated medians. Biometrika, 69(1):242 244, 1982.
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing Properties of Neural Networks. In International Conference on Learning Representations (ICLR), 2014.
Taylor, B. Earnshaw, Mabey, M. Victors, and Yosinski. Rxrx1: An image set for cellular morphological variation across many experimental batches. In International Conference on Learning Representations (ICLR), 2019.
Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions Pattern Analysis and Machine Intelligence, 30(1 ():11):110 1958-1970, 2008
Haohan Wang. songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems, pp. 10506-10518, 2019.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger. Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-ar art natural language processing. In Proceedings the 2020 Conference Empirical Methods in Natural Language Processing:
System Demonstrations, pp. 38 45. Association for Computational Linguistics, 2020.
Chhavi Yadav and Léon Bottou, Cold case: The lost mnist digits. In Advances in Neural Information Processing Systems 32, 2019.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv: 161 1.03530, 2016.
Hongjie Zhang, Ang Li, Jie Guo, and Yanwen Guo. Hybrid models for open recognition. In European Conference on Computer Vision, pp. 102-117. Springer, 2020.
Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain Adaptation Under Target and Conditional Shift. International Conference Machine Learning (ICML), 2013.
Wenda Zhou, Vietor Veitch, Morgane Austern, Ryan Adams, and Peter Orbanz. Non-vacuous generalization bounds at the imagenet scale: pac-bayesian compression approach. arXiv preprint 10X15v 2018
13

---

Published as a conference paper at ICLR 2022
APPENDIX
A PROOFS FROM SEC. 3
Before proving results from Sec. we introduce some notations. Define {(f(z),)) =
I y $ arg ja fz(2)]. We express the population evvor on distribution D ED(f)
(( ())
Proof of Proposition L Consider a binary classification problem. Assume P be the set of possible target conditional distribution of labels given ps(x,, y) and (r()..
The forward direction is simple. If P = {p<(ul=)) } is singleton given ps(a, y) and pr(r), then the error of any classifier on the target domain identified and is given by
E (f) = (a--
arg max fz( * jEy
(3)
For the reverse direction assume that given px()) and ps (2r, y), we have two possible distributions 'DT' and D with pe(y)x), p (1)x)) E P such that on some with pe
0, we have (+(l/x) * (1 Consider Xr = fz E X' [x() > 0 and (v 1|x) * p3(3/ = 1|x)} be the set of all input covariates where the two distributions differ. We will now choose classifier such that the eтror on the two
distributions differ. On a subset X3 {ir E ()))(r) 0 and (. (xx) > 16(y 1/rr), assume
f(x) and on a subset X3 {Ir E X)p((=) > and prly 1 [a:) pl(y 1 [z))), assume
f(x) = 1. We will show that the error of f on distribution with (r x:) is strictly greater than the error of f on distribution with pl (v/la) Formally,
Epr (f) (f)
[
E (e
T
arg max fz(x) / jEy
E px
arg max fz(x) * jEy
= =s JNEX
Ilf(x) * (pr 0/2) (v 0/z)) pa (x))drr
[[[(x) * 1] (pr(y 1|x) 2klu = 1|x)) pa pr())d
JIEXM
(prly = -)x) 1 = 0/z)) (x) + [ JEEX]
(pelv = 1/x) - p:(1 1|x)) pe (r())dr
JEER
0,
where the last step follows by construction of the set Xh and X3g Since Epr (f) Epr (f), given the information of (x() and (x. y) it impossible to distinguish the two values of the error with classifier f. Thus, we obtain contradiction must pose restrictions on the nature of shift such that P is singleton to to identify accuracy on the target.
the assumption that p<(ylar) * (y]arr). Hence,
Proof of Corollary 7 The corollary follows directly from Proposition | Since two different target conditional distribution can lead different error estimates without assumptions on the classifier, no method can estimate two different quantities from the same given information. We illustrate this in Example next.
B ESTIMATING ACCURACY IN COVARIATE SHIFT OR LABEL SHIFT
Accuracy estimation under covariate shift assumption Under the assumption that (v(l)))
Pe (yla (r),, accuracy on the target domain can be estimated as follows:
(x. v)
Ep (f) E (ar.))=D
I[f(x) * vl
(5)
(x(x,))
(+(x))
[[f(x) vl
(6)
(s(x)
14

---

Published as a conference paper at ICLR 2022
Given access to pz(2) and p. (z)), one directly estimate the expression in (6)
Accuracy estimation under label shift assumption Under the assumption that pr(zly) = px(x))),
accuracy the target domain can be estimated follows:
Pr Pr v) [[((a)) * w]
Eπr( E (x.y))
(2(2,
(r()) [f(x) # u1
= (Ds
(8)
(v)
Estimating importance ratios Pa(x)//x(x:) is straightforward under covariate shift assumption when the distributions (r( and pa(x) are known. For label shift, one can leverage moment matching
approach called BBSE (Lipton et al. 2018) or likelihood minimization approach MLLS (Garg et al.
2020). Below we discuss the objective of MLLS:
arg max (x) [lo (slx))))
weW
where W = (w Vy: Wy 3 0 and Σi HyPs (1) = 13. MILS objective is guaranteed to obtain consistent estimates for the importance ratios w =
under the following condition. Theorem 2 (Theorem (Garg et al. 2020)). If the distributions (p(x)lu) y = k) are strictly linearly independent. then w* is the unique maximizer of the MLLS objective
p:(v)/p.())) (x)
We refer interested reader to Garg et al. (2020) for details.
Above results of accuracy estimation under label shift and covariate shift can be extended to generalized label shift and covariate shift settings. Assume function h X' such that y is independent of z given h(z). In other words ((x) contains all the information needed to predict label y. With help of h, we can extend estimation to following settings: (i) Generalized covariate shift, i.c., Ps hxz))
(z)) and p. (h(x)) > 0 for all Xg; (ii) Generalized label shift, ie.. p. (h(x)ly) yy) = Ps (h()))))) and (v()) > 0 for all y E V4. By simply replacing x with h(x) in (6) and will obtain consistent error estimates under these generalized conditions.
99)
Proof of Example I. Under covariate shift using 6).
we get
pe(x))
E, (law))pp
[ff()) # y]
(x
(x()))
pa(x)) I[f(x) 11]
P. (x(x) -a
[[f(2) * 0] E Eu (w.w=1
[p((z)
= -f(((()) * + (r) 0/z)dx + [[((()) * 1)p((x)p_(v- = 1/x)dz
Under label shift using
8 we get
Ps (v)
Eg- B(I
11 [x) # y]
(v(
B [ll((x)= # 0]
3 [lf()) #11
+ (z=pa(z.0=1)
- -fIL((0) 0]
B pplly (1)))
[IU()) 1] (1
(1 (1- 2 ps
1 (x))dr
Then E1 Eg is given by
Pr(x) B
& 8
1[f(x) # ()p.(v= = 0/z)
a (x) dx
(z(x) (1- 13)
+ I[f(z) # 1) = 1|x)
(1- - a) (x( dx
[If(x) # 0)ps(y== = 0/2)
(a 3)
6(p2)dz
(a 3)
+ [IF(x)) * 1]pb(y 1|x)
d(11) )dr.
(10)
1 - a
15

---

Published as a conference paper at ICLR 2022
If > B, then & > Eg and if a < B, then & < Ez. Since & # Eg for arbitrary f. given access to Pa(a, y), and (z)), any method that consistently estimates error under covariate shift will give an incorrect estimate under label shift and vice-versa. The reason being that the same (a(x) and (z y) can correspond to error & (under covariate shift) or error &g (under label shift) either of which not discernable absent further assumptions on the nature of shift.
C ALTERNATE INTERPRETATION OF ATC
Consider the following framework: Given a datum (z, y), define a binary classification problem of whether the model prediction arg max fx) was correct or incorrect. In particular, if the model prediction matches the true label, then we assign a label I (positive) and conversely, if the model prediction doesn't 't match the true label then we assign label 0 (negative).
Our method can be interpreted as identifying examples for correct and incorrect prediction based on the value of the score function s (f(())), i.e., if the score (x) greater than or equal to the threshold & then our method predicts that the classifier correctly predicted datum (x,, y) and vice-versa if the score less than f. A method that can solve this task will perfectly estimate the target performance. However, such an expectation is unrealistic. Instead, ATC expects that most of the examples with score above threshold are correct and most of the examples below the threshold are incorrect. More importantly, ATC selects threshold such that the number of falsely identified correct predictions match falsely identified incorrect predictions on source distribution, thereby balancing incorrect predictions. We expect useful estimates of accuracy with ATC if the threshold transfers to target, i.e. the number of falsely identified correct predictions match falsely identified incorrect predictions target. This interpretation relates our method the OOD detection literature where Henddycks & Gimpel] (2016): Hendrycks et al. (2019) highlight that classifiers tend to assign higher confidence in-distribution examples and leverage maximum softmax confidence (or logit) to perform OOD detection.
D DETAILS ON THE TOY MODEL
Skews observed in this toy model In Fig. we illustrate the toy model used in our empirical experiment. In the same setup, we empirically observe that the margin on population with less density large, i.e., margin much greater than 7 when the number of observed samples is small (in Fig. [ (d)). Building on this observation, Nagarajan et al. (2020) showed in cases when margin decreases with number of samples, max margin classifier trained on finite samples bound to depend on the spurious features in such cases. They referred to this skew geometric skew.
Moreover, even when the number of samples are large so that we do not observe geometric skews, Nagarajan et al. (2020) showed that training for finite number of epochs, linear classifier will have non zero dependency on the spurious feature. They referred to this skew as statistical skew. Due both of these skews, we observe that a linear classifier obtained with training for finite steps training data with finite samples, will have a non non-zero dependency on the spurious feature. We refer interested reader to Nagarajan et al. (2020) for more details.
Proof of Theorem I Recall, we consider a easy-to-learn binary classification problem with two features x: [Tane Txpl R²² where Tin is fully predictive invariant feature with a margin y >> 0 and Tap E 1-1, } is spurious feature (i.e., a feature that is correlated but not predictive of the true label). Conditional on y, the distribution over Tiny is given as follows:
[UiTc] cl
==1
Finy y
-U--c
y --
(11)
where e is a fixed constant greater than 7. For simplicity, we assume that label distribution on source is uniform on 1,,, Tep is distributed such that (2y 11y 1)= > 0] Psp. where Pop (0.5, 1.0) controls the degree of spurious correlation. To model distribution shift, we simulate target data with different degree of spurious correlation, i.e., in target distribution P. [ (2y - 1) > 0] = p', [0. 1]. Note that here we do not consider shifts in the label distribution but our result extends to arbitrary shifts in the label distribution well.
16

---

Published as a conference paper at ICLR 2022
Source data
Target Data (1)
1.00
1.00
0.75
Pos Neg
0.75
Pos
0.50
0.50
Neg
0.25
0.25
0.00
0.00
-0.25
-0.25
-0.50
-0.50
-0.75
-0.75
-1.00
-1.00
-10.0-7.5-5.-2255
0.0
5.0
7.5
-10.0-7.5-500
0.0
75
Xinv
Xinv
(a)
(b)
Target Data (2)
Margin on Xin minority
1.00
Pos
0.35
Neg
0.25
Margin
0.00
0.25
-0.25
-0.50
-0.75
0.15
-1.00
10.0-7.5
5500
-25
5.0 0.0
2.5
10.0
200
400
600
800
1000
Xinv
Num samples
(c)
(d)
Figure 4: Illustration of toy model. (a) Source data at n 100. (b) Target data with p. 0.5. (b) Target data with P/, 0.9. (c) Margin of Tay in the minority group in source data. As sample size increases the margin saturates to true margin = 0.1.
In this setup, we examine linear sigmoid classifiers of the form f(x) Ite
where
I+ewrs
w [Win,, Wap] E 2 We show that given a linear classifier that relies on the spurious feature and achieves non-trivial performance the source (i.e., Winy 0), ATC with maximum confidence score function consistently estimates the accuracy on the target distribution. Define XM {xr|r
(2y - 1) < 0} and Xc {2r ssp (2y- 1) 0}. Notice that target distributions, we are changing the fraction of examples in XM and X but we are not changing the distribution of examples within individual set.
Theorem 3. Given any classifier ff with Wine > O in the above setting, assume that the threshold t is obtained with finite sample approximation of ( i.e., t is selected such tha
Σ
max jey
$ Σ
arg max fz(2,) # Vi
(12)
jEy
where (x;; 1/ } (DSyn are samples from source distribution.
Fix 5 O. Assuming
n > log 4/0)/(1 (1- Psp 2 then the estimate of accuracy by ATC as in 2 satisfies the following with probability at least 1 8,
[ arg fz(2) #
log(8/ (8/))
E B [[s(f(z)) <t] - (D
<
(13)
jEy
Cap
where DT is any target distribution considered in our setting and Cup (1 - Pop) Wsp and Cap Pap otherwise.
Note that this is possible because linear classifier with sigmoid activation assigns unique score to each point in distribution.
17

---

Published as a conference paper at ICLR 2022
Proof. First consider the case of Up 0. The proof follows in two simple steps. First we notice that the classifier will make an eтror only on some points in XM and the threshold t will be selected such that the fraction of points in X with maximum confidence less than the threshold will match the error of the classifier on XM. Classifier with Wsp > 0 and Winy > will classify all the points in X correctly. Second, since the distribution of points is not changing within XM and Xc, the same threshold continues to work for arbitrary shift in the fraction of examples in XM, i.c., Pag
Note that when ssp > 0, the classifier makes no error on points in X and makes an error on a subset Xar { stp (2y- - 1) < 0& (iin-Tinn + Wsppss) (2y - 1) & 0} of XM. i.e., Xen E X Consider Xutess = {2| arg yey fz(z) & t} as the set of points that obtain a score less than or equal to t. Now
will show that ATC chooses threshold such that all points in X gets score
above 1, i.e., Xihres S XM- First note that the score of points close to the true separator in Xc. i.e., at 31 (7; 1) and 22 (-7,- -1) match. In other words, score x1 matches with the score of 22 by symmetricity, i.e.
e e U:pp
arg max fz arg max (x22 yey
(14)
(1 + eWwnrttw)))
yey
Hence, i 3 arg max y fy (21) then we will have [Xanl < [ which is contradiction violating definition of as (2). Thus Xthres XMM.
Now we will relate LHS and RHS of (12) with their expectations using Hoeffdings and DKW inequality to conclude (33. Using Hoeffdings' bound, we have with probability at least 1 8/4
arg max jey (1;) o us]l B(E,))
loo( (8/0)
[L
[I
arg max f5(x) y jEy
<
[
[x
TEXM
(15)
With DKW inequality, we have with probability at least - 6/4
II [maxjjy fz()))
t'll
log 8/6) X
(
I
max fz(z) < jey
(16)
[M
MEXM
for all t' > O. Combining
15 and 16) at 1' = f with definition
12 we have with probability at
least 1 - 5/2
arg max f # y w
log (8/(8/)) 2 XM1
E [[s(f(z)) $11 - E (aa)) Dr
I
(17)
Jey
Now for the case of Wsp < 0, we can use the same arguments Xe. That is, since now all the error will be on points in XC and classifier will make no error XMs we can show that threshold f will be selected such that the fraction of points in Xc with maximum confidence less than the threshold will match the error of the classifier Xc. Again, since the distribution of points is not changing within Xa and Xc, the same threshold continues to work for arbitrary shift in the fraction of examples in Xar, i.e., Pap Thus with similar arguments, we have
EA [[)(f(z)) < t]] ((r,)) Dr
[
arg arg max fz(z) # V JeY
1
log(8/8) 2 Xeel
(18)
Using Hoeffdings' bound, with probability least 1 - -5/2, we have
XM - n (1 Psp)l < log 4/6)
(19)
2
With probability least 1 5/2, we have
n log (4/0)
X n Popl E
(20)
2
Combining 19 and (7]. we get the desired result for wap > 0. For Wap < 0, we combine (18) to get the desired result.
20 and
18

---

Published as a conference paper at ICLR 2022
100
90
Accuracy
80
70
60
True Accuracy
ATC (Our)
(a)
Figure 5: Failure of ATC in our toy model. Shifting the support of target class conditional pa pr( ly) may introduce a bias in ATC estimates, e.g., shrinking the support cj(< c) (while maintaining uniform distribution) in the target leads to overestimation bias.
Issues with IM in toy setting described in App. E. we observe that IM is sensitive binning strategy. In the main paper, we include IM result with uniform mass binning with 100 bins. Empiri- cally, we observe that we recover the true performance with IM we use equal width binning with number of bins greater than 5.
Biased estimation with ATC in our toy model We assumed that both in source and target Tin uniform between [y, c] and æryy - is uniform between [-c, Shifting the support of target class conditional Pe( (Tiiw])) may introduce a bias in ATC estimates, e.g., shrinking the support to cx(< e) (while maintaining uniform distribution) in the target will lead to over-estimation of the target performance with ATC. We show this failure Fig. 5 The reason being that with the same threshold that we see more examples falsely identified correct compared examples falsely identified as incorrect.
D.1 A MORE GENERAL RESULT
Recall, for given threshold we categorize example (xr, y) falsely identified correct prediction (ficp) if the predicted label δ = arg max f(x) is not the same as but the predicted score fg(x) is greater than Similarly, example is falsely identified incorrect prediction (fiip) if the predicted label ÿ is the same as but the predicted score fz(r) is less than t.
In general, believe that our method will obtain consistent estimates in scenarios where the relative distribution of covariates doesn't change among examples that are falsely identified as incorrect and examples that are falsely identified as correct. other words, ATC is expected to work the distribution shift is such that falsely identified incorrect predictions match falsely identified correct prediction.
D.2 ATC PRODUCES CONSISTENT ESTIMATE SOURCE DISTRIBUTION
from Proposition 2. Given labeled validation data {(I,, v.) from distribution DD and model f. choose a threshold L as in (1). Then for O, with probubility at least 1 ô, we have
log 4/5)
E ((
fz(2) jey
arg fz) * v JEY
2
(21)
2n
Proof. The proof uses (i) Hoeffdings" inequality to relate the accuracy with expected accuracy: and (ii) DKW inequality to show the concentration of the estimated accuracy with our proposed method. Finally, combine (i) and (ii) using the fact that at selected threshold the number of false positives is equal to the number of false negatives.
Using Hoeffdings' bound, have with probability least -8/2
log 4/8)
arg max (xx) # y4 I
((a.
I
arg max (x) # jey
(22)
2n
jey
19

---

Published as a conference paper at ICLR 2022
With DKW inequality, we have with probability at least 6/2
[m max (xx) < & E (o.,)) jey
(4/(4/))
Σ I
max fz(z) <t' Jey
(23)
2n
for all >'> Finally by definition, have
I1 Σ
Σ
max (a, < jey
arg max [5(x;) # yi
(24)
jey
Combining 22
at t' f, and 24 we have the desired result.
E BASLINE METHODS
Importance-r--wighhtnng (IM) If we can estimate the importance-ratios P((r)
(. with just the unla-
beled data from the target and validation labeled data from source, then we can estimate the accuracy as on target as follows:
po(x)
T [f(x) * w| (x(x)
Epr (f) (( DD5
(25)
As previously discussed, this is particularly useful in the setting of covariate shift (within support) where importance ratios estimation has been explored in the literature in the past. Mandolin (Chen al. (02T)) extends this approach. They estimate importance-wer weights with use of extra supervision about the axis along which the distribution is shifting.
In our work, we experiment with uniform mass binning and equal width binning with the number of bins in [5, 10, 50]. Overall, we observed that equal width binning works the best with 10 bins. Hence throughout this paper we perform equal width binning with 10 bins to include results with IM.
Average Confidence (AC) If we expect the classifier to be argmax calibrated on the target then average confidence equal to accuracy of the classifier. Formally, by definition of argmax calibration of f on any distribution D, we have
$ arg max (((r) | ((x max (z(
Ep(
E (x.v)~
I
(26)
jey
jEY
Difference Of Confidence We estimate the on source and target (as a distributional distance (Guillory et al.] (021) ))) from expected error on
target by subtracting difference of confidences
source distribution, i.c, DOCpt EE-Ds [I Jarg maXjey fy(z) y + EE Er [max]ey fy()]
Eg=Ds [maxjey [z(=]] This is referred to DOC-Feat in (Guillory et 2021).
2021) proposed average disagreement of two models (trained the same training but with different initialization and/or different data ordering) as approximate measure of accuracy on the underlying data, i.c..
Generalized Disagreement Equality (GDE) Jiang et al.
Ep(f) = E [[f(x) * f(()]]
(27)
They show that marginal calibration of the model is sufficient have expected test error equal the expected of average disagreement of two models where the latter expectation is also taken over the models used calculate disagreement.
F DETAILS ON THE DATASET SETUP
In our empirical evaluation, we consider both natural and synthetic distribution shifts. We con- sider shifts on ImageNet (Russakovsky al. 2015). CIFAR Krizhevsky & Hinton (2009). FMoW. WILDS (Cristtie et al. 2018). RxRxI- WILDS Taylor et al. 2019), Amazon- WILDS (Ni et al. 2019 CivilComments- WILDS (Borkan al. 2019). and MNIST LeCun al. (1998) datasets.
20

---

Published as a conference paper at ICLR 2022
Train (Source)
Valid (Source)
Evaluation (Target)
MNIST (train)
MNIST (valid)
USPS, SVHN and Q-MNIST
CTFAR 1O (train)
CIFARID 1O (valid)
CIFARIÓV2, 95 CIFARIO-C. datasets (Fog and Motion blur, erc.
CIFARIO0 (train)
CIFAR100 (valid)
95 CIFARI00-C datasets (Fog and Motion blur, etc.
FMoW {(2013-15, 2016-17) x
FMoW (2002-12) (train)
FMoW (2002-12) (valid)
(All, Africa, Americas, Oceania, Asia, and Europe))
RxRx1 (train)
(idva()d-val)
RxRxI (id-test. OOD-val, OOD-test)
Amazon (train)
Amazon (id-val)
Amazon (OOD-val, OOD-test)
CiviComments demographic identities male, female. LGBTQ. Christian, Muslim, other religions, Black, and White)
CivilComments (train)
CivilComments (id-val)
ImageNetv2 datascts, ImageNet-Sketchh
ImageNet (train)
ImageNet (valid)
95 ImageNet-C dutascts
3 ImageNett200k 200v2 datasets, ImageNet-R, ImageNNr200-Skeer Sketch, 95 ImageNet200-C
ImageNet-200 (truin)
ImageNet-200 (valid)
datasetssss datasets
Same subpopulations as train but unseen images from natural and syntheric shifts in ImageNet, Novel subpopulations on natural and synthetic shifts
BREEDS (train)
BREEDS (valid)
Table 2:
Details of the test datasets considered in our evaluation.
ImageNet setup. First, we consider synthetic shifts induced to simulate 19 different visual corruptions (e.g., shot noise, motion blur, pixelation etc.) each with 5 different intensities giving us a total of 95 datasets under ImageNet-C (Hendrycks & Dietterich, 2019). Next, consider natural distribution shifts due to differences in the data collection process. In particular, we consider 3 ImageNetv2 (Recht al., 2019) datasets each using a different strategy collect test sets. We also evaluate performance on images with artistic renditions of object classes, i.e., ImageNet-R (Hendrycks et al. 2021 and ImageNet-Sketee (Wang et al., 2019) with hand drawn sketch images. Note that renditions dataset only contains 200 classes from ImageNet. Hence, in the main paper we include results on ImageNet restricted to these 200 classes, which we call as ImageNet-200, and relegate results on ImageNet with 1k classes to appendix.
We also consider BREEDS benchmark (Santurkar et al. 2020 in our evaluation to assess robustness to subpopulation shifts, in particular, to understand how accuracy estimation methods behave when novel subpopulations not observed during training are introduced. BREEDS leverages class hicrarchy in ImageNet to repurpose original classes be the subpopulations and defines classification task on superclasses. Subpopulation shift is induced by directly making the subpopulations present in the training and test distributions disjoint. Overall, BREEDS benchmark contains 4 datasets ENTITY-13, ENTITY-30, LIVING-17, NON-LIVING-26, each focusing on different subtrees in the hierarchy. To generate BREEDS dataset top of ImageNet, we use the open source library: (https //github. com/ MadryLab BREEDS Bew.Beenhhhhrre We focus on natural and synthetic shifts as in LetgeNet on same and different subpopulations in BREEDs. Thus for both the subpopulation (same or novel), we obtain a total of 99 target datasets.
CIFAR setup. Similar the ImageNet setup, we consider (i) synthetic shifts CIFAR-1O-C) due to common corruptions; and (ii) natural distribution shift (i.e., CIFARv2 (Recht et al., 2018 Torralba
et al. 2008)) due to differences in data collection strategy on on CIFAR- 10 (Krizhevsky & Hinton On CIFAR-100, we just have synthetic shifts due to common corruptions.
2009
WMOW-WIL.DS setup. order to consider distribution shifts faced in the wild, we consider FMoW- WILDS (Koh et al. 2021: Christic et al.) 2018) from WILDS benchmark. which contains satellite images taken different geographical regions and at different times. We obtain 12 different OOD target sets by considering images between years 2013-201 and 2016-2018 and by considering five geographical regions as subpopulations (Africa, Americas, Oceania, Asia, and Europe) separately and together.
WILDS setup. Similar to FMoW, we consider RxRXI-WILDS (Taylor et al. 2019) from WILDS benchmark, which contains image of cells obtained by fluorescent microscopy and the task
21

---

Published as a conference paper at ICLR 2022
is to genetic treatments the cells received. We obtain 3 target datasets with shift induced by batch effects which make it difficult to draw conclusions from data across experimental batches.
Amazon-WILDS setup. For natural language task, consider Amazon- WILDDS (Ni et al. 2019) dataset from WILDS benchmark, which contains review text and the task is get a corresponding star rating from 1 to 5. We obtain target datasets by considered shifts induced due to different set of reviewers than the training set.
CivilCommennt- WILDS setup. We also consider CivilComments- WILDS (Borkan et al. 2019) from WILDS benchmark, which contains text comments and the task is to classify them for toxicity. We obtain 18 target datasets depending on whether a comment mentions each of the demographic identities male, female, LGBTQ. Christian, Muslim, other religions, Black, and White.
MNIST setup. For completeness, we also consider distribution shifts on MNIST (LeCun al. 1998) digit classification as in the prior work (Deng & Zheng 202I). We use three real shifted datasets, i.c., USPS (Hull 1994). SVHN (Netzer et al. [001]) 1) and QMNIST (Yadav Bottou 2019
G DETAILS ON THE EXPERIMENTAL SETUP
All experiments were run on NVIDIA Tesla V100 GPUs. We used PyTorch (Paszke et al. 2019) for experiments.
Deep nets We consider 4-layered MLP. The PyTorch code for 4-layer MLP follows:
nn
Sequentttaal Fllatee (),
nn. Linear (input.dim, 5000, bias=True), nn. nn.ReLU(),
In.einear (5000, 5000, bias=True), nn.ReLU
In.eir (5000, 50, bias=True), nn.ReLU(), Ln. (50, numelabel, bias-True)
We mainly experiment convolutional nets. In particular, we use ResNet18 (He al., 2016). ResNet50, and DenseNet121 (Huang al.) 2017) architectures with their default implementation in Py PyTorch. Whenever we initial our models with pre-trained models, we again use default models in Py Py/occ.
Hyperparameters and Training details As mentioned in the main text do not alter the standard training procedures and hyperparameters for each task. We present results at final model, however, we observed that the same results extend an early stopped model well. For completeness, include these details below:
CIFARIO and CIFAR100 We train DenseNet121 21 and ResNer18 architectures from scratch. We use SGD training with momentum of 0.9 for 300 epochs. We start with learning rate 0.1 and decay it by multiplying with 0.1 every 100 epochs. We use a weight decay of 4. We use batch size of 200. For CIFARIO, we also experiment with the same models pre-trained on ImageNet.
ImageNet For training, we use Adam with batch size of 64 and learning rate 0.0001. Due to huge size of ImageNet, we could only train two models needed for GDE for 10 epochs. Hence, for relatively small scale experiments, we also perform experiments on ImageNet subset with 200 classes, which we call as ImageNet-200) with the same training procedure. These 200 classes are the same classes as in ImageNet-R dataset. This not only allows us to train ImageNet for 50 epochs but also allows us to ImageNet-R R in our testbed. On the both the datasets, observe similar superioriy with ATC. Note that all the models trained here were initialized with a pre-trained ImageNet model with the last layer replaced with random weights.
SMoW-WILDS For all experiments, we follow Koh et al. (2021) and use two architectures Ne1 and ResNet50, both pre-trained on ImageNet. We use the Adam optimizer (Kingma & Ba 2014) with an initial learning rate of 10 -4 that decays by 0.96 per epoch, and train for 50 epochs and with batch size of 64.
22

---

Published as a conference paper at ICLR 2022
RxRx/-WILDS For all experiments, follow Koh et al. (2021) and use two architectures DenseNet121 and ResNet50, both pre-trained on ImageNet. We use Adam optimizer with learning rate of Le 4 and i2-regularization strength of 1e - 5 with batch size of 75 for 90 epochs. We linearly increase the learning rate for 10 epochs, then decreasing it following cosine learning rate schedule. Finally, we pick the model that obtains highest in-distribution validation accuracy.
Smazonn-ILLDS For all experiments, we follow Koh et al. (202T) and finetuned DistiBBERT- base-uncased models (Sanh al., 2019), using the implementation from Wolf al.) (2020), and with the following hyperparameter settings: batch size 8; learning rate le - 5 with the AdamW optimizer Loshchilov Hutter, 2017); L2-regularization strength 0.01; epochs with early stopping;
and maximum number of tokens of 512.
CivilCommentt-vee WILDS For all experiments, we follow Koh et (2021) and fine-tuned DistilBERT- base-uncasee models (Sanh et al. (019). using the implementation from Wolf et al. (2020) and with the following hyperparameter settings: batch size 16; learning rate le 5 with the AdamW optimizer (Loscchllov & Hutter 2017) for 5 epochs; L2-regularization strength 0.01; and a maximum number tokens of 300.
Living/7 7 and Nonliving26 from BREEDS For training, we use SGD with a batch size of 128, weight decay 10-1, and learning rate 0.1. Models were trained until convergence. Models were trained for total of 450 epochs, with 10-fold learning rate drops every 150 epochs. Note that since want to evaluate models for novel subpopulations no pre-training was used. We train two architectures DenseNet121 and ResNet50.
Entity13 and Entity30,rro BREEDS For training. we use SGD with a batch size of 128, weight decay of 10- 0, and learning rate 0.1. Models were trained until convergence. Models were trained for a total of 300 epochs, with 10-fold learning rate drops every 100 epochs. Note that since we want used. We train architectures
to evaluate models for novel subpopulations no pre-training DenseNet121 and ResNet50.
MNIST For MNIST, we train MLP deseribed above with SGD with momentum 0.9 and learning rate 0.01 for 50 epochs. We use weight decay of 10-5 and batch size as 200.
We have single number for CivilComments because it is binary classification task. For multiclass problems, ATC-NE and ATC-MC can lead to different ordering of examples when ranked with the corresponding scoring function. Temperature scaling on top can further alter the ordering of examples. The changed ordering of examples yields different thresholds and different accuracy estimates. However for binary classification, the two scoring functions are the same as entropy (i.e.)) plg((p) + (1 p) log(p)) has a one-to-one mapping to the max conf for e [o, 1]. Moreover, temperature scaling also doesn't change the order points for binary classification problems. Hence for the binary classification problems, both the scoring functions with and without temperature scaling yield the same estimates. We have made this clear in the updated draft.
Implementation for Temperature Scaling We temperature scaling implementation from https: /ttp:/ggttti com/ kundajelee weaeeaababbaeeeeeeee /aaaaatsttniiion We use validation set (the same we use to obtain ATC threshold or DOC source error estimate) tune single temperature parameter.
G.1 DETAILS ON FIG. (RIGHT) SETUP
For vision datasets, we train DenseNet model with the exception FCN model for MNIST dataset. For language datasets, we fine-tune we use the exact same setup as described Sec. G Importantly, obtain errors on the same scale, we rescale all the eтrors by subtracting the error of Average Confidence method for each model. Results are reported as mean of the re-scaled errors over 4 seeds.
model. For each of these models,
23

---

Published as a conference paper at ICLR 2022
H
SUPPLEMENTARY RESULTS
H.1
CIFAR PRETRAINING ABLATION
CIFARIO-Pretraining
90
Predicted Accuracy
80
70
60
DOC
50
GDE
40
ATC-NE (Ours)
30
IM
20
AC
20
40
60
80
OOD
Accuracy
Figure 6: Results with pretrained DenseNet121 model on CIFARIO. We observe similar behaviour as that with a model trained from scratch.
H.2
BREEDS RESULTS WITH REGRESSION MODEL
Entity13
Entity30
Nonliving26
90
90
DOC (w/o fit]
90
Predicted Accuracy
80
Predicted Accuracy
DOC (w/ fi))
Predicted Accuracy
80
70
ATC (w/l
20
60
60
60
50
50
50
40
(w/e mt)
40
40
(w/o fi)
DOC
DOC (w/ tt)
30
ATC (w/a li)
30
ATC (w/l
20
20
20
20
40
60
00D Accuracy
0OD Accuracy
00D Accuracy
Figure Scatter plots for DOC with linear fit.
Results parallel to Fig.
3Middle) on other
BREEDS dataset.
Dataset
DOC (w/o fit)
DOC (w fit)
ATC-MC (Ours) (wo fit)
LIVING-17
24.32
13.65
10.07
NONLIVING-26
29.91
18.13
19.37
ENTITY-13
22.18
8.63
8.01
ENTITY-30
24.71
12.28
10.21
Table Mean Absolute estimation Error (MAE) results for BREEDs datasets with novel populations in our setup. After fitting robust linear model for DOC on same subpopulation, we show predicted accuracy on different subpopulations with fine-tuned DOC (i.c., DOC (w/ fit)) and compare with
ATC without any regression model, i.e., ATC (w/o fit). While observe substantial improvements in MAE from DOC (w/o fit) to DOC (w/ fit), ATC (w/o fit) continues to outperform even DOC (w/ fit).
24

---

Published as a conference paper at ICLR 2022
CIFAR10
CIFAR100
ImageNet200
90
80
Predicted Accuracy
Predicted Accuracy
70
Predicted Accuracy
80
80
60
70
60
60
DOC
DOC
DOC
50
GDE
GDE
40
GDE
ATC-NE
(Ours)
30
ATC-NE
(Ours)
AT C-NE (Ours)
40
IM
20
IM
20
30
AC
10
AC
AC
60
20
40
20
80
OOD Accuracy
OOD Accuracy
OOD Accuracy
ImageNet
FMoW-Wilds
75
RxRx-WWilds
70
37.5
Predicted Accuracy
60
Predicted Accuracy
65
Predicted Accuracy
35.0
32.5
50
30.0
40
55
20
30
ATC-MC
DOC
GDE
(Ours)
45
50
DOC
GDE
ATC-NE (Ours)
27.5
22.5
25.0
GDE
ATC-NE (Ours)
DOC
10
40
IM
20.0
IM
AC
AC
AC
17.5
20
40
60
40
50
60
70
20
25
30
35
OOD Accuracy
OOD Accuracy
0OD Accuracy
CivilComments Willds Wilds
Amazon-Wol Wilds
MNIST
Predicted Accuracy
87.5
90.0
DOC
GDE
Predicted Accuracy
90
Predicted Accuracy
100
ATC.NE (Ours)
80
85.0
80
82.5
AC
DOC
60
DOC
80.0
70
GDE
GDE
77.5
ATC:NE
(Ours)
40
ATC-NE (Our)
60
75.0
AC
20
72.5
75
80
85
90
60
70
80
90
20
40
60
80
100
00D
Accuracy
0OD Accuracy
00D
Accuracy
Living17
Entity13
Entity30
Predicted Accuracy
Predicted Accuracy
80
Predicted Accuracy
80
60
60
60
40
40
DOC
DOC
20
GDE
20
GDE
ATC-NE (Ours)
ATC-NE (Ours)
ATC-NE (Ours)
20
40
60
80
20
60 00D Accuracy
40
80
20 40
60
80
0OD Accuracy
00D Accuracy
Figure 8: Scatter plot of predicted accuracy (true) 0OD accuracy. For vision datasets except MNIST we use a DenseNer121 model. For MNIST, we use a FCN. For language datasets, we use Results reported by aggregating accuracy numbers over 4 different seeds.
25

---

Published as a conference paper at ICLR 2022
CIFAR10
CIFAR100
ImageNet200
80
90
Predicted Accuracy
Predicted Accuracy
70
Predicted Accuracy
80
80
60
70
60
50
60
50
x
DOC
GDE
40
DOC
GDE
40
DOC
GDE
40
ATC-NE
(Ours)
30
ATC-NE
(Ours)
AT C-NE (Ours)
20
30
IM
IM
AC
AC
AC
20
60
80
20
60
OOD Accuracy
OOD Accuracy
OOD Accuracy
ImageNet
FMoW-Wilds
RxRx-WWilds
70
37.5
Predicted Accuracy
60
Predicted Accuracy
Predicted Accuracy
35.0
50
55
32.5
40
50
30.0
30
DOC
GDE
45
GDE
DOC
27.5
25.0
GDE
DOC
20
ATC-MC (Ours)
ATC-NE (Ours)
22.5
ATC-NE (Ours)
10
AC
35
IM
AC
20.0
IM
AC
17.5
20
40
60
40
60
20
25
30
35
OOD Accuracy
OOD Accuracy
OOD Accuracy
Living17
Nonliving26
Entity13
90
80
80
Predicted Accuracy
70
Predicted Accuracy
70
Predicted Accuracy
70
60
60
60
50
50
40
40
50
30
40
20
GDE
DOC
20
DOC
GDE
30
GDE
DOC
10
ATC-NE (Ours)
10
ATC-NE (Ours)
20
ATC.NE (Ours)
10
20
40
60
80
20
40
60
80
20
40
60
80
OOD Accuracy
0OD Accuracy
OOD Accuracy
Entity30
80
Predicted Accuracy
70
60
50
40
30
20
10
ATC-NE (Ours)
20
40
60
80
00D Accuracy
Figure 9: Scatter plot of predicted accuracy versus (true) OOD accuracy for vision datascts except MNIST with ResNet50 model. Results reported by aggregating MAE numbers over different seeds.
26

---

Published as a conference paper at ICLR 2022
IM
AC
DOC
GDE
ATC-MC (Ours)
ATC-NE (Ours)
Dataset
Shift
Pre T
Post T
Pre T
Post T
Pre T
Post
Post T
Pre.
Post
Pre T
Post
6,60
5.74
9.88
6.89
7.25
6.07
4.77
3.21
3.02
2.99
2.85
Natural
(0.35)
(0.30)
(0.16)
(0.13)
(0.15)
(0.16)
(0.13)
(0.49)
(0.40)
(0.37)
(0.29)
CIFARIO
12.33
10.20
16.50
11.91
13.87
11.08
6.55
4.65
4.25
4.21
3.87
Syntheric
(0.51)
(0.48)
(0.26)
(0.17)
(0.18)
(0:17)
(0.35)
(0.55)
(0.55)
(0.55)
(0.75)
CIFARI00
Syntheric
13.69
11.51
23.61
13.10
14.60
10.14
9.85
5.50
4.75
4.72
4.94
(0.55)
(0.41)
(1.16)
(0.80)
(0.77)
(0.644)
(0.57)
(0.70)
(0.73)
(0.74)
(0.74)
12.37
8.19
22.07
8.61
15.17
7.81
5.13
4.37
2.04
3.79
1.45
Natural
(0.25)
(0.33) (0.08)
(0.25)
(0.11)
(0:29)
(0.08)
(0.39)
(0.24)
(0.30)
(0.27)
ImageNet200
Synthetic
19.86
12.94
32.44
13:35
25.02
12.38
5.41
5.93
3.09
5.00
2.68
(1.38)
(1.81)
(1.00)
(1.30)
(1.10)
(1.38)
(0.89)
(1.38)
(0.87)
(1.28)
(0.45)
7.77
6.50
18.13
6.02
8.13
5.76
6.23
3.88
2.17
2.06
0.80
Natural
(0.27)
(0.33)
(0.23)
(0.34)
(0.27)
(0.37)
(0.41)
(0.63)
(0.62)
(0.54)
(0.44)
ImageNet
13.39
10.12
24.62
8.51
13.55
7.90
6.32
3.34
2.53
2.61
4.89
Synthetic
(0.53)
(0.63)
(0.64)
(0.71)
(0.61)
(0.72)
(0.33)
(0.53)
(0.36)
(0.33)
(0.83)
5.53
4.31
33.53
12.84
5.94
4.45
5.74
3.06
2.70
3.02
2.72
FMoW- WILDS
Natural
(0.33)
(0.63)
(0.13)
(12.06)
(0.36)
(0.77)
(0.55)
(0.36)
(0.54)
(0.35)
(0.44)
5.80
5.72
7.90
4.84
5.98
5.98
6.03
4.66
4.56
4.41
4.47
RXRXI-WILDS Natural
(0.17)
(0.15)
(0.24)
(0.09)
(0.15)
(0.13)
(0.08)
(0.38)
(0.38)
(0.31)
(0.26)
2.40
2.29
8.01
2.38
2.40
2.28
17.87
1.65
1.62
1.60
1.59
Amazon-WILDD
Natural
(0.08)
(0.09)
(0.53)
(0.17)
(0.09)
(0.09)
(0.18)
(0.06)
(0.05)
(0.14)
(0.15)
12.64
10.80
16.76
11.03
13.31
10.99
16.65
7.14
CivilCom.
Natural
(0.52)
(0.48)
(0.53)
(0.49)
(0.62)
(0.49)
(0.25)
(0.41)
18.48
15.99
21.17
14.81
20.19
14.56
24.42
5.02
2.40
3.14
3.50
MNIST
Natural
(0.45)
(1.53)
(0.24)
(3.89)
(0.23)
(3.47)
(0.41)
(0.44)
(1.83)
(0.49)
(0.17)
16.23
11.14
24.97
10.88
19.08
10:47
10.71
5.39
3.88
4.58
4.19
Same
(0.77)
(0.65)
(0.70)
(0.77)
(0.65)
(0.72)
(0.74)
(0.92)
(0.61)
(0.85)
(0.16)
ENTITY-13
28.53
22.02
38.33
21.64
32.43
21.22
20.61
13.58
10.28
12.25
6.63
Novel
(0.82)
(0.68)
(0.75)
(0.86)
(0.69)
(0.80)
(0.60)
(1.15)
(1.34)
(1.21)
(0.93)
18.59
14.46
28.82 78.82
14.30
21.63
13:46
12.92
9.12
7.75
8.15
7.64
Same
(0.51)
(0.52) (0.43)
(0.71)
(0.37)
(0.59)
(0.14)
(0.62)
(0.72)
(0.08)
(0.88)
ENTITY-30
32.34
26.85
44.02
26.27
36.82
25.42
23.16
17.75
14.30
15.60
10.57
Novel
(0.60)
(0.58)
(0.56)
(0.79)
(0.47)
(0.68)
(0.12)
(0.76)
(0.85)
(0.86)
(0.86)
18.66
17.17
26.39
16.14
19.86
15.58
16.63
10.87
10.24
10.07
10.26
Same
(0.76)
(0.74) (0.82)
(0.81)
(0.67)
(0.76)
(0.45)
(0.98)
(0.83)
(0.92)
(1.18)
NONLIVING-26
33.43
31.53
41.66
29.87
35.13
29.31
29.56
21.70
20.12
19.08
18.26
Novel
(0.67)
(0.65)
(0.67)
(0.71)
(0.54)
(0.64)
(0.21)
(0.86)
(0.75)
(0.82)
(1.12)
12.63
11.05
18.32
10.46
14.43
10.14
9.87
4.67
3.95
3.81
4.21
Same
(1.25)
(1:20) (1.01)
(1.12)
(1.11)
(1.16)
(0.61)
(0.71)
(0.48)
(0.22)
(0.53)
LIVING-17
29.03
26.96
35.67
26.11
31.73
25.73
23.53
16.15
14.49
12.97
11.39
Novel
(1.44)
(1.38)
(1.09)
(1.27)
(1.19)
(1.35)
(0.52)
(1.36)
(1.46)
(1.52)
(1.72)
Table 3: Mean Absolute estimation Error (MAE) results for different datasets in our setup grouped by the nature of shift. 'Same' refers to same subpopulation shifts and 'Novel' refers novel subpopulation shifts. We include details about the target sets considered in each shift in Table 2 Post T denotes use TS calibration on source. For language datasets, we use DistiBBERT--ass--nnee dataset we report results with DenseNet model with the exception of MNIST where we use FCN. Across all datasets, we observe that ATC achieves superior performance (lower MAE is better). For GDE post T and pre estimates match since TS doesn't alter the argmax prediction. Results reported by aggregating MAE numbers over 4 different seeds. Values in parenthesis (i.e., (-)) denote standard deviation values.
for vision
27

---

Published as a conference paper at ICLR 2022
DOC
GDE
ATC-MC (Ours)
ATC-NE (Ours)
Dataset
Shift
Pre T
Post T
Pre T
Post T
Pre T
Post T
Post T
Pre T
Post T
Pre T
Post T
7.14
6.20
10.25
7.06
7.68
6.35
5.74
4.02
3.85
3.76
3.38
Natural
(0.14)
(0.11)
(0.31)
(0.33)
(0:28)
(0.27)
(0.25)
(0.38)
(0.30)
(0.33)
(0.32)
CIFARIO
12.62
10.75
16.50
11.91
13.93
11.20
7.97
5.66
5.03
4.87
3.63
Synthetic
(0.76)
(0.71)
(0.28)
(0.24)
(0.29)
(0.28)
(0.13)
(0.64)
(0.71)
(0.71)
(0.62)
12.77
12.34
16.80
12.73
11.18
9.63
12.00
5.61
5.55
5.65
5.76
CIFARIO0
Synthetic
(0.43)
(0.68)
(0.20)
(2.59)
(0.35)
(1.25)
(0.48)
(0.51)
(0.53)
(0.35)
(0.27)
12.63
7.99
23.08
7.22
15.40
6.33
5.00
4.60
1.80
4.06
1.38
Natural
(0.59)
(0.47)
(0.31)
(0.22)
(0.42)
(0.24)
(0.36)
(0.63)
(0.17)
(0.69)
(0.29)
ImageNet200
20.17
11.74
33.69
9.51
25.49
8.61
4.19
5.37
2.78
4.53
3.58
Synthetic
(0.74)
(0.80)
(0.73)
(0.51)
(0.66)
(0.50)
(0.14)
(0.88)
(0.23)
(0.79)
(0.33)
8.09
6.42
21.66
5.91
8.53
5.21
3.90
3.93
1.89
2.45
0.73
Natural
(0.25)
(0.28)
(0.38)
(0.22)
(0.26)
(0:25)
(0.44)
(0.26)
(0.21)
(0.16)
(0.10)
ImageNet
13.93
9.90
28.05
7.56
13.82
6.19
6.70
3.33
2.55
2.12
5.06
Synthetic
(0.14)
(0.23)
(0.39)
(0.13)
(0.31)
(0.07)
(0.52)
(0.25)
(0:25)
(0.31)
(0.27)
5.15
3.55
34.64
5.03
5.58
3.46
3.08
2.50
2.33
2.52
2.22
FMoW-WILDS Natural
(0.19)
(0.41)
(0.22)
(0.29)
(0.17)
(0.37)
(0.46)
(0.32)
(0.28)
(0:25)
(0.30)
6.17
6.11
21.05
5.21
6.54
6:27
6.82
5.30
5.20
5.19
5.63
RXRXI-WILDS
Natural
(0.20)
(0.24)
(0.31)
(0.18)
(0:21)
(0:20)
(0.31)
(0.30)
(0.44)
(0.43)
(0.55)
18.32
14.38
27.79
13.56
20.50
13.22
16.00
9.35
7.50
7.80
6.94
Same
(0.29)
(0.53)
(1.18)
(0.58)
(0.47)
(0.58)
(0.84)
(0.79)
(0.65)
(0.62)
(0.71)
ENTITY-13 13
28.82
24.03
38.97
22.96
31.66
22.61
25.26
17.11
13.96
14.75
9.94
Novel
(0.30)
(0.55)
(1.32)
(0.59)
(0.544
(0.58)
(1.08)
(0.84)
(0.93)
(0.64)
(0.78)
16.91
14.61
26.84
14.37
18.60
13.11
13.74
8.54
7.94
7.77
8.04
Same
(1.33)
(1.11)
(2.15)
(1.34)
(1.69)
(1.30)
(1.07)
(1.47)
(1.38)
(1.44)
(1.51)
ENTITY-30
28.66
25.83
39.21
25.03
30.95
23.73
23.15
15.57
13.24
12.44
11.05
Novel
(1.16)
(0.88)
(2.03)
(1.11)
(1.64)
(1.11)
(0.51)
(1.44)
(1.15)
(1:26)
(1.13)
17.43
15.95
27.70
15.40
18.06
14.58
16.99
10.79
10.13
10.05
10.29
Same
(0.90)
(0.86)
(0.90)
(0.69)
(1.00)
(0.78)
(1.25)
(0.62)
(0.32)
(0.46)
(0.79)
NONLIVING-26
29.51
27.75
40.02
26.77 77
30.36
25.93
27.70
19.64
17.75
16.90
15.69
Novel
(0.86)
(0.82)
(0.76)
(0.82)
(0.95)
(0.80)
(1.42)
(0.68)
(0.53)
(0.60)
(0.83)
14.28
12.21
23.46
11.16
15.22
10.78
10.49
4.92
4.23
4.19
4.73
Same
(0.96)
(0.93)
(1.16)
(0.90)
(0.96)
(0.99)
(0.97)
(0.57)
(0.42)
(0.35)
(0.24)
LIVING-17 17
28.91
26.35
38.62
24.91
30.32
24.52
22.49
15.42
13.02
12.29
10.34
Novel
(0.66)
(0.73)
(1.01)
(0.61)
(0.59)
(0.74)
(0.85)
(0.59)
(0.53)
(0.73)
(0.62)
Table 4: Mean Absolute estimation Error (MAE) results for different datasets in our setup grouped by the nature of shift for ResNet model. 'Same' refers to same subpopulation shifts and 'Novel' refers novel subpopulation shifts. We include details about the target sets considered in each shift in Table 2 Post T denotes use of TS calibration on source. Across all datasets, we observe that ATC achieves superior performance (lower MAE is better). For GDE post T and pre T estimates match since TS doesn't alter the argmax prediction. Results reported by aggregating MAE numbers over 4 different seeds. Values in parenthesis (i.e., (-)) denote standard deviation values.
28


Reviews:
[{"review_id": "mFuSgfI-qIa", "reviewer": "Reviewer_s1vP", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n---\nI am not an expert in this particular area, but to the best of my knowledge, the proposed technique is novel. Moreover, it is demonstrably more effective than prior methods and particularly simple to implement and use.\n\nThe paper is of relatively high quality. An extensive number of experiments are carried out on different distribution shift benchmarks, and it seems like the proper prior methods are compared to, though again, I am not an expert.\n\nThe paper is well-written and well-structured. This is aided in particular by the fact that the proposed technique is so simple to begin with.\n\nFinally, the paper is likely to be of practical interest for researchers and practitioners studying and dealing with distribution shift.\n\n\nWeaknesses\n---\nThere are still a few substantive ways to improve the quality of the paper. First, it seems that neither of the theoretical sections (3 and 6) tie in well with the rest of the paper. In my view, the paper is primarily an empirical contribution. Section 3 ostensibly says nothing specific about the proposed technique, it just presents some general results that seem relatively obvious (though I am unsure whether they have been explicitly stated in prior work). But does the theory here motivate the proposed technique in any way? This is unclear to me. Section 6 is more closely related to the empirics, but this connection could still be improved. What are the key insights we can take away from the toy problem? How does it allow us to better understand for which real world problems the technique may succeed or fail? In my view, the paper could do a better job at addressing these questions.\n\nRegarding empirics, the paper is significantly stronger here, but there is still some room for improvement. Most concretely, evaluations on the other WILDS datasets could be quite interesting. This is not (just) a generic \"add more experiments\" comment: all of the current experiments focus on natural (with the exception of some rendition and sketch shifts) object-centric images, and WILDS contains other modalities such as medical and satellite images, which makes for a rather interesting additional testbed. WILDS also contains language tasks, which would be separately interesting to evaluate on, though I acknowledge this may take more time to set up.\n\nRegarding clarity, there are some minor ways to improve the paper. Section 4 could be expanded to provide more intuition and formal arguments for why the proposed technique makes sense and is justified, ideally from first principles. In fact, I may go as far as to say that this type of exposition would be more useful than Section 3 entirely, to the concerns I have noted above. Figures 2 and 3 were confusing to parse at first -- I think the lines are regression fits, but this should be made explicit if it currently isn't. Table 1 is a bit confusing in the first two columns because the datasets are listed but not necessarily the test sets. Finally, some of the comparisons such as IM should be explained in more detail.", "summary_of_the_paper": "This paper proposes a simple yet effective technique for estimating a classifier's accuracy on test data which exhibits distribution shift from the training data. Given a score function for the model and a data point, the technique finds a threshold such that such that the relative proportions of validation data above and below the threshold are the same as correctly vs incorrectly classified points, respectively. This threshold is then used to predict the model's accuracy on test data, and the paper demonstrates that this predictor is significantly more accurate than prior methods on a range of distribution shift benchmarks. There are also some theoretical and empirical results on a toy problem that seem to indicate that the proposed technique is well-suited for at least this problem.", "summary_of_the_review": "Primarily due to my concerns above, I am initially recommending a weak accept of this paper. I am happy to engage in discussion with the authors and other reviewers in order to reach a more confident final recommendation.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RlqWWFDUDlk", "reviewer": "Reviewer_t6Wr", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The experiments are thorough, conducted on a variety of types of shifts and models.\n\nWeaknesses:\n1. The proposed method lacks theoretical rigor. The threshold for source and target would differ based on different types of shifts, and there's no reason why the source threshold should be used to predict target accuracy. The fact that it works empirically is not a good justification.\n2. No theoretical analysis are are done on NE vs. MC. When should we use one over the other, and why?\n3. Section 3 doesn't fit with the rest of the paper. Since there's no general-purpose estimator, why does this paper still propose such an estimator? The impossible results are not surprising. It is perhaps more meaningful to study principled estimators under specific settings, just like the covariate shift or label shift settings.\n4. I wish to see the theoretical analysis on the toy setting generalized. What are the necessary and sufficient conditions for ATC to work?\n5. I wish the authors compare with other calibration methods beyond temperature scaling.", "summary_of_the_paper": "This work proposes a simple method, Average Thresholded Confidence (ATC), to predict the OOD accuracy of a classifier based on a labeled source distribution and an unlabeled target distribution. First, the authors show a theoretical lower bound that such task is impossible without assumption on the nature of the shift. Then, the authors proposes ATC. Experiments show that ATC empirically performs better than baselines on different datasets and models. Theoretically, they show that ATC is a consistent estimator for a toy setting with spurious features.", "summary_of_the_review": "The paper is strong empirically but lacks theoretical rigor.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "JrBFN1SNp_4", "reviewer": "Reviewer_uW7x", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I find the paper quite interesting and intuitive. The mechanism is simple and seems to be effective compared to prior work. Given that this is an important real-world problem, I think the paper deserves to be accepted at ICLR. \n\nHowever, I am not a 100% familiar with the related work in order to judge the novelty of the paper. \n\n## Strengths\n\n1. an appropriate amount of experiments\n2. interesting discussion of related work\n3. relevant theoretical results to motivate the problem\n\n## Weaknesses and other Remarks\n\n1. **Derivation and/or Interpretation of ATC**:\nThe authors did a decent job of providing some related theoretical results in the beginning. However, when the authors continue to introduce ATC this seems fairly ad-hoc without any kind of deeper explanation. I think this would merit a longer discussion.\n\n2. **Proper contextualization of theoretical results**:\nAs I said before, I appreciate the theoretical results as motivating examples for considering empirical scores. However, I don't think they can be considered actual strong contributions of the paper. The main contributions are a heuristic-based score and accompanying experimental results showing the effectiveness. So please make sure that this is reflected in the writing. \n\n3. **Clarification around some experimental settings**: \n   a. Could you provide some confidence intervals around the scatter plot in Figure 2 and related figures, e.g., using bootstrapping. Why are there so many more sampled points around higher accuracies? \n   b. Could you add more discussion around temperature scaling (cf. _For all methods, we implement post-hoc calibration on validation source data with Temperature Scaling_?) Why is this relevant and what's the intuition? If such an approach improves your performance, could also try improving your performance using more sophisticated calibration methods? That would be interesting to see. \n\n\n## Ways to Improve My Score\n\nPlease address the weaknesses and I have listed above. I would be happy to engage in discussions during the rebuttal period.\n\n--- \n\n\n## Update after rebuttal\n\nThe authors addressed my concerns in their responses. Consequently, I am raising my score from 6 to 8. My confidence score will remain at 3 given my missing knowledge about some of the prior work. ", "summary_of_the_paper": "The paper proposes a new scoring mechanism, Average Thresholded Confidence (ATC), to estimate out-of-distribution performance (i.e. accuracy) of a trained classifier using only labelled data from the source distribution (train distribution) and unlabelled data from the target distribution (test distribution). \n\nThe threshold is based on the logit output of a classifier and can thus be used on unlabelled data.\n", "summary_of_the_review": "The paper is well-written and I enjoyed reading it. I cannot 100% verify the novelty of the paper since I am not so familiar with the related work but given what the authors wrote in their related work section and their results it seems novel. \n\nOverall, I didn't have any major complaints about the paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DGsQbtSW1Ti", "reviewer": "Reviewer_zA7T", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The problem of predicting accuracy of the models is interesting and novel to me. How to identify the accuracy of a model under distribution shift is a practical challenge. The proposed method of ATC is extremely simple and makes intuitive sense. It is also reasonable to claim that identifying the accuracy is as hard as identifying the optimal predictor, and the efficacy of any method depends on the assumptions on the nature of the shift. \n\nQuality: The submission is technically sound. The claims in the contribution are well-supported by theoretical analyses and empirically results. It is a complete piece of work that outperforms prior work empirically.\n\nClarity: This paper is well-written and easy to follow. The problem is also well-motivated. The experimental details are also very specific, such that reproducing the results should be possible. ", "summary_of_the_paper": "This paper studies how to predict the target domain accuracy using only labeled source data and unlabeled target data. They propose Average Thresholded Confidence (ATC) to learn a threshold on the model's confidence. In particular, ATC predicts accuracy as the fraction of unlabeled examples for which model confidence exceeds the threshold. Extensive experiments have been conducted to show that ATC outperforms prior work across multiple model architectures, types of distribution shifts, and datasets. Theoretically, they prove that identifying the accuracy is as hard as identifying the optimal predictor, and the efficacy of any method depends on the assumptions on the nature of the shift.", "summary_of_the_review": "This paper proposes a novel method, Average Thresholded Confidence (ATC) to learn a threshold on the model's confidence. It is well-written and well-motivated. The proposed idea is simple and technically sound. The claims are well-supported by theoretical analyses and extensive experimental results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "9DJP-Z8Sz3g", "reviewer": "Reviewer_M2yw", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "-The proposed method is simple and easy to understand. I also think that their setting where no labeled data from the test distribution is available is highly relevant.\n\n-Their evaluation contains a large number of datasets and distribution shifts and clearly demonstrates that ATC is superior to baseline methods. The only thing that is not quite clear is which scoring function should be used in practice, as there exist certain configurations where ATC-MC and ATC-NE perform the best. \nThey also compare to a regression model that is trained with labeled data and are able to achieve similar performance without this extra information. \n\n-Is there an intuitive understanding why temperature rescaling helps? While temperature rescaling preserves the classification of the model, it can vary the ordering of the scores on different samples, however I would not have expected such large changes, especially in the order of ATC-NE pre and post T. Are there specific logit configurations that clearly show how temperature rescaling helps? \n\n-I liked the initial theoretical discussion of the problem. The authors state that \"every method of estimating accuracy on target data is tied up with some assumption on the nature of the shift\", is it possible to elaborate on the assumptions of ATC in this context? \n\n", "summary_of_the_paper": "The paper introduces Average Thresholded Confidence (ATC), a method that aims to predict a model's accuracy on an unlabeled test distribution that varies from the original training distribution by some form of distribution shift. \nThe paper first gives a theoretical analysis that shows that, in general, it is impossible to estimate test accuracy without assumptions on the particular distribution shift and that each method that tries to solve this problem has some underlying assumption on the distribution shift.\n\nTheir method ATC selects a threshold t on a scoring function (max confidence, or negative entropy) based on the error in the original source domain. The estimated error is then equal to the number of points in the test set that have a score lower than t. Importantly, this is done without using any label information from the test set. \n\nIn their experiments, they demonstrate improvements over various baseline methods on a large selection of datasets and distribution shifts. \nFinally, they have a toy example with very strong assumptions on which ATC-MC is a consistent accuracy estimator. ", "summary_of_the_review": "The paper offers some good theoretical insight into the general problem and introduces a simple method that clearly outperforms the baselines. The problem statement is interesting and very relevant to the community. The evaluation is extensive and clearly demonstrates the strengths of this paper. While the theory does not cover all aspects (eg temperature rescaling), I liked the paper overall. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a simple method to estimate the accuracy of a classifier on an unlabeled dataset given an in-distribution validation set. In extensive experiments the authors show that the proposed method is significantly more accurate than previous methods and other baselines. 

The reviewers are quite consistent in their judgement, just the weighting of the different aspects is different.
After the rebuttal four out of five reviewers recommend acceptance.

Strong points:
- simplicity of the method
- strong experimental results for various tasks and domain shift problems

Weak points:
- there is no clear theoretical statement when the method is supposed to work
- the discussion in Section 3.1 is pretty obvious and seems a bit like a waste of space whereas the motivation for the actual method is very short

While I agree with the reviewers that there is little theoretical justification for the method, the strong experimental results on various datasets, tasks and different domain shifts make this paper interesting for a large audience. Thus this paper is a nice contribution to ICLR and I recommend acceptance.
However, I strongly recommend to the authors to add more motivation in Section 4 and add a limitation section where the cases are discussed where the method is definitely not working. Section 3 is pretty obvious and could be significantly shortened or integrated into the limitations section. 

One case which is highly relevant for this limitations section is the provable asymptotic overconfidence of neural networks which is discussed in

Hein et al, Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem, CVPR 2019

This would definitely lead to a failure of the presented method as all predictions would get a score above the threshold. I would also assume that the method would predict high accuracy values for out-of-distribution tasks which are semantically similar e.g. training on CIFAR10 and then using CIFAR100 as unlabeled dataset. In that context it would be interesting to evaluate OOD-aware classifiers using ATC such as discussed in

Hendrycks et al, Deep Anomaly Detection with Outlier Exposure, ICLR 2019

Also it would be helpful to understand better the influence of the classifier performance on the original task on the performance of ATC on unlabeled data.
Decision: Accept (Poster)