Agreement-on-the-line: Predicting the Performance of Neural Networks under Distribution Shift
Authors: Christina Baek
Yiding Jiang
Aditi Raghunathan
J Zico Kolter
Venue: NeurIPS 2022 Accept (2022)
Keywords: generalization
out-of-distribution generalization
robustness

Abstract: Recently, Miller et al. showed that a model's in-distribution (ID) accuracy has a strong linear correlation with its out-of-distribution (OOD) accuracy, on several OOD benchmarks, a phenomenon they dubbed ``accuracy-on-the-line''.  While a useful tool for model selection (i.e., the model most likely to perform the best OOD is the one with highest ID accuracy), this fact does not help to estimate the actual OOD performance of models without access to a labeled OOD validation set. In this paper, we show a similar surprising phenomena also holds for the agreement between pairs of neural network classifiers: whenever accuracy-on-the-line holds, we observe that the OOD agreement between the predictions of any two pairs of neural networks (with potentially different architectures) also observes a strong linear correlation with their ID agreement. Furthermore, we observe that the slope and bias of OOD vs ID agreement closely matches that of OOD vs ID accuracy. This phenomenon which we call agreement-on-the-line, has important practical applications: without any labeled data, we can predict the OOD accuracy of classifiers, since OOD agreement can be estimated with just unlabeled data. Our prediction algorithm outperforms previous methods both in shifts where agreement-on-the-line holds and, surprisingly, when accuracy is not on the line. This phenomenon also provides new insights into neural networks: unlike accuracy-on-the-line, agreement-on-the-line only appears to hold for neural network classifiers.

A Agreement-on-the-Ling, Predicting the Performance of Neural Networks under Distribution Shift
Christina Back! Yiding Jiang' Aditi Raghunathan' Zico Kolter
Carneeiee Mellon University, Bossh Center for Al (kbaek, yidingji, raditi, zkolter)@cs c edu
Abstract
Recently, Miller al. [59] showed that model's in-distribution (ID) accuracy has a strong linear correlation with its out-of-distributinn (OOD) accuracy on several phenomenon they dubbed "accuracy-on-the-ine"" While a useful tool for model selection (i.e., the models with better ID accuracy are likely to have better OOD accuracy), this fact does not help estimate the actual OOD performance In this paper, show similar but surprising phenomenon also holds for the agreement between pairs of neural network classifiers: whenever accuracy-on- the-line holds, we observe that the OOD agreement between the predictions of any two pairs neural networks (with potentially different architectures) also observes strong linear correlation with their ID agreement. Furthermore, we observe that the slope and bias of OOD vs. ID agreement closely matches that of OOD vs. ID accuracy. This phenomenon, which we call ggreement-on-the-llee has important practical applications: without any labeled data, we can predict the OOD accuracy of classifiers, since OOD agreement unlabeled data. Our prediction algorithm outperforms previous methods both in shifts where agreementtonn-he--iee on the line. This phenomenon also provides new insights into deep neural networks: unlike accuracy-on-the-line, agreement-on-ther -inte--liee appears only hold for neural network classifiers.
OOD benchmarks
models without access to a labeled OOD validation set.
be estimated with just
holds and, surprisingly, when accuracy is not
1 Introduction
Machine learning operates well when models observe and make decisions on data coming from the same distribution the training data. Yet in the real world, this assumption rarely holds. Environ- ments are never fully controlled. Robots interact with their surroundings, effectively changing what they see in the future. Self-driving cars face constant distribution shift when driving to new cities under changing weather conditions. Models trained on clinical data from one hospital face challenges when deployed for different hospital with different subpopulations. Under these premises, practi- tioners constantly face the problem of estimating a model's performance on new data distributions cout-of-disttibution, OOD) that are related to but different from the data distribution that the model was trained on (in-distribution, or ID). Depending on the distribution shift, models may sometimes break catastrophically under new conditions, or may only suffer small degradation in performance. Differentiating between such cases is crucial in practice.
Assessing OOD performance is difficult because in reality, labeled OOD data is often very costly to obtain. On the other hand, unlabeled OOD data is much easier to obtain. A natural question is whether can leverage unlabeled OOD data for estimating the OOD performance. This paradigm of using unlabeled data to predict the OOD generalization performance has received much attention 10 31). Although many different metrics have been proposed, their success varies widely depending on the shift and the ID performance of the model. While it is
recently 229 9 87 22 23
36th Conference on Neural Information Processing Systems (NeurIPS 2022).

---

CIPAR-10.1
CIFAR-10.2
ImageMiet2
Camelyon17 77-4-
90
Accuracy
CIFAR10 10.3 3ese
Agscement
90
Imagetheny Textt
50-
T52
76
70
-0
V10
10
10
10
10
30
50
70
30
30
30
50
70
90
30
50
70
90
10
30
50
70
90
CITAB-11 Tint
CIFAR-10 les.
Imagatfat Tind
Tual
CIFAR, Fog
WILDS
70
F0g Tost
OCID Tem
1.0
9:-
-00
10-
50
30
Figure 1: When ID vs. OOD accuracy is linearly correlated, the vs. OOD agreement is also linearly correlated. Additionally, when ID vs. OOD accuracy is not linearly correlated, agreement is also not linearly correlated. Each blue point in the seatter plot represents the accuracy of single model. Each pink point represents the agreement between pair of models. To avoid cluttering the figure, given models of interest, we only plot random pairs. The are probit scaled as described in Section 3.2
impossible for method to always work with assumptions (291, major obstacle to using these methods is that there currently no understanding when they work or a recipe to detect when their predictions might be unreliable.
demonstrated that in a wide variety of 67]. ImageNetV2 (67.. CIFAR-10C [4), MMW-WILDS [33,, there exists an almost perfect positive linear correlation between the test vs. OOD accuracy of models. When this phenomenon, called accuracy-on-the-ling performance on the ID test data directly leads to improvements on OOD performance. Furthermore, if we have access to the slope and bias of this correlation, predicting OOD accuracy becomes straightforward. Unfortunately, accuracy-on-ther on-the-line not a universal phenomenon. some datasets, such as Camelyon 1amelyon!7-WILDD 17-WILDS [], models with the same ID test performance have OOD performance phenomenon interesting, its practical use
In a separate but related line of work, Miller et al. 55
common OOD prediction benchmarks such as CIFAR-10. 10.1
occurs, improving
that varied largely. Thus, while the accuracy-on-tie is somewhat limited since determining whether sccuracy-on-the-ling holds requires labeled OOD data in the first place.
In this work, we begin by observing an analogous phenomenon based upon agreement rather than accuracy. Specifically, if we consider pairs of neural network of classifiers, and look at the agreement of their predictions (the proportion of cases where they make the same prediction, which requires no labeled data compute), we find that there also often exists strong linear correlation between ID vs. OOD agreement. We call this phenomenon this phenomenon appears be tightly coupled with cot-uaccy-oo-th-lliie when agreement-on-te line holds, accuracy-on -orrrac--o---e-lineee also holds; and when agreement-on-the-leee does sentraccyoonthe-ling Furthermore, when these properties hold, the linear correlations of both and agreement-on-the-iing appear to have roughly the same slope and bias. Interestingly, unlike accuracy-on-ther tineel which appears be a general phenomenon, agreement-on- the-line, especially the fact that the slope and bias of the linear correlation agree across accuracy and agreement, appears occur only for neural networks. Indeed, the phenomenon is quite unintuitive, given that there is no a-priori reason to helieve that agreement and accuracy would be connected in such manner, nonetheless, find this phenomenon occurs repeatedly across multiple datasets and vastly different neural network architectures.
siteement-on-theeee
Importantly, however,
does not hold, neither
This phenomenon is of immediate practical interest. Since agreement-on-the-le can be validated without any labeled OOD data, we can use it as proxy to assess whether accuracy-on-the-line holds, and thus whether ID accuracy is reasonable OOD model selection criteria. Furthermore, since the slope and bias of the agreement-on-the data, (for the cases where seneemeet-on-th--ling holds) we can use this approach to derive simple algorithm for estimating the OOD generalization of classifiers, without any access to labeled OOD data. The approach outperforms competing methods and predicts OOD test error with mean
can also estimated without labeled OOD

---

absolute estimation error of < 2% on datasets where agreement- on-the-ling holds. On datasets where agreement-on-tee -ontteeen---- does not hold, the method as expected does not perform as well, but surprisingly still outperforms competing methods in terms of predicting OOD performance.
To summarize, our contributions are as follows:
1. We discover and empirically analyze the
phenomenon: that ID vs. OOD agreement for pairs of neural network classifiers lies a line preciscly when the corresponding ID vs. OOD accuracy also lies on a line. Furthermore, the slope and bias of these two lines are approximately equal.
2. Leveraging this phenomenon, we develop a simple method for estimating the OOD perfor- mance of classifiers without any access to labeled OOD data (and by observing whether ggreement-on-the-t holds, the method also provides a "sanity check" that these estimates are reasonable). The proposed method outperforms all competing baselines for this task.
2 Related Works
Accuracy-on-the-line Before Miller et al. [59] several other works proposing new benchmarks for performance evaluation 58 69 85
have also observed a strong linear correlation between ID and OOD performance of models. Theoretical analysis of this phenomenon is limited. Miller al. 59 constructs distribution shifts where securacy-on-the..
can be observed assuming the data is Gaussian, however they do not incorporate any assumptions about the classifier. Another work by Mania and Sra [55] analyzes the phenomenon under the assumption that given two models, is unlikely that the lower accuracy model classifies a data point correctly while the higher accuracy model classifies it incorrectly. Such model similarity has been observed by [56] 30]. Recent works have also observed some nonlinear correlations [47] and negative correlations in performance [79 41). In this work, we focus on the results from Miller et al. [59] and leave these nuances for future study.
Estimating ID generalization via agreement. Departing from conventional approaches based on uniform convergence 663 26 60], several recent works 62 30
propose different approaches for estimating generalization error. In particular, this work is closely related to Jiang al. (39]. which shows that the disagreement between two models trained with different random seeds closely tracks the ID generalization error of the models, if the ensemble of the models are well-calibrated. Predicting ID generalization via disagreement has also previously been proposed by Madani et al. [54) and Nakkiran and Bansal [61]. Our method also uses disagreement for estimating performance but, unlike these works, we focus on OOD generalization, and, more importantly, we do not require calibration or models with the same architecture.
39
OOD generalization. The problem of characterizing generalization in the OOD setting is even more challenging than the ID setting. Ben-David et al. [5] provides one of the first uniform-convergent based bounds for domain adaptation, a related but harder framework of improving the OOD perfor- mance of models given unlabeled OOD data and labeled ID data. Several works (57 18 46 build on this approach and extend it to other learning scenarios. Most of these works attempt to bound the difference between ID) and OOD performance via a certain notion of closeness between the original distribution and shifted distribution (e.g., the total variation distance and the HAH H divergence which is related to agreement), and build on the uniform-convergence framework (68) As pointed out by Miller al. [59], these approaches provide upper bounds the OOD performance that grows looser as the distribution shift becomes larger, and the bounds do not capture the precise trends observed in practice. Predicting the actual OOD performance using unlabeled data has gained interest in the past decade. These methods can roughly be divided into three categories:
1. Placing assumptions on the distribution shift. Donmez et al. 24! assume knowledge of the marginal of the shifted label distribution P(y) and show that OOD accuracy can be predicted if the shifted distribution satisfies several properties. Steinhardt and Liang (74) work under the assumption that the data x can be separated into "views" that are conditionally independent given label y. Chen et al. [10] assume prior knowledge about the shift and use importance weighting procedure.
Placing assumptions on the classifiers. Given multiple classifiers of interest, Platanios et al. 64 65] form logical constraints based on assumptions about the hypothesis distribution to identify
3

---

Dataset
Accuracy
Agreement
Confidence Interval
Slope
Bias
R²
Slope
Bias
R²
CIFAR-10.Iv6
0.842
-0.216
0.999
0.857
-0.205
0.997
(-0.046, 0.017)
CIPAR-10.2
0.768
-0.287
0.999
0.839
-0.226
0.996
(-0.120, -0.030)
ImageNetv2
0,946
-0.309
0.997
0.972
-0.274
0.993
(-0.0720, 0.061)
CIFAR-IOC-FOO
0.834
-0.228
0,995
0.870
-0.239
0.996
(-0.077, 0.053)
CIFAR-IOCE IIFRR-1OC-Snow
0.762
-0.289
0.974
0.766
-0.266
0.974
(-0.067, 0.047)
fMoW-WILDS
0.952
-0.163
0.998
0.954
-0.121
0.995
(-0.042, 0.030)
Camelyon 7-WILDS
0.373
0.046
0.263
0.381
0.075
0.226
iWildCam- WWIDDD
0.700
-0.037
0.738
0.411
-0.094
0.424
Table 1: Slope, bias, and cocfficients of determination (R²) values of linear correlations between vs. OOD accuracy and ID vs. OOD agreement. The slope/bias of these linear correlations match when I?² value high (i.e. strong linear correlation). We also look at the 95% confidence interval of the differences in slope for the datasets where we observe strong correlation and are interested in whether the slopes are similar. See Section 3.3 for experimental details.
individual classifier's error. On the other hand, Jaffe et al. (3 relates accuracy to the classifiers' covariance matrix under the assumption that classifiers make independent errors and do better than random.
train regression model over metrics that measure the severity of the distribution shift. Inspired by the observation that the maximum softmax probability (or confidence) for OOD points is typically lower
3. Empirically measuring the distribution shift. A group of works 27
22
23
335 34), Guillory et al. [3 and more recently Garg et al. (29] utilize model confidence to predict accuracy. Chuang al. [14] uses agreement with a set of uomain-invariant predictors as a proxy for the unknown, true target labels. This method was extended upon by Chen et al. [9] which improves the predictors by self-training. Yu et al. [87] observed that the distance between the model of interest and a reference model trained on the pseudolabels of f showed strong linear correlation with OOD accuracy.
Though large number of methods have been proposed, for the large majority, it is not well- understood when they will work. Intuitively, method will work on all shifts without additional assumptions [29] But there some simple general structure to shifts the real world that allows us to reliably predict OOD accuracy? Even if such a structure is not universal, can we easily inspect this structure holds? What is plausible assumption we can make about the OOD behaviour of classifiers? The novelty and significance of our work comes from trying to better understand and address these questions, specifically for neural networks. In this work, observe a phenomenon related to, but stronger than accuracy-on- ao-the-line that allows us to reliably predict the OOD accuracy of neural networks.
3 The agreement-on-the-liee phenomenon
3.1 Notation and setup
Let H denote a set of neural networks trained (Xerain: Mtrain) {(x; v:)) sampled from DID-
Given any pair of models h, h E H, for a distribution D, the expected accuracy and agreement are defined as:
Acc(h) En.y- ((((x)))((>))>--->))))>>)))>>))) - wHe Agr(h, 1') = E.- (h(x) n' (z)
(I)
We assume access to labeled validation set (Xval: Bval) = ((x,, v1) Mer sampled from Dip that allows to estimate the ID accuracy Accip (h) as the sample average of [(h(z) ツ} over the validation set. We do not assume access to labeled OOD validation set, as this often impractical
to obtain, and thereby cannot directly estimate AccooD (h) in similar manner.
Agreement, on the other hand, only requires access to unlabeled data. We assume access to unlabeled samples XooD {z, (2))) mooe from the shifted distribution of interest DooD. Hence, we estimate

---

both the ID and OOD agreement as follows:
Agr (h,
1
Σ
1[(())== = [(()), Agrooo(b,1))
Σ
1(M(c) = (())) (2)
Myal 2xXi
moOD =EXooD
3.2 Experimental setup
We study the ID vs. OOD aceuracy and agreement between pairs of models across more than 20 common OOD benchmarks and hundreds of independently trained neural networks.
Datasets. We present results on dataset shifts in the main paper, and include results for other distribution shifts in the Appendix C These datasets span:
1. Dataset reproductions: CIFAR-10.1 667. CIFAR- 10.2 52
reproductions of CIFAR-10
and ImageNetV2 [67] reproduction of ImageNet 21
2. Synthetic corruptions: CIFAR-10C Fog and CIFAR-10C Snow
134
3. Real-world shifts from 42 satellite images (ILDW-WILDS) images from camera traps in and images of cancer tissue (Camelyon17-w1a.
the wildlife (iWildCam-Winl
Appendix C includes results CINIC-10 19 STL-10 16. and other WILDS and CIFAR-10C benchmarks. We also investigate an analogous phenomenon for the F1-score used to assess the reading comprehension performance of language models on the Amazon-SQuAD benchmark 58
Models. For ImageNetV2, we evaluate 50 ImageNet pretrained models from the timm package. On the remaining 7 shifts, we evaluate all independently trained models in the testbed created and utilized by [59] consisting of 2 150 models for each shift. The evaluated models span a variety of convolutional neural networks (e.g. ResNet [3 DenseNet [36). EfficientNet [78]. VGG [49]) as well as various Vision Transformers 225 All architectures and models are listed in the Appendix [
Probit scaling. Miller et al. [59] report their results after probit scaling & --) the ID vs. OOD accuracies due to better linear apply the same probit transform both accuracy and agreement in our experiments.
3.3 Observations
We empirically observe peculiar phenomenon in deep neural networks, which we call agreement- on-the-line characterized by the following three properties:
Prop(i) When ID vs. OOD accuracy observes a strong linear correlation (2 0.95 R² values), we see that ID OOD agreement is also strongly linearly correlated.
Prop(ii) When both accuracy and agreement observe strong linear correlation, we see that these linear correlations have almost the same slope and bias.
Prop(iii) When the linear correlation of ID vs. OOD accuracy weak (S 0.75 R² values), the linear correlation between ID and OOD agreement is similarly weak.
show the agreement--nn-the-line phenomenon on datasets in Figure aand Table On CIFAR- 10.1, CIFAR- ImageNerV2, CIFAR- 10C Fog/Snow, and RMoW-WILDS. we find that both ID vs. OOD accuracy and agreement observe strong linear correlations, and the linear fits have same slope and bias (Prop(i), Prop(ii)) On the other hand, on the datasets Camelyon 1i--yon77-wwwDD and (WildCam-WII.DS where accuracy is not linearly correlated, agreement also not linearly correlated (Pro)(ii)).
To ensure that the differences in the slopes is not statistically significant, we construct the following hypothesis test. For each dataset, we randomly sample 1000 subsets of 10 models from the model
The probit transform is the inverse of the cumulative density function of the standard Gaussian distribution. "The n² thresholds were only chosen to discretize the strength of the linear correlations as strong or weak for the shifts. As shown in Appendix C the phenomenon actually follows gradient i.e. when the Rº value is higher, the slope/bias of ID OOD accuracy and ID vs. OOD agreement become closer to each other.
5

---

CIFAR 10-C Fog
CIFAR-10 Agreement
Figure 2:
We observe whether the agreement on-the-line phe- nomenon happens across various model classes on the CIFAR-I R- Fog dataset. As shown on the left, the ID vs. OOD accuracy of all model classes lie the same line. We plot ID vs. OOD agreement between pairs of mod- els from the same model class and observe that only the linear corre- lation between ID OOD agree- ment neural networks match that ID vs. OOD accuracy (in red).
Fog Accurary
Fog Agreemeett
30
50
90
30
50
70
90
CIFAR-10 Test Accuracy
CIFAR-10 Test Agroement
Neural Network
Random Forest
AcaBoost Accuracy
testbed and compute the corresponding difference in the slope of the linear fits: SlOPeAcc Slopeagr [ we look the 95% confidence interval the distribution of these slope differences for the 6 datasets where we observe strong correlation. For all datasets with the exception of CIFAR 10.2, the null hypothesis (i.e. difference slope 0) lies within the 95% confidence interval thus cannot be rejected.
3.4 What makes agreement-onn-hhe-ing interesting?
First, agreement can be estimated with just unlabeled data. Hence, the agreement-on-the-lle phenomenon has important practical implications for both checking whether the distribution shift observes accuracy-on-the-le and predicting the actual value of OOD accuracy without any OOD) labels. We present method to estimate OOD) error using this phenomenon in Section
Second, agreement-on ciee--ooo-tttttite tee-ttheeeeeee does not directly follow from accuracy-on sotyy--o--tee-iinne tine-ine. Prior work has observed that expected ID accuracy often equals ID agreement over pairs of models with the same architecture, trained on the same dataset but with different random seeds [39]. Agreement-or on-the-line goes beyond these results in two ways: (i) agreement between models with different architectures (Fig. 2 and (ii) agreement between different checkpoints on the same training run (Fig. 4 is also on the ID vs. OOD agreement line. These ID/OOD agreements do not equal the expected ID/OOD accuracy. Indeed, understanding why agreement-on-ther the theoretical conditions presented in the prior work (39 which do not hold for this expanded set of models. Furthermore, the phenomenon ID and OOD (Accuracy can be thought of as the agreement with the perfect classifier). Even pairs of bad models display this phenomenon. See Appendix for further discussion.
holds requires going beyond
matching slopes not due to the models performing well
Finally, we emphasize that there is something special about neural networks that makes the ID vs. OOD agreement trend identical the ID vs. OOD accuracy trend. This is unlike accuracy-on-the line that holds across a wide range of models including neural networks and classical approaches. Figure 2 shows CIFAR-10 Test CIFAR-106 Fog accuracy and agreement of linear models (e.g. logistic and ridge regression) and various non-linear models (e.g. Kernel SVM 17), k-Nearest Neighbors, Random Forests in Appendix
(6) Random Features 15] AdaBoost [90).. See plots for other datasets [ We look at agreement between pairs of models from the same model family. While Prop(i) seems to hold for several other model families several shifts, Prop(ii) only holds for neural networks, i.e. the slope and bias of the agreement line do not match the slope and bias of the accuracy line for other model families.
4 A method for estimating OOD accuracy
In this section, we describe how the phenomenon of agreement-on-the-iee (described Section 3) offers simple practical method to perform model selection and estimate accuracy under distribution shifts. Recall from Section 3.1 that we have labeled ID validation data (X Mvat) and unlabeled OOD data XooD-

---

Model selection. Without OOD labeled data, can we determine which model is likely to achieve the best OOD performance? When conuracc-oo-th-liiie holds and ID vs. OOD accuracy is lin- early correlated, we can simply pick the model with highest ID accuracy. In practice, how does one determine if accuracy-on-tic agreement-on-then -ottee---n implies Ceeuracy-oo-thelliiee agreement (which can be estimated in (2) are linearly correlated, in order to know if our model selection criterion based on ID accuracy is valid.
holds without labeled OOD data? By Prop(i) and Prop(iii)
Hence, we simply need to check if IID and OOD
OOD error prediction. Agreement-on -ieeee--inee allows us to go beyond model selection and actually predict OOD accuracy. Intuitively, we can estimate the slope and bias of the agreement line with just unlabeled data. By Prop(ii), they match the slope and bias of the accuracy line and hence, we can estimate the OOD accuracy by linearly transforming the ID accuracy (with the appropriate probit scaling). We formalize this intuition below and provide an algorithm for OOD accuracy estimation in Algorithm [ Implementation of our method is available at https: //gggtttbbb.. www.keaa/.///..e agreement- on con-the-line
Recall (Section 3.1 that given n distinct models of interest H. = { we can estimate Acci (h), Agrip (h, h') and AgrooD (h., h²) sample averages over ID labeled validation data and OOD unla- beled data for all h, h H. We now describe an estimator AcCooD(h) for the OOD accuracy of model h E H.
From agreement-on-the-line we know that when ID vs. OOD agreement lies a line for all h, h E H x H. ID vs. OOD accuracy for all h E H would approximately also lie on the same line:
Φ (Accood (h)) a -- (Acci(h))) + 4 -- (AgrooD h')) a + (Agri (h, h'))
(3)
We estimate the slope and bias of the linear fit by performing linear regression after applying a probit transform the agreements follows.
à, arg min D.DER
Σ (Arrooohohh a "(Agri hy)) b)²
(4)
For each model h E H. given its ID validation accuracy, one could simply plug the estimated slope à and bias from (4) and (hc (sample average over validation set) into 3 get estimate of the model's ' OOD accuracy. We call this simple algorithm ALine-S.
Notice that S does not directly use the OOD agreement estimates concerning the model of interest- only use agreements indirectly via the estimates à and b. We find that a better estimator can be obtained by directly using the model's OOD agreement estimates via simple algebra as follows.
First, note that for any pair of models h, h E H. it directly follows from 3 that
1 (AccooD(()) + - (AccooD())) a. Φ (Accppc)))))
+ + -- (ACCID
+
(5)
2
2
By substituting b Φb (Agr ooD ( h')) क --'(AArn (h, into 5). we can get that average
OOD accuracy of any pair of models h, h' E H is
Φb (AAcooo 2
+ (AccooD (A'))
unknown
unknown
④ (Agrood (h, h')) + + (Acci (h)) + x (A'C(ccp x -- (A-((Arr((h,h)))
(6)
2
known (can estimate via sample average over Xoop and (Ni)l)Mai)) Mal))
We can plug in estimates of the terms on the right hand side (á from linear regression and the rest from sample averages. In this way, we can construct a system linear equations of the form (6) involving "unknown" estimates of the probit transformed OOD accuracy of models and other "known" quantities. We solve the system via linear regression to obtain the unknown estimates. We call this procedure ALine-D, and it is described more explicitly in Algorithm [ Note that there must be least models in the set of interest H for the system of linear equations in (6) to have a unique solution.
7

---

Algorithm ALine-D: Predicting OOD Accuracy
1: Input: warD validation samples (XIn-va) val, NID-val), mooD unlabeled samples XOOD, set
containing n models of interest
2: Get (h;) Vi E [n]]
3: Get AgriD (hi, h2) and AgrooD (h,, 1 h2) for all pairs of models i o
4: Get a, b = arg bER Eiz, (Agr (Agrood
(h,, (h a " (bg b)²
nín-1) xn be nin-1 1
5: Initialize A R
6: i i=0
7: for hT. h4 H do
8:
Asj = 2/2 Ask - {i,k)
9:
b. Φ (Agroon((e hx)) + a. (Ac + (Acaa((h:))
(Ag hx))
10:
i=i+1
end for
12: Get wt = arg win ||Aw - bll2
13: return k(w() Vi [n]
Experiments
Datasets and models. We evaluate our methods. the simple plug in of slope/bias estimate ALine-S and the more involved A, on the same models and datasets from Section aad two additional datasets CIFAR- 1OAA--OO-SSaturtee and RxRxI- -WILDS (See Appendix C and [ for details on these datasets). Specifically, we look at CIFAR-10.1 10.1, CIFAR-10.2, ImageNet CIFAR-10C, fMoW- WILDS, and EXRxI-WILLD where we observe strong correlation. We also look the performance of models on datasets where we do not see strong linear correlation, specifically Camelyon-WiInn and FillCCa--WIL.D
Baseline methods. We choose 4 existing unlaheled estimation methods for comparison: Average Threshold Confidence (ATC) by Garg et al. 29 DOC-Feat in Guillory et al. [31, Average Confidence
(AC) in 355, and naive Agreement [54 61 39 All of these methods, like ALine, are based on the
softmax outputs of the model. See Appendix A for more details about previous methods.
We implement the version of ATC that performed best in the paper, i.e. with negative entropy the score function and temperature scaling to calibrate the models in-distribution. Although DOC was deemed the best method in Guillory al. (31. we use DOC-Feat since DOC requires information from multiple OOD datasets. For ATC, DOC, and AC, consistent with the experimental design in Garg al. (28) report the best number achieved between before versus after temperature scaling. We also compare with the more recent, ProjNorm by Yu et al. 87 which showed stronger linear correlation with OOD) accuracy than Rotation (2 and ATC (29 We compare with this method separately Section 5.1 as they do not provide a way to directly estimate the OOD accuracy.
Dataset
ALine-D*
ALine-S
ATC
AC
DOC
Agreement
CIFAR-10.1
1.11
1.17
1.21
4.51
3.87
5.98
CIFAR-10.2
3.93
3,93
4.35
8.23
7.64
5.42
ImageNetV2
2.06
2.08
1.14
66.2
11.50
6.70
CIFAR-
1.45
1.75
1.78
4.47
3.93
3.47
CIFAR-10C-Snen
1.32
1.97
1.31
5.94
5,49
2.57
LITAR-10C-Satur
0.41
0.77
0.69
2.03
1.51
4.14
IMOW-WILDS
1.30
1.44
1.53
2.89
2.60
8.99
RxRXI-WILDS
0.27
0.52
2.97
2.46
0.75
8.69
Camelyon! 7-WILDS
5.47
8.31
11.93
13.30
13.57
6.79
iWildCam WILDS
4.95
6.01
12.12
4,46
5.02
7.35
Table 2: Mean Absolute Error (MAE) of the OOD accuracy predictions with % as units. ALine-D outperforms other methods on both shifts where we do and do not see accuracy- cntteey--o-he--iiie line. denotes our methods.

---

CIFAR-10.2
CIFAR-10-CO
Camelyon17
y-x
ATC
0OD Accuracy
6-8
DOD Accuracy
DOD Accuracy
0.8-
ATC,
ALine-D
0.6
9.8
y=x
0.4
ATC
0.4
ATC
ALine-D
ALine-D
ATC
ALine-D
ALine-D
0.4 Prediction
04 0.5
0.2
64
6+
Prediction
Prediction
Figure 3: Prediction vs. OOD accuracy. We observe the scatter plot of prediction vs. OOD accuracy of ALine-D and ATC, the second best performing method from Table 2 We observe that our linear fit is closer to the diagonal, as ATC underperforms models that have low OOD accuracy.
5.1 Main results: comparison to other methods.
In Table 2 we observe that ALine-D generally outperforms other methods on datasets where agreementtoon-the-liee holds. On ImageNet to ImageNetV2 and CIFAR-10 CIFAR-10C-Snow ATC performs marginally better. As can be seen in Figure ATC generally cannot accurately predict the model's OOD performance for models that do not perform very well. This is consistent with experimental results in [29] and [87). On the other hand, ALine perform equally well on "bad" models land "good" models. In some sense, given collection of models where in the performance of each, ATC, AC, DOC-Feat, and Agreement only utilize information from the model of interest, whereas ALine utilizes the collective information from all models for each individual prediction.
are interested
As expected, on datasets where we do not observe a linear correlation between ID and OOD agreement (and accuracy), ALine does not perform very well, with mean absolute estimation error of around 5%. Interestingly, the other methods also do not perform very well on these datasets. No method successfully predicts the OOD accuracy for every distribution shift. The advantage ALine is that there is a concrete way to verify when the method will successfully predict the OOD accuracy (i.e. check whether agreement is on the line). Other prediction methods do not have any way of characterizing when they will be successful. Finally, we note that ALine-D actually surpasses previous methods even when accuracy-or cecuracy-on-the-iieee -ontte--inee does not hold, suggesting that the algorithm has additional beneficial properties that require further study.
5.2 Correlation analysis
Rather than predicting OOD accuracy, it could be useful to have a metric that just strongly cor- relates with the OOD accuracy, if the application simply requires an understanding relative per- formance such model selection. Recently, Yu et al. 87 proposed ProjNorm, a measure- ment they show has a very strong linear corre- lation with OOD accuracy, moreso than other recent methods including Rotation (23 and ATC
Dataset
ALine-D
ProjNorm
H²
H²
CIFAR-1OC 0.995
0.974
0.98
0.973
Table 3: Correlation analysis. We compare the coeffi-
cients of determination (R*) and rank correlations (p) between ALine-D and ProjNorm, metric shown have stronger correlation than ATC and Rotation.
[29]. To compare Aline-D with ProjNorm, we replicate the CIFAR- 10C study in Yu et al. [87].
where they train a ResNet18 model and predict its performance across corruptions and severity levels of CIFAR-IOC (See their Table 1 in (87). Since ALine-D is an algorithm that requires a set of models for prediction, we use the pretrained models from the CIFAR-10 testbed of Miller et al. (59). as the other models in the set. We look at the linear correlation of the estimates of OOD accuracy and the true accuracy across corruptions and find that ALine-D achieves stronger correlation than ProjNorm (Table 3 See Appendix for more experimental details.

---

Along
Along One Trajectory
Alanp Tragestary
fatuiiy
0.9
Bir poey
Agrarmint
0.5
Irago-eecc Teet
0.4
02
0.3
0.0-
10
0.5
10
A00
CIAA-32
Epocha
Imagehiet Text
Eppcha
Figure 4: ALine-D tracks OOD accuracy across training epochs with a MAE of 2.19% for CIFARIO.1 and 3.61% for ImageNetv2.
5.3 Estimating performance along a training trajectory
We assess whether ALine can be utilized even in situations where the practioner only cares about the performance of a few models. In such situations, one could efficiently gather many models by training single model and saving checkpoints along the way. We analyze whether our phenomenon helpful for predicting such highly correlated hypotheses, instead of independently trained models. In Figure 4 we collect the ID and OOD test predictions every 5 epochs across CIFAR-10 training of a single ResNet18 model and every epoch across ImageNet training of a ResNet50 model. We see that even the agreement between every pair of checkpoints of a model across training is enough to get a good linear fit that matches the slope and bias of ID vs. OOD accuracy. By applying ALine-D to these checkpoints, we get very good estimate of the OOD performance of the model across training epochs though slightly worse than for collection of independently trained models. This suggests that given model of interest, ALine does not require practitioners to train large number of models, but just train one and save its predictions across training iterations. We a more careful ablation study in Appendix G looking at the number of models required for close estimates of accuracy.
6 Conclusion
The contributions of this work are two-fold. First, we observe the agreement-on-tiee on-tee-line phenomenon, and show that it correlates strongly with accuracy-on- the--ine over a range of datasets and models. We also highlight that certain aspects of this phenomenon, namely the fact that the slope and bias of the linear fit is largely the fundamentally seem connected to these classes of models. Second, using this empirical phenomenon, propose surprisingly simple but effective method for predicting OOD accuracy of classifiers, while only having access to unlabeled data from the new domain (and one that can "sanity checked" via testing whether agreement-on-the line holds). Our method outperforms existing fate-of-thheans approaches to this problem. Importantly, we do not claim that this phenomenon is universal, but we found it to be true across extensive range of neural networks and OOD benchmarks that we experimented on. In addition to its practical relevance, this observation itself reveals something very interesting about the way neural networks learn, which leave for future study.
across agreement and accuracy,
specific to neural networks, and thus
Acknowledgments and Disclosure of Funding
We thank Rohan Taori and Saurabh Garg for valuable discussions regarding the model testbeds and temperature scaling used in this work, respectively. Christina Back was supported by Presidential Fellowship sponsored by Carnegie Mellon University. Yiding Jiang was supported by funding from the Bosch Center for Artificial Intelligence. Aditi Raghunathan was supported by Open Philanthropy AI Fellowship.
References
[1] Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, al. From detection of individual metastases classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Transactions Medical Imaging, 2018
10

---

[2] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers, In International Conference on Learning Representations, 2022.
[3] Peter L. Bartlett, Dylan J. Foster, and Matus J. Telgarsky. Opectrally-normalicn margin bounds for neural networks. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017.
[4] Sara Beery, Elijah Cole, and Arvi Gjoka. The iwildcam 2020 competition dataset. arXiv preprint arXiv:2004. 10340, 2020.
[5] Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006.
[6] Leo Breiman. Random forests. Machine Learning. 45(1):5-32, 2001. doi: 10.1023/A:
1010933404324.
[7] Andrew Brock, Soham De, Samuel L. Smith, and Karen Simonyan. High perf-mneree large- scale image recognition without normalization. arXiv, 2021. URL/attss://arxivcco
2102.06171
[8] Chun-Fu Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross- multi-scale vision transformer for image classification. International Conference on Computer Vision, 2021.
[9] Jiefeng Chen, Frederick Liu, Besim Avci, Xi Wu, Yingyu Liang, and Somesh Jha. Detecting errors and estimating accuracy on unlabeled data with self-training ensembles. arXiv preprint arXiv:2106. 15728, 2021.
[10] Mayee Chen, Karan Goel, Nimit S Sohoni, Fait Poms, Kayvon Fatahalian, and Christopher Re. Mandoline: Model evaluation under distribution shift. International Conference on Machine Learning, page 1617-1629, 2021.
[11] Yunpeng Chen, Jianan Li, Huaxin Xiao, Xiaojie Jin, Shuicheng Yan, and Jiashi Feng. Dual path networks. Advances in Neural Information Processing Systems, 2017.
[12] François Chollet. Xception: Deep learning with depthwise separable convolutions. CConference on Computer Vision and Pattern Recognition, 2017.
[13] Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 2018.
[14] Ching Yao Chuang, Antonio Torralba 0001, and Stefanie Jegelka. Estimating generalization under distribution shifts via domain-ine invariatn representations. In Proceedings of the 37th International Conference on Machine Learning. ICML 2020, 13-18 July 2020, Virtual Event, volume 119, pages 1984-1994, 2020.
[15] Adam Coates and Andrew Y Ng. Learning feature representations with k-meuns. Springer, 2012. URL. htt.tttttt///wwwee.ee stanford....oo eetiteeduu//// acwates http://aaaaatpppaepeeee/o/a
pdd]
[16] Adam Coates, Honglak Lee, and Andrew Y. Ng. An analysis of single layer networks in unsupervised feature learning. AISTATS, 2011.
[17] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20(3): 273-297, 1995. doi: 10. 10.1007/BF00994018.
[18] Corinna Cortes, Yishay Mansour, and Mehryar Mohri. Learning bounds for importance weighting. Advances in neural information processing systems, 23, 2010.
[19] Luke N Darlow, Elliot J Crowley, Antreas Antoniou, and Amos J Storkey Cinic- 10 is not imagenet or cifar-10. arXiv, 2018. URL https:////// http://w.rg/b/////aaeeeoo
[20] Stéphane D' Ascoli, Hugo Touvron, Matthew I. Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. In Internation Conference on Machine Learning, pages 2286-2296, 2021.

---

[21] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. Ieee, 2009.
[22] Weijian Deng and Liang Zheng. Are labels always necessary for classifier accuracy evaluation? In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15064- 15073. TEEE Computer Society, 2021. doi: 10. 1 109/CVPR46437. (TW1PPR44477.22....00 2021,01482
[23] Weijian Deng. Stephen Gould, and Liang Zheng. What does rotation prediction tell us about classifier accuracy under varying testing environments? arXiv preprini vrXiv:2106.05000 2021.
[24] Pinar Donmez, Guy Lebanon, and Krishnakumar Balasubramanian. Unsupervised supervised learning i: Estimating classífication and regression errors without labels. Journal of Machine Learning Research, 1444): 2010.
[25] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
[26] Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv: 1703. 1 1008, 2017.
[27] Hady Elsahar and Matthias Gallé, To annotate or not? predicting performance drop under domain shift. In Proceedings of the 2019 Conference Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2163-2173, Hong Kong, China, November 2019, Association for Computational Linguistics. doi: 1/D3911111
[28] S. Garg, Sivaraman Balakrishnan, J. Zico Kolter, and Zachary Chase Lipton. Ratt: Leveraging unlabeled data to guarantee generalization. In International Conference of Machine Learning, 2021.
[29] Saurabh Garg, Sivaraman Balakrishnan, Zachary Chase Lipton, Behnam Neyshabur, and Hanie Sedghi. Leveraging unlabeled data to predict duitr--diittibution performance. In International Conference Learning Representations, 2022.
[30] Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. Advances in Neural Information Processing Systems, 34:23885- 23899, 2021.
[31] Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. Predicting with confidence on unseen distributions. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1114-1122 1124, 2021. doi: 10. (11/////CCCC49222211..0000
[32] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. European Conference on Computer Vision, 2016.
[33] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Computer Vision and Pattern Recognition, 2016.
[34] Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In 7th International Conference on Learning Representations, ICLR, 2019.
[35] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribation examples in neural networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
[36] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Conference Computer Vision and Pattern Recognition, 2017.
12

---

[37] Forrest N. landola, Song Han, Matthew W. Moskewicz. Khalid Ashraf, Williamm J. Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5mb model size. arXiv, 2016.
[38] Ariel Jaffe, Boaz Nadler, and Yuval Kluger. Estimating the accuracies of multiple classifiers without labeled data. In Guy Lebanon and S. V. Vishwanathan. editors, Proceedings of the Eighteenth International Conference on Artificial Intelligence and Stafisfics, volume 38, pages 407-415, 2015.
[39] Yiding Jiang, Vaishnavh Nagarajan, Christina Back, and J Zico Kolter. Assessing generalization of SGD via disagreement. In International Conference on Learning Representations, 2022.
[40] Li Shen Jie Hu and Gang Sun. Squeeze-and-excitatio networks. Conference on Computer Vision and Pattern Recognition, 2018.
[41] Gal Kaplun, Nikhil Ghosh, Saurabh Garg, Boaz Barak, and Preetum Nakkiran. Deconstructing distributions: A pointwise framework of learning, arXiv preprint urXiv:2202.099333 2022.
[42] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang, Wilds: A benchmark of in-the-wild distribution shifts. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5637-5664. PMLR, 18-24 Jul 2021.
[43] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[44] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012.
[45] Alexey Kurakin, Ian J. Goodfellow, Samy Bengio. Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang. Jun Zhu, Xiaolin Hu, Cihang Xie, Jianyu Wang. Zhishuai Zhang, Zhou Ren, Alan Yuille, Sangxia Huang, Yao Zhao, Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, and Motoki Abe. Adversarial attacks and defences competition. CoRR, abs/1804.00097. 2018. http:ttt://rritee org/ /as///b///// 1804/.44.00 1804.00097
[46] Ilja Kuzborskij and Francesco Orabona. Stability and hypothesis transfer learning. In Interna- tional Conference on Machine Learning, pages 942-950 PMLR, 2013.
[47] Weixin Liang, Yining Mao, Yongchan Kwon, Xinyu Yang, and James Zou, On the nonlinear correlation of ml performance across data subpopulations. In JCML 2022: Workshop on Spurious Correlations, Invariance and Stability.
[48] Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei- Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. European Conference on Computer Vision, 2018
[49] Shuying Liu and Weihong Deng. Very deep convolutional neural network based image clas- sification using small training sample size. In 2015 3rd IAPR Asian Conference Pattern Recognition (ACPR), pages 730-734, 2015. doi: 10. 109//CCPR.211..4866999
[50] Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[51] Uya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. International Conference on Learning Representations, 2017.
[52] Shangyun Lu, Bradley Nott, Aaron Olson, Alberto Todeschini, Hossein Vahabi, Yair Car- mon, and Ludwig Schmidt. Harder or different? a closer look at distribution shift in dataset reproduction. ICML Workshop Uncertainty and Robustness in Deep Learning, 2020.
13

---

[53] Ningning Ma, Xiangyu Zhang, Hai-Tao Tao Zheng, and Jian Sun. Shufflenet v2: practical guidelines for efficient enn architecture design. European Conference on Computer Vision, 2018.
[54] Omid Madani, David Pennock, and Gary Flake. Co-validation: Using model disagreement on unlabeled data to validate classification algorithms. In L. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems, volume 17, 2004.
[55] Horia Mania and Suvrit Sra. Why do classifier accuracies show linear trends under distribution shift? arXiv preprint arXiv:2012. 15483, 2020.
[56] Horia Mania, John Miller, Ludwig Schmidt, Moritz Hardt, and Benjamin Recht. Model similarity mitigates test set overuse. Advances in Newral Information Processing Systems, 32, 2019.
[57] Yishay Mansour. Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms, arXiv preprint arXiv:0902.3430) 2009.
[58] John Miller, Karl Krauth, Benjamin Recht, and Ludwig Schmidt. The effect of natural distribu- tion shift on question answering models. In International Conference on Machine Learning. pages
PMLR, 2020.
[59] John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidtš. Accuracy on the line: On the strong correla- tion between out-of-distribution and in-distribution generalization. International Conference on Machine Learning, 2021.
[60] Vaishnavh Nagarajan and Zico Kolter. Deterministic pac-bayesian generalization bounds for deep networks via generalizing resileresi arXiv preprint a9X.: 2019.
[61] Preetum Nakkiran and Yamini Bansal. Distributional gemeralization: A kind of generaliza- tion, 2021. URL htpp:::p/e//eeeeeeieeee http:://pppppeeeeeeeeeeeeeeeerep
[62] Jeffrey Negrea, Gintare Karolina Dziugaite, and Daniel M. Roy. In defense of uniform conver- gence: Generalization via derandomization with an application to interpolating predictors. In International Conference on Machine Learning. ICML'20. JMLR.org. 2020.
[63] Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning. In Advances in Neural Information Processing Systems 30, NeurlPS 2017, 2017.
[64] Emmanouil Platanios, Hoifung Poon, Tom M Mitchell, and Eric J Horvitz. Estimating ac- curacy from unlabeled data: A probabilistic logic approach. In Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017
[65] Emmanouil Antonios Platanios, Avrim Blum, and Tom Mitchell. Estimating accuracy from un- labeled data. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI'14, 14, page 682-691, 691, Arlington, Virginia, USA, 2014. AUAI Press. ISBN 9780974903910.
[66] Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. Design- ing network design spaces. Computer Vision and Pattern Recognition, 2020.
[67] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet classifiers generalize to ImageNet? In Kamalika Chaudhuri and Ruslan Salakhutdinov, edi- tors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5389-5400. PMLR, 09-15 Jun 2019.
[68] levgen Redko, Emilie Morvant, Amaury Habrard, Marc Sebban, and Younès Bennani. A survey on domain adaptation theory: learning bounds and theoretical guarantees. arXiv preprint arXiv:2004. 1829, 2020.
[69] Rebecca Roelofs, Vaishaal Shankar, Benjamin Recht, Sara Tridovich-Keil, Moritz Hardt, John Miller, and Ludwig Schmidt. A meta-analysis of overfitting machine learning. Advances in Neural Information Processing Systems, 32, 2019.
14

---

[70] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang Chigc Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. Conference on Computer Vision and Pattern Recognition, 2018.
[71] Sebastian Schelter, Tammo Rukat, and Felix Biessmann. Learning validate the predictions of black box classifiers on unseen data. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, page 1289-1299, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450367356.
[72] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv, 2015. URL httts:://arxiii awgg/bbs/4499 1556
[73] Aravind Srinivas, Tsung- Yi Lin, Niki Parmar, Jonathon Shlens, Pieter Abbeel, and Ashish Vaswani. Bottleneck transformers for visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16519-16529, June 2021.
[74] Jacob Steinhardt and Perey Liang, Unsupervised risk estimation using only conditional in- dependence structure. In Advances in Neural Information Processing Systems (NeurIPS). 2016.
[75] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. Conference Computer vision and Pattern Recognition, 2015.
[76] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. CoRR, abs/1512.00567, 2015. URL http://arxiv. org/ abs http:/abs111100000 1512.
[77] Mingxing Tan and Quoe Le. Efficientnet: rethinking model scaling for convolutional neural networks. International Conference on Machine Learning. 2019.
[78] Mingxing Tan and Quoe Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97, pages 6105-61 14, 2019.
[79] Damien Teney, Yong Lin, Seong Joon Oh, and Ehsan Abbasnejad. Id and ood performance sometimes inversely correlated on real-world datasets. arXiv preprint arXiv:2209.0066 00020, 2022.
[80] Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Go- ing deeper with image transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 32-42, October 2021.
[81] Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Hechtman, and Jonathon Shlens. Scaling local self-attention for parameter efficient visual backbones. Confer- ence on Computer Vision and Pattern Recognition, 2021.
[82] Ross Wightman. Pytorch image models.
sawerrhh-imagemmmeel. 2019.
[83] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. Computer Vision and Pattern Recognition, 2017.
[84] Weijian Xu, Yifan Xu, Tyler Chang, and Zhuowen Tu. Co-scale conv-attentional image transformers. International Conference on Computer Vision, 2021.
[85] Chhavi Yadav and Léon Bottou. Cold case: The lost mnist digits. Advances in neural information processing systems, 32, 2019.
[86] Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell. Deep layer aggregation, 2019.
[87] Yaodong Yu, Zitong Yang, Alexander Wei, Yi Ma, and Jacob Steinhardt. Predicting out-of- distribution error with the projection norm, 2022.
15

---

[88] Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. Computer Vision and Pattern Recognition. 2018.
[89] Lijia Zhou, Danica J. Sutherland, and Nati Srebro. On uniform convergence and low-norm interpolation learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6867-6877. Curran Associates, Inc., 2020.
[90] Zhu, Hui Zou, Saharon Rosset, and Trevor Hastie. Multi-class adaboost. Statistics and Its Interface, 2, 2009.
16


Reviews:
[{"review_id": "W9fSQIUTuME", "reviewer": "Reviewer_2n7P", "summary": "The paper investigates empirically the important and practically very relevant question of OOD performance. The approach taken builds on previous work of 'accuracy-on-the-line' (which observes the occurrence of strong linear (transformed) correlation of ID vs OOD accuracy in some datasets) and adds the simultaneous evaluation of agreement between pairs of models in the OOD vs ID.\nOn a varied set of ID/OOD Image classification tasks, the work empirically finds that a similar linear (transformed) relationship exists or does not exists jointly for accuracy and agreement. Further the findings show a strong similarity between the slopes and bias of the agreement and accuracy ID vs OOD lines on a significant subset of the studied datasets. Interestingly outside of neural networks this similarity breaks down.\nThrough these observations the authors address an important practical problem of assessing existing models' performance on OOD data, critically, without the availability of labels for the OOD data. \nThe proposed method relies on the observed joint existence (or lack there-off) of accuracy-on-the-line and agreement-on-the-line phenomena --- such that if there is no agreement-on-the-line one can test (verify) it.\nThen, by exploiting the similarity of the lines, they show how to estimate the expected OOD accuracy.\n\nThe paper proposes both a pair-model based assessment as well as a multi-model based assessment which allows for noise and potential bias reductions (e.g. from the transform mismatch).\nFinally and importantly, the paper shows for a specific case that agreement-on-the-line holds also across training. \n\n\n", "questions": "see above", "limitations": "yes", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "$\\textbf{Strengths:}$\n\n\nThe paper adds simple, novel and very powerful set of observations:\n* The existence (or lack there-off) of strong linear (transformed) correlation of accuracy is empirically found to go hand in hand with a strong linear (transformed) correlation of agreement between ID and OOD.\n* When both exist the correlation coefficient are 'similar' (in a large subset of the cases) for neural networks examined.\n\nThese observations are interesting both theoretically (for further research) and practically --- for the pertinent question of how one may assess OOD expected accuracy of a given model given OOD, unlabeled, data.\n\nBy building on the above observations together the authors demonstrate how this practical question may be addressed:\nOOD accuracy feasibility can be verified (if linear correlation is high) and carried out (assuming identical correlation coefficients between accuracy ID vs OOD line and agreement ID vs OOD line hold).\n\nThe method is straightforward and clear given the assumptions. \n\nThroughout the paper is clearly written.\n\n$\\textbf{Weaknesses / areas of improvement:}$\n\n1. Most important --- assessment of single model OOD performance:\n\nIn practice, it is most often the case that the practical question is for a given model --- what would be its OOD accuracy.\nThe authors introduce a path to address that (this is a strength of the paper) though the throughout-training agreement vs accuracy experiment, however, the authors only cursorily touch on the same-model OOD performance with a single example.\nSince this a very powerful and practical setting, seeing that is generalizes across the datasets is important. \nIt is advisable the authors expand this experiment across the datasets and in particular for the imagent/V2 datasets.\n\n2. Generalizability and Implications --- Is agreement-on-the-line a fundamental phenomena, does it hold outside of image classification?\nIt would increase the confidence in the generalizability of these finding (and their applicability) if demonstrated in the NLP context.\nIt would be very good to add for example a checkpointed pre-trained transformers (single model across training for simplicity) and evaluate loss and agreement loss for ID vs OOD.\nEven if the phenomena does not hold in this setting that would increase the insight provided by this paper for further research.\n\nAdding these items would change this to an even stronger paper\n\n$\\underline{Minor:}$\n\nstylistic / typos:\n\nline 112 --- reference 42 duplication\n\nline 231 --- the $\\Phi^{-1}$ probate transform is undefined (left for the reader to piece together)\n\nline 233 --- summation index typo, should be $i \\neq j$  \n\nline 242 --- should read 'substituting $b=...$ into (5)'\n\nIn practice (and in the appendix) the 'almost the same slope and bias' observation seems to hold to various degrees across the data examined and is left unquantified, or only quantified indirectly (e.g. through the MSE of the accuracy estimation in a subset of the cases).\nconsider adding a systematic quantification in the paper across the datasets (with details in the appendix) in terms of MSE OOD accuracy prediction.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KOU0CB_74vi", "reviewer": "Reviewer_ov7g", "summary": "This paper identifies and extensively analyzes an interesting phenomena relating the agreement and accuracy of models on in-distribution and out-of-distribution data. In particular, they find that when there is a linear correlation between in-distribution test accuracy and out-of-distribution test accuracy across a set of distinct models trained on this data (a concept discovered in an earlier paper), there is also a linear correlation between the *agreements* of pairs of models trained on that data. This phenomena is identified to be unique to neural networks, and somewhat surprisingly, can be used in the reverse direction as a method of unsupervised estimating out-of-distribution accuracy by measuring agreement - even in cases where in-distribution and out-of-distribution agreement is only roughly linearly correlated. ", "questions": "   * Is there a statistically significant difference between the slopes and biases of neural networks accuracy-on-the-line vs agreement?", "limitations": "The paper does a nice job describing its limitations, both in the amount of models that need to be trained as well as its reliance on the accuracy-on-the-line phenomena. The paper could benefit from further discussion on kinds of domain shift where agreement-on-the-line is guaranteed to not hold, e.g. if there were a label shift. ", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "   * **Originality**\n     While other papers have used agreement to determine how well a model performs on unlabeled data, to the best of my knowledge, this is a completely novel phenomena that has been identified comparing agreement-on-the-line with the recently discovered accuracy-on-the-line. It is clear how this work differs from previous contributions, and the analysis shows new insights that were not described in previous work. \n   * **Quality**\n     The claims are mostly well-substantiated, with extensive experimentation supporting the major claims. The methods used are appropriate, and both the strengths and the weaknesses of the work is remarked on. A possible area of improvement could be in the theory that agreement-on-the-line and acccuracy-on-the-line having the same slope and bias for neural networks but not other models - for example, this could be rigorously tested with a hypothesis test to give further evidence to that claim. Another area of improvement could be increased evaluation with respect to the methods for predicting OOD accuracy - for example, the ALine method that only uses a single model would be useful to have in Table 2, especially if it is in the main body of the paper. \n   * **Clarity**\n     The paper is very clear, both in its organization as well as its figures. It is clear how to run the experiments and reproduce these results, as well as exactly what the two different algorithms are doing. A minor nitpick would be to remove the citation in the abstract so that it can be read completely stand-alone. \n   * **Significance**\n     The paper is significant both from a theoretical and a practitioners point of view. From the theoretical viewpoint, it provides further evidence that some domain shifts have very particular properties that create both agreement and accuracy-on-the-line phenomena. It is very unclear what that phenomena is or how one might be able to predict whether it is occuring, but it clearly has some interesting implications. For the practitioner, this method provides a method for unsupervised prediction of performance on OOD data that is significantly better than other methods (especially when agreement-on-the-line holds), which is a very valuable contribution that I expect will be used. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5jEoRIC3Gf", "reviewer": "Reviewer_7Nem", "summary": "This paper introduces a finer-grained version of accuracy-on-the-line called agreement-on-the-line, capable of estimating OOD performance without access to OOD labels. Rather than analyzing the correlation between ID and OOD performance metrics like accuracy-on-the-line, the authors analyze the predictions made by pairs of models, and observe that strong agreement between model predictions on ID data correlates with prediction agreement on OOD data. Moreover, agreement-on-the-line can be used as a test for when accuracy-on-the-line is appropriate to use due to the apparent co-occurrence of the two phenomenon. The main advantages of their method over previous works is not having to rely on assumptions about data shift magnitude, and the ability to aggregate information from many pairs of models. also The authors validate their claims empirically on a variety of datasets, and for several model classes.", "questions": "Related to W1, it's not obvious if Proposition 1 is unique to neural networks, or just highly accurate models. Specifically, if the ID and OOD accuracy are strongly correlated across the set of models considered, and the models are also accurate, how do you think minimum accuracy affects agreement correlation? The lower bound on agreement is Acc(M1) - (1 - Acc(M2)), so just by following this lower bound, agreement correlation would be strong. \n\nThere is a bit of a contradiction in the claim that \"In fact, ALine-D supercedes previous methods even when accuracy-on the-line does not hold\". Namely, agreement-on-the-line is introduced as a way of determining when accuracy-on-the-line holds, and then leveraging accuracy-on-the-line to predict OOD performance, so if accuracy-on-the-line does not hold, how can it be the case that ALine-D performs so well when the underlying conditions break down?", "limitations": "There is almost no discussion of limitations, but a couple come to mind. For example, the baselines used in Figure 2 such as linear models and SVMs are not appropriate for CIFAR10 classification. Understandably the authors wanted to see if their results hold for classical ML models, but such models should be evaluated on datasets where they can achieve good performance since conclusions drawn from poorly performing models can be misleading. Also, the experiments section does not investigate why agreement-on-the-line is so effective. It would be nice to see if there is something particular about the optimization procedure used, i.e. the inductive bias of SGD, that enables agreement-on-the-line to work when using model checkpoints. Since the premise of the paper is that the observed phenomenon is peculiar to neural networks, the conclusion should at least discuss what role architecture plays.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "S1: Agreement between models is a clever surrogate for whether accuracy-on-the-line holds. It is fast and simple to compute.\n\nS2: Results are impressive, and consistent across the several datasets considered, particularly for the Camelyon and iWildCam datasets. \n\nS3: The ability to use multiple checkpoints for a single model rather than having to train many independent models is very appealing. This was something not investigated by the accuracy-on-the-line paper, so it is a novel and intriguing result.\n\nW1: An analysis of agreement vs accuracy would be welcome. For example, what is the biggest possible discrepancy in agreement for two models M1 and M2 that have the same ID accuracy Acc? This could be used to bound the difference in slope between the agreement and accuracy lines, when accuracy is not computable.\n\nW2: The definitions of weak (R^2 < 0.75) and strong (R^2 > 0.95) correlation seem a bit arbitrary. Such definitions vary throughout the literature, though an R^2 of 0.75 is generally considered to be quite strong.\n\nW3: Some of the claims in section 5 are a bit hand-wavy. For example, \"Interestingly, the other benchmarks also did not perform very well on these datasets, suggesting that perhaps the success of these prediction methods could also partially be attributed to accuracy-on-the-line\" needs some more justification. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "23SIP-BTD_9", "reviewer": "Reviewer_7TCt", "summary": "This paper describes the agreement-on-the-line phenomenon: instead of relying on accuracy-on-the-line, the paper shows that it is possible to use the agreement of a pair of models as a proxy for the ID vs OOD performance tradeoff. Agreement-on-the-line can be estimated solely from unlabelled data and can be used to predict potential OOD performance. Empirical results based on CIFAR-10, ImageNet, and WILDS OOD data show good OOD performance predictions.", "questions": "- Under what circumstances would we expect an OOD data point to have a widespread label agreement across models? Is it enough for accuracy-on-the-line to hold for this to be true? If so, why?\n- What happens for $0.75 < R^2 < 0.95$?\n- Agreement (equation 2) is defined pairwise between two models. Would it be more appropriate to directly formulate it as a quantity (potentially) involving more than just two models and thereby bypass estimation of individual $A_{ij}$'s and $b_i$'s?\n- Can you provide more details as to why Prop (ii) only holds for neural networks? Is this merely an empirical finding or can you provide a theoretical justification?", "limitations": "The paper does not appear to explicitly discuss major limitations of the presented work. I was not able to find other limitations explicitly mentioned. It would be great if the authors could comment on the first weakness (and question) listed above.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n- Estimating OOD detection and generalization performance of ML models is an important topic.\n- The paper is well written and easy to follow.\n- The proposed method of considering the disagreement between multiple models is simple and does not require labeled OOD data access.\n- Disagreement ensembling can also be performed over checkpointed models instead of multiple fully trained models.\n\n## Weaknesses\n\n- Although it seems unlikely that a broad set of models would all agree on the same prediction for an OOD data point, this event probably has non-zero probability, especially for a low number of models. This means that the proposed method can probably not guarantee to be a good OOD performance predictor under all circumstance. This difficulty seems to stem from the fact that behavior on in-distribution data is well understood while this is not the case for OOD data.\n- Baseline methods for experimental section are not well discussed. It would be helpful to introduce these methods in the related works section.\n- Limitations are not well discussed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept