A Note on "Assessing Generalization of SGD via Disagreement"
Authors: Andreas Kirsch
Yarin Gal
Venue: Accepted by TMLR ()

Abstract: Several recent works find empirically that the average test error of deep neural networks can be estimated via the prediction disagreement of models, which does not require labels. In particular, Jiang et al. (2022) show for the disagreement between two separately trained networks that this `Generalization Disagreement Equality' follows from the well-calibrated nature of deep ensembles under the notion of a proposed `class-aggregated calibration.' In this reproduction, we show that the suggested theory might be impractical because a deep ensemble's calibration can deteriorate as prediction disagreement increases, which is precisely when the coupling of test error and disagreement is of interest, while labels are needed to estimate the calibration on new datasets. Further, we simplify the theoretical statements and proofs, showing them to be straightforward within a probabilistic context, unlike the original hypothesis space view employed by Jiang et al. (2022).

Published in Transactions on Machine Learning Research (10/2022)
A Note on "Assessing Disagreement"
Generalization of SGD via
Andreas Kirsch
ondreas. kiwsskhcsschsccooo o...ee.ssrr.... uk varin.gal@es.co or.
Yarin Gal
OATML, Department of Computer Science University of Owford
Reviewed on OpenReview: https: openrevie.. net/ formm? id= oRPBurZ8Fa
Abstract
Several recent works find empirically that the average test error of deep neural networks can be estimated via the prodiction disagreement of models, which does not require labels. In particular, Jiang et al. (2022) show for the disagreement between two separately trained networks that this 'Generalization Disagreement Equality' follows from the well-calibrated nature of deep ensembles under the notion of proposed 'class-agrregated calibration.' In this reproduction,
show that the suggested theory might be impractical because deep ensemble's calibration can deteriorate prediction disagreement increases, which is precisely when the coupling of test error and disagreement is of interest, while labels are needed to estimate the calibration on new datascts Further, we simplify the thcorctical statements and proofs, showing them to be straight/orward within a probabilistic context, unlike the original hypothesis space view employed by Jiang al. (2022).
1 Introduction
Machine lcarning models can cause harm when their predictions become unreliable, yet we trust them blindly. This is not fiction but has happened in real-world applications of machine learning (Schneider, 2021). Thus, there has been significant research interest model robustness, uncertainty quantification, and bias mitigation, In particular, finding ways to bound the error of a trained deep neural network without access the labels would be of great importance: one could estimate the performance of models in the wild where unlabeled data ubiquitous, labeling is expensive, and the data often does not match the training distribution. Crucially, it would provide signal when to trust the output of model and when to defer to human experts instcad.
Several recent works (Chen et al., 2021; Granese et al., 2021; Garg et al., 2022; Jiang et al., 2022) look at the question of how model predictions in a non- non-Bayesian setting can be used to estimate model accuracy, this paper, we focus on one work? (Jiang et al., 2022) specifically, and examine the thooretical and empirical results from Bayesian perspective.
In Bayesian setting, epistemic uncertainty (Der Kiureghian & Ditlevsen, 2009) captures the uncertainty of model about the reliability of its predictions, that is epistemic uncertainty quantifies the uncertainty of model about its predictive distribution, while aleatoric uncertainty quantifies the ambiguity within the predictive distribution and the label noise, c.f. Kendall & Gal (2017). Epistemic uncertainty thus tells us whether we can trust a model's predictions or not. Assuming a well-specified and well-calibrated Bayesian model, when its predictive distribution has low epistemic uncertainty for an input, can be trusted. But likewise, a Bayesian model's calibration ought deteriorate epistemic uncertainty increases for a sample: in that case, the predictions become less reliable and does the model's calibration.
I Code at https::///// http://ggthaab.cco/laaattttt///t coc///laaccC//222. 01851
2 An ICL.R. 2022 Spotlight, which has spawned additional follow-up works, egg. Back (2022).

---

Published in Transactions on Machine Learning Research (10/2022)
In this context, calibration is an alcatoric metric for model's reliability (Gopal, 2021). Calibration captures how well a model's confidence for a given prediction matches the actual frequency of that prediction in the limit of observations in distribution: when a model is 70% confident about assigning label A, does A indeed occur with 70% probability in these instances?
Jiang et al. (2022); while not. Bayesian, make the very interesting empirical and theoretical discovery that deep ensembles satisfy 'Generulization Disagreement Equality" when they are well-calibrated according to a proposed "class-agge agges-gggeeeted colibration' (or a class-wise calibration") and empirically find that the respective calibration error generally bounds the absolute difference between the test error and 'disogreement rate!' Jiang et al. (2022)'s theory builds upon Nakkiran & Bansal (2020)'s 'Agreement Property' and provides backing for an empirical connection between the lest ertor and disagreement rule of two separately trained networks on the same training data. Yet, while Nakkiran & Bansal (2020) limit the applicability of their Agreement Property to in-distribution data, Jiang et al. (2022) carefully extend it: 'our theory is general and makes no restrictions on the hypothesis class, the algorithm, the source of stochasticity, or the test distributions (which may be different from the training distribulion) with qualified evidence: 'we presenl preliminary observalions showing that GDE is approzimately satisfied even for certain distribution shifts within the PACS (Li et al., 2017) dataset.'
In this paper, we present a new perspective on the theoretical results using a standard probabilistic approach for discriminative (Bayesian) models, whereas Jiang et al. (2022) use a hypothesis space of models that output one-hot predictions. Indeed, their theory does not require one-hot predictions, separately trained models (deep ensembles) or Bayesian models. Moreover, as remarked by the authors, their theoretical results also apply to a single model that outputs softmax probabilities. We will see that our perspective greatly simplifies the results and proofs.
This also means that the employed notion of disagreement rate does not capture epistemic uncertainty but overall uncertainty, similar to the predictive entropy, which is a major difference to Bayesian approaches which can evaluate epistemic uncertainty separately (Smith & Gal, 2018). Overall uncertainty is the sum of aleatoric and epistemic uncertainty. For in-distribution data, epistemic uncertainty will generally be low: overall uncertainty will mainly capture aleatoric uncertainty and align well with (aleatoric) calibration measures. However, under distribution shift, epistemic uncertainty can be a confounding factor.
Importantly, we find that the connection between the proposed calibration metrics and the gap between test. eгror and disagreement rate exists because the introduced notion of dlass-aggregated calibration is so strong that this connection follows almost at once.
Moreover, the suggested approach is circular3: calibration must be measured on the data distribution we want to evaluate. Otherwise, we cannot bound the difference between the test error and the disagreement rate and obtain signal on how trustworthy dataset, limiting practicality. Alternatively, change for different datascts or under distribution shifts, which we show not to hold: doep ensembles are less calibrated the more the ensemble members disagree (even on in-distribution data).
model is. This reintroduces the need for labels the unlabeled
would have to assume that these calibration metrics do not
Lastly, we draw connections and show that the 'class- aggreggggee calibration error' and the 'class-win wise calibration e are equivalent to the 'adaptive calibration crror' and 'static calibration error' introducod in Nixon et al. (2019) and its implementation.
Outline. We introduce the necessary background and notation in $2. In $3 we rephrase the theoretical statements from Jiang et al. (2022) using a paramcter distribution (instead of a version space) and auxiliary random variables. This allows us to simplify the theoretical statements and proofs greatly in $4 and to examine the connection to Nixon et al. (2019). Finally, in $5, we provide empirical evidence that deep ensembles are less calibrated exactly when their ensemble members disagree.
3This was added caveat the camera-ready version of Jiang al. (2022) after reviewing a preprint of this paper. Which not explicitly introduced Jiang et al. (2022) but can be analogously constructed.
2

---

Published in Transactions on Machine Learning Research (10/2022)
2 Background & Setting
We introduce relevant notation, the initial Bayesian formalism, the connection to deep ensembles, and the probabilistic model. We restate the statements from Jiang et al. (2022) using this formalism in $3.
Notation. We use an implicit notation for expectations E[f(X)] when possible. For additional clarity, we also use Ex (X)) and Ep(x) f(x), which fix the random variables and distribution, respectively, when needed.
We will use nested probabilistic expressions of the form E[p(Y = =Y] Prima facie, this seems unambiguous, but is p( Y IX) a transformed random variable of only X or also of Y (and Q): what are we taking the expectation over? This is not always unambiguous, so we disambiguate belween the probability for an event. defined by an expression P[...] = E [J.]. where 1(...} is the indicator function", and probability given specific outcomes for various random variables p(O x)) c.f.:
pl- =Y x E / V3 | |x] = p(). 1 = v),
(1)
which is a transformed random variable of X, while p( = Y X) is simply a (transformed) random variable, applying the probability density on the random variables Y and X. Put differently, Y is bound within the former but not the latter: PI... X is a transformed random variable of X, and any random variable that appcars within the ... is bound within that expression.
Probabilistic Model. We assume classification with K classes. For inputs X with ground-truth labels Y, we have a Bayesian model with parameters Q that makes predictions Ý:
ply. 0. xx) p(y) æ) p(0 z.w) p(w).
We focus on model evaluation. (Input) samples x can come either from 'in-distribution data' which follows the training set or from samples under covariate shift (distribution shift). The expected prediction over the model parameters is the maryinal predictive distribution:
p(O x) En(p(0 [,]
(3)
On p(w). The main emphasis in Bayesian modelling can be Bayesian inference or Bayesian model averaging (Wilson & Izmailov, 2020). Here we concentrate on the model averaging perspective, and for simplicity take the model averaging to be with respect to some distribution p(w). Hence, we will use p(w) the push-forward of models initialized with different initial seeds through SGD to minimize the negative log likelihood with weight decay and a specific learning rate schedule (MLE or MAP) (Mukhoti et al., 2021):
We assume that p(w) is a distribution of possible models we obtain by training with a specific training regime on the training data with different seeds. single w identifies a single trained model.
Assumption 1.
Deep Ensembles. We cast deep ensembles (Hansen & Salamon, 1990; Lakshminarayanan al., 2016), which refer to training multiple models and averaging predictions, into the introduced Bayesian perspective above by viewing them as an empirical finite sample estimate of the parameter distribution p(w). Then, WN,...,Wy ~ p(w) drawn i.i.d. are the ensemble members.
Again, the implicit model parameter distribution p(w) is given by the models that are obtained through training. Hence, we can view the predictions of a deep ensemble or the ensemble's prediction disagreement for specific (or over the data) as empirical estimates of the predictions or the model disagroement using the implicit model distribution, respectively.
Calibration. A model's calibration for given measures how well the model's top-1 (argmax) confidence
Confrop := p(Ý arg max p(Ý k (x)
matches its top-1 accuracy
AcC == P(Y arg max p(p = =x|
(5)
The indicator function 1 when the predicate '...? is true and otherwise.
3

---

Published in Transactions on Machine Learning Research (10/2022)
where we define both as transformed random variables of X. The calibration error is usually defined as the absolute difference between the two:
CE == ACCTopp Confropil
(6)
In general, are interested in the expected calibration error (ECE) over the data distribution (Guo et al., 2017) where we bin samples by their top-1 confidence. Intuitively, the ECE will be low when can trust. the model's top-1 confidence on the given data distribution.
We usually use top-1 predictions machine learning. However, if we were to draw y according to p(ù x)) instead, the (expected) accuracy would be:
Acc == P(Y = 8
(7)
EM( = k x) p(p xI
(8)
= Evlp(( = Y -XI)
(9)
random variable of X. Usually are interested in the accuracy over the whole dataset:
pl = Y| E[Acc] =|Acc] = Ex [Pl Y X]] = Ex,y -Y|])())))))))))
(10)
For example, for binary classification with two classes A and B, if class A appears with probability 0.7 and a model predicts class A with probability 0.2 (and thus class B appears with probability 0.3, which model predicts as 0.8), its accuracy is 0.7 x 0.2 + 0.3 x 0.8 = 0.38, while the top-1 accuracy is 0.3. Likewise, the predicted accuracy is 0.2 + 0.8² = 0.68 while the top-1 predicted accuracy is 0.8.
3 Rephrasing Jiang et al. (2022) in a Probabilistic Context
We present the same theoretical results Jiang et al. (2022) but use a Bayesian formulation instead of a hypothesis space and define the relevant quantities as (transformed) random variables. As such, our definitions and theorems are equivalent and follow the paper but look different. We show these equivalences in SA in the appendix and prove the theorems and statements themselves in the next section.
First, however, we note a distinctive property of Jiang et al. (2022). It is assumed that each p(0 (,w) is always onc-hot for any In practice, this could be achieved by turning neural network's softmax probabilities into one-hot prediction for the arg max class. We call this the Top1- Topl-Output-Proppe
(TOP).
Assumption 2. The Bayesian model p(β, w x)) satisfies TOP: p(0 w.w) is one-hot for all x and w.
Definition 3.1. The test error and disagreement rate, as transformed random variables of Ω (and 2'), are:
Test == p( # |
(11)
= 1 - p| Y [2]
(12)
= 1 Ex,r/p(8 88 =Y 9)),
(13)
= 1 - Ep(c.)) p(p= = ar, R))),
(14)
Dis P(y D |2,522
(15)
= 1 P/ = 8' 1222
(16)
1 E Ex,ylp(f* y (x))") 22
(17)
1 E)(e.0)) p(8' = x, f2))
(18)
where for the disagreement rate, we expand our probabilistic model to take a second model S' with prediction 5r into account (and which uses the same parameter distribution), so:
p(u. G. w. 3 w/ (x) ply x) p(0 w.w) p(w) P(V = 1x w') p(Ω w').
4

---

Published in Transactions on Machine Learning Research (10/2022)
Jiang et al. (2022) then introduce the property of interest:
Definition 3.2. A Bayesian model p(ū, w a:) fulfills the Generalization Disagreement Equality (GDE) when:
Eg [[]] = En.or (Di((2, 12) (+ E[TestError] = E(Dis)).
(19)
When this property holds, we seemingly do not require knowledge of the labels to estimate the (expected) test error: computing the (expected) disagreement rate is sufficient.
Two different types of calibration are then introduced, class-wise and class-agreggtted calibration, and it is shown that they imply the GDE:
Definition 3.3. The Bayesian model p(g.w| x)) satisfies class- wise calibration when for any E [0, 1] and any class E [K]
P(Y lo p( ke X) =)
9) =q
(20)
Similarly, the Bayesian model P(O, w x)) satisfies class-oggregated calibration when for any E [0, 1]: - = = = =
= =
=k =
(21)
Theorem 3.4. When the Boyesian model p(D.wl w x)) salisfies class-wise or class-aggregated calibration, il also satisfies GDE.
Finally, Jiang et al. (2022) introduce the dass-aggregated calibration error similar to the ECE and then use it bound the magnitude of any GDE gap:
Definition 3.5. The oggregagge calibration ertor (CACE) the integral of the absolute difference of the two sides in eq- (21) over possible q [0, 1]:
CACE
p(Y p(P = xI = 9) Eple0 =k x|
x1 = 1))dg.
(22)
4E(0,11
Theorem 3.6. For any Bayesion model p(O, w x)), we have:
E[TestEroor]] - E[Dis]| S CACE.
In the following section, we simplify the definitions and prove the statements using elementary probability theory, showing that notational complexity the main source of complexity.
4 GDE is Class-Aggregan Calibration in Expectation
We show that proof for Theorem 3.6 is trivial if we use different but equivalent definitions of the class-wise and class-aggregate calibration. First though, we establish better understanding for these definitions by examining the GDE property E[TostError] E[Dis]. For this, we expand the definitions of E[TestError) and E[Dis], and use random variables to our advantage.
We define a quantity which will be of intuitive use later on: the predicted accuracy
= = xf k PredAcc := Ep(p(E [x | = =
(23)
random variable of X. It measures the expected accuracy assuming the model's predictions correct, that is the true labels follow p(O (x). This also assumes that we draw Y accordingly and do not always use the top-1 prediction.
Revisiting GDE. On the one hand, we have:
E E[TestError) [P # Y 2]]
(24)
5

---

Published in Transactions on Machine Learning Research (10/2022)
1 P| Y|
(25)
Ex Ip( = [X]
(26)
= 1 E[Acc]
(27)
and on the other hand, we have:
E[Dis] = Eg [P| #8'12, 12, 21]]
(28)
En.r [F[ 8 [2
(29)
= 1- - Ple - Y4
(30)
1 Ex (p(" =
. x)]
(31)
- E (p(Ý) |x]
(32)
E.PredAcc].
(33)
The step from (30) to (31) is valid because γ 8' X, and the step from (31) to (32) is valid because p(f'
2) p( x)). Thus, we can rewrite Theorem 3.4 as:
Lemma 4.1. The model p(ù x) satisfies GDE, when
E[Acc] = Elp(Y =X Elp X)] E[PredAcc].
(34)
i.e. in expectation, the accuracy of the model equals the predicled accuracy of the model, or equivalently, the error of the model equals its predicted error.
Crucially, while Jiang al. (2022) calls Ex, [p(P X)] the (expected) disagreement rate E(Dis), it actually is just the predicted crror of the (Baycsian) model as whole.
Equally important, all dependencies on Ω have vanished. Indeed, we will not use Ω anymore for the remainder of this section. This reproduces the corresponding remark from Jiang et al. (2022)6;
The theoretical statements in Jiang et al. (2022) can be made about any discriminative model with predictions p(y] z).
Conclusion 1.
When is lxy = y Xx] = Ex,y lp( [x)]? Or in other words: when does p(Y = 0 (x) equal p( 9 x) in expectation over p(z, U)9
As a trivial sufficient condition, when the predictive distribution matches our data distribution i.e. when the model p(O r)) is perfectly calibrated on average for all classes-and not only for the top-1 predicted class. ECE = not sufficient because the standard calibration error only ensures that the data distribution and predictive distribution match for the top-1 predicted class (Nixon al., 2019). But dassswise calibration entails this equality.
Class Wise and Class-Aggregater Calibration. To see this, we rewrite class-wise and class-aggregated calibration slightly by employing the following tautology:
p k p() = - 9) =4
(35)
which is obviously true due its self-referential nature. We provide formal proof in $D in the appendix. Then we have the following equivalent definition:
Lemma 4.2. The model p(ğ æ)) satisfies class-wisc calibration when for any q E [0, 1] and any class k E [K]:
p(Y k. pf k p(8 k. kp = k (X) q)
(36)
6Te remark did exist in the first preprint version.

---

Published in Transactions on Machine Learning Research (10/2022)
Similarly, the model p( x)) satisfies class lags-aggre calibration when for any q [0, 1]:
p(p(Y - Y [X) [ - p(p(y [x] =)), 4),
(37)
and class-wisc calibration implies class-agregate calibration.
The straightforward proof found in SD in the appendix.
Jiang et al. (2022) mention 'level sets' as intuition in their proof sketch. Here, we have been able to make this even clearer: class-aggregated calibration means that level-sets for accuracy p( = Y [X) and predicted
accuracy p(Y X)) random variables of Y and X, and y and X, respectively-hate equal mcasure, that is probability, for all 9.
GDE. Now, class-nggregated calibration immediately and trivially implies GDE. To see this, the following property of expoctations:
Lemma 4.3. For a random voriable X, function t(x), and the random variable T =((),, it holds that Er(7] [[T] = Ex (x))
(38)
This basic property states that can either compute expectation over T' by integrating over p(T = t) or by integrating t(x) over p(X = z). This is just change of variable (push-forward of a measure).
We can use this property together with the class-aggrogated calibration to see:
E[Acc]
E[PredAcc] II
Ex,y = =Y 1- X)]
x (p(Y [x)] II
E(p(E =Y]
E[p(Ý x)]
E v
x
(39)
which is exactly Lemma 4.1, where we start with the equality following from class-aggregated calibration and then apply Lemma 4.3 along each side. Thus, GDE but an expectation over class-aggregated calibration; have:
Theorem 4.4. When model p(ğ x) satisfies class-wiss wise or class-aggregated calibration, it satisfies GDE.
Proof. We can formalize the proof to be even more explicit and introduce two auxiliary random variables:
S : p XX)
(40)
as transformed random variable of Y and X, and
T p( X),
(41)
transformed random variable of y and X. Class-wise calibration implies class-aggregated calibration. Class- Dlass-aggre calibration then is p(S q) pTT q) (*). Writing out eq. (39), we have
Elp(Y Y X)) = Ex,y [S] = E[S] [s([S]
(42)
/ p(s 9) dy
(43)
( / p(T g) dg
(44)
Ey[T] E[T] = Ex 77 =Elp( {X),,
(45)
which concludes the proof.

---

Published in Transactions on Machine Learning Research (10/2022)
The reader is invited to compare this derivation to the corresponding longer proof in the appendix of Jiang et al (2022). The fully probabilistic perspective greatly simplifies the results, and the proofs are straightforward. CACE. Showing that CACE bounds the gap between test error and disagreement is also straightforward:
Theorem 4.5, For any model p(ğ ()), we have:
B[TestError]] - E[[iss] s CACE.
Proof. First, we note that
CACE = S vE(011)
|
=Y -0p((P |x)= - (x)=))ld4
(46)
following the equivalences in the proof of Lemma 4.2. Then using the triangle incquality for integrals and 0 S S 1, we obtain:
CACE
(47)
[ p(p(F =Y
| X) q) - -(p(p = 8 (X] 4) dg
(48)
VEjO.11
a ppp(((8 Y x) q) -PPP( & () = 9) dq
(49)
gEfo.11
vee 9 p(p(} = Y x) = q) dg / ! p(p(() (X) = 9) dg/-
(50)
[[S] E[T]]
(51)
E Error - E[Dis]]
(52)
where we have used the monotonicity of integration in (49) and the triangle inequality in (50).
The bound also serves another-ever even simpler-proof for Theorem 4.4:
Conclusion 2. When the Bayesian model satifies class-wise class-aggregated calibration, have CACE = 0 and thus E E(TestError] E[Dis], i.c. the model satisfies GDE.
Furthermore, note again that Bayesian model not necessary for the last two theorems. The model parameters Ω were not mentioned except for the specific definitions of TestError and Dis which depend Ω following Jiang et al. (2022) but which we only use in expectation.
Moreover, we see that we can easily upper-bound CACE using the triangle inequality by 2, narrowing the statement in Jiang et al. (2022) that CACE can lie anywhere in [0, K]: Conclusion 3. CACE < 2.
Additionally, for completeness, we can also define the class-wist wise calibration error formally and show that it is bounded by CACE using the triangle inequality: Definition 4.6. The class-wise colibration error (CWCE) definod as:
CWCE = Σ/ & gE(o.1]
p(Y = kpp = x = 4) - pp(p = kk = x = 4)
(53)
Lemma 4.7. CWCE 2 CACE E [Acc] E Predd
Note that when compute CACE empirically, divide the dataset into several bins for different intervals of p( = k X). Jiang et al. (2022) use 15 bins. If we were to use a single bin, we would compute [[[cc] - E[PredAce]] directly.
In $B we show that CWCE has previously been introduced as 'adaptive calibration error' in Nixon et al. (2019) and CACE as 'static calibration error" (with noteworthy differences between Nixon et al. (2019) and its implementation).
8

---

Published in Transactions on Machine Learning Research (10/2022)
Measure
CWCE
CACE
Test Error
ECE
Test Error - Predicted Error
CIFAR-10 (In-Distribution)
0.04
0.03
Valve
0.02
0.01
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Predicted Error (Disagreement Rate)
Predicted Error (Disagreement Rate)
CINIC-10 (Distribution Shift)
Value
0.15
0.10
0.05
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0,2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Predicted Error (Disagreement Rate)
Predicted Error (Disagreement Rate)
(a) Ensemble with TOP
(b) Ensemble without: TOP (same underlying models)
Figure 1: Rejection Plot of Calibration Metrics for Increasing Disagreement In-Distribution (CIFAR-10) and Under Distribution Shift (CINIC-10). Different calibration metrics (ECE, CWCE, CACE) vary across CIFAR-10 and CINIC-10 on an ensemble of 25 Wide Sidde-ResNet-28-11 model trained on CIFAR-10, depending the rejection threshold of the predicted error (disagreement rate). Thus, calibration cannot be assumed constant for in-distribution data or under distribution shift. The test error increases almost linearly with the predicted error (disagreement rate), leading to 'GDE gap' Test Error - Predicted Error| becoming almost. flat, providing evidence for the empirical observations in Nakkiran & Bansal (2020); Jiang et al. (2022). The mcan predicted error (disagreement rate) is shown on the x-axis. (a) shows results for an ensemble using TOP (following Jiang et. al. (2022)), and (b) for a regular deep ensemble without TOP. The regular deep ensemble is better calibrated but has higher test error overall and lower test error for samples with small predicted error.

---

Published in Transactions on Machine Learning Research (10/2022)
5 Deterioration of Calibration under Increasing Disagreement
Generally, we can only hope to trust model calibration for in-distribution data, while under distribution shift, the calibration ought to deteriorate. In our empirical falsífication using models trained on CIFAR-10 and evaluated on the test sets of CIFAR-10 and CLNIC-10, as dataset with distribution shift, we find in both cases that calibration deteriorates under increasing disagreement. We further examine ImageNET and PACS in $E.2. Most importantly though, calibration markedly worsens under distribution shift.
Specifically, we examine an ensemble of 25 WideResNet models (Zagoruyko & Komodakis, 2016) trained on CIFAR-10 (Krizhevsky et al., 2009) and evaluated on CIFAR-10 and CINIC-10 test data. CINIC-10 (Darlow et al., 2018) consists of CIFAR-10 and downscaled ImageNet samples for the same classes, and thus includes a distribution shift. The training sctup follows the one described in Mukhoti et al. (2021), see appondix $E.1.
Figure 1 shows rejection plots under increasing disagreement for in-distribution data (CIFAR-10) and under distribution shift (CINIC-10). The rejection plots threshold the dataset increasing levels of the predicted error (disagrecment ratc)-which is mcasure of epistemic uncertainty when there is no expected alcatoric uncertainly in the dataset. We examine ECE, class-aggregated calibration error (CACE), class-wise calibration error (CWCE), error E| ErrorrE and 'GDE, gap', | [E[Acc - EccredAcc]], as the predicted error (disagreement rate), E[Dis] = 1 - E[Pre,Acc]] increases. We observe that all calibration metrics, ECE, CACE and CWCE, deteriorate under increasing disagreement, both in distribution and under distribution shift, and also worsen under distribution shift overall.
We also observe the same for ImageNet (Deng et al., 2009) and PACS (Li et al., 2017), which we show in appendix $E.2.
This is consistent with the experimental results of Ovadia et al. (2019) which examines dataset shifts. However, given that the calibration metrics change with the quantity of interest, we conclude that:
Conclusion 4. The bound from Theorem 3.6 might not have as much expressive power as hoped since the calibration metrics themselves deteriorate as the model becomes more 'uncertain' about the data.
At the same time, the 'GDE gap', which is the actual gap between test error and predicted error, flattens, and the test error develops an almost linear relationship with the predicted error (up to bias). This shows that there seem to be intriguing empirical properties of deep ensemble as observed previously (Nakkiran & Bansal, 2020; Jiang et al., 2022). However, they are not explained by the proposed calibration metrics".
As described previously, the results are not limited to Bayesian or version-space models but also apply to any model P(0 r)) including regular deep ensembles without TOP. In our experiment, we find that regular deep ensemble is better calibrated than the same ensemble made to satisfy TOP. We hypothesize that each ensemble member's own predictive distribution is better calibrated than its one-hot outputs, yielding a better calibrated ensemble overall.
Given that all these calibration metrics require access to the labels, and we cannot assume the model to be calibrated under distribution shift, we might just as well use the labels directly to asses the test error.
6 Discussion
Here, we discuss connections to Bayesian model disagreement and epistemic uncertainly, as well as connections to information theory. We expand on these points in much greater detail in appendix $C.
Bayesian Model Disagreement. From a Bayesian perspective, as the epistemic uncertainty increases, we expect the model to become less reliable in its predictions. The predicted error of the model is a measure of the model's overall uncortainty, which is the total of alcatoric and epistemic uncertainty and thus correlated with epistemic uncertainty. Thus, we can hypothesize that as the predicted error increases, the model should
The simplest explanation that very few samples have high predicted emoor and thus the rejection plots flatten. This not true. For CINIC-10, 10, the first. bucket, contains 50k samples, and each latter buckets adds additional ~10k samples.
10

---

Published in Transactions on Machine Learning Research (10/2022)
become less reliable, which will be reflected in increasing calibration metrics. This is exactly what we have empirically validated in the previous section.
At first sight, Jiang et al. (2022) seems disconnected from information theory. However, we can draw a connection by using h(p) = as a linear approximation for Shannon's 's information content h(p) = - logp. p. Semantically, both this approximation and Shannon's 's information content. quantify surprise (i.e., prediction error). Both are 0 for certain events. For unlikely events, the former tends to while the latter tends to +00.
Connection to Information Theory.
This leads to common-sense definitions and statements from an information- point of view. We can even formulate parallel statements using information theory and see that the statements relate to total uncertainly and not epistemic uncertainly in a Bayesian sense.
Other Related Literature. Beyond Jiang et al. (2022), this note offers a perspective on Granese et al. (2021) and Garg et al. (2022), which are proposing related approaches.
Granese et al. (2021) use the predicted error (disagreement rate), referred to Da and the predicted top-1 error, D3, to estimate when the model will be wrong. As noted in $C, the predicted error can be seen an approximation of Shannon's entropy. Thus, Da is effectively using an approximation of the prediction entropy for OOD detection and rejection classífication. Similarly, D3 is the maximum class confidence. Both are well-known baselines for OOD detection (Hendrycks & Gimpel, 2016). There is no ablation to see how Da and D3 differ from these baselines. We leave this to future work. The paper frames the question of whether model's predictions will be correct as binary classification problem on top of the underlying model's 's output probabilities and investigate this from theoretical point of view. They also examine using input. perturbations similar to Liang et al. (2017) and Lee et al. (2018).
Garg et al. (2022) threshold the predictive entropy or maximum class confidence to estimate the test error under distribution shift. They estimate the threshold by calibrating it on in-distribution labelled data: the threshold is chosen such that the percentage of rejected in-distribution validation data approximately equals the test error on this in-distribution data. They call this approach Average Thresholded Confidence (ATC). They find that ATC using entropy performs better than ATC using the maximum confidence and other approaches, including GDE. Their results show that ATC also degrados under increasing distribution shifts similar to what we have seen for GDE in $5 the choice of threshold is explicitly lied to the in-distribution Garg et al. (2022) explicitly examine the theoretical limits when no further assumptions are made.
7 Conclusion
We have found that the theoretical statements in Jiang et al. (2022) can be expressed and proven more concisely when using probabilistic notation for (Bayesian) models that output softmax probabilities.
Morcover, we empirically found the proposed calibration metrics to deteriorate under increasing disagreement for in-distribution data, and as expected, we have found the same behavior under distribution shifts.
While Jiang et al. (2022) are careful to qualify their results for distribution shifts, above results should give pause: strong assumptions are still needed to conjecture about model generalization, and we need to beware of circular arguments.
Acknowledgements
The authors would like to thank the anonymous TMLR reviewers for their kind, constructive, and helpful feedback during the review process. Moreover, we would like to thank Jannik Kossen, Joost van Amersfoort, and the members of OATML in general for their feedback at various stages of the project, and the authors for Jiang al. (2022) for constructively engaging with a preprint of this work. AK is supported by the UK EPSRC CDT in Autonomous Intelligent Machines and Systems (grant reference EP FPPTL055897/1).
8 SSee also Figures 7, 8, and 9 the appendix Garg al. (2022)
11

---

Published in Transactions on Machine Learning Research (10/2022)
References
Christina Baek, Yiding Jiang, Aditi Raghunathan, and Zico Kolter. Agreement-on-tth-linne Predicting the performance of neural networks under distribution shift, 2022.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers, 2021.
Jiefeng Chen, Frederick Liu, Besim Avci Xi Wu, Yingyu Liang, and Somesh Tha. Detecting errors and estimating accuracy unlabeled data with self-training ensembles, 2021.
Luke Darlow, Elliot J Crowley, Antreas Antoniou, and Amos Storkey. Cinic-10 is not imagenet or cifar- 10. Xiv preprint Xr: 03000, 2018.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pottern recognàtion, pp. Iece, 2009.
Armen Der Kiureghian and Ove Ditlevsen. Alcatory or epistemic? does it matter? Structural safety, 31(2): 105-112, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition at scale, 2020.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In PMLR, 2017.
International Conference on Machine Learning, pp.
Saurabh Garg, Sivaraman Balakrishnan, Zachary C. Lipton, Behnam Neyshabur, and Hanie Sedghi. Leveraging unlabeled data to predict distribiist performance, 2022.
Achintya Gopal. Why calibration error is wrong given model uncertainty: Using posterior predictive checks with deep learning, 2021.
Federica Granese, Marco Romanelli, Daniele Gorla, Catuscia Palamidessi, and Pablo Piantanida. Doctor: A simple method for detecting misclassification errors, 2021.
Chuan Gui, Gcoff Plciss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pp.
PMLR, 2017.
Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE transactions on pottern analysis and machine intelligence, 12(10):993-100) 1001, 1990.
Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. Bag of tricks for image classification with convolutional neural networks, 2018.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks, 2016.
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.57 2011.
Yiding Jiang, Vaishnavh Nagarajan, Christina Back, and J Zico Kolter. Assessing generalization of SGD via disagroement. Tn International Conference on Learning Representations, 2022. URL https://openreviev. net//forumm?ddewWoCCAAAA
Alex Kendall and Yarin Gal. What uncortainties do we need in bayesian deep learning for computer vision?, 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arvi: 1412.6980 2014.
12

---

Published in Transactions on Machine Learning Research (10/2022)
Andreas Kirsch and Yarin Gal. A practical & unified notation for information-theorett quantities in ml. arXiv preprint arXim:2100 12062, 2021.
Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. In Advances in Neural Information Processing Systems, pp. 7024-7035, 2019.
Andreas Kinsch, Clare Lyle, and Yarin Gal. Unpacking information bottlenecks: Unifying informationn objectives in deep learning. arXiv preprint arXi::2003. 12537, 2020.
Alex Krizhevsky, GeofTrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint or Ar1:::1622.014 2016.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of- distribution samples and adversarial attacks, 2018
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pp. 5542-55 5550, 2017.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks, 2017.
Dennis V Lindley. On a measure of the information provided by an experiment. The Annals of Mathemalionl Statistics, pp. 986-1005, 1956.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xic. A convnct for the 2020s. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2022. doi: 1411099cvpr5268..0 000.5526688.222.1166..7 URL http://dx. doi aoi..or//00.0 10. 10t.1119////v c109//vpr52688.... 2022. 01 01167.
Andrew McCallum and Kamal Nigam. Employing em and pool-based active learning for text classification. In ICML, 1998.
Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philipp HS Torr, and Yarin Gal. Deterministic neural networks with appropriate inductive biases capture epistemic and aleatoric uncertainty. arXiv preprint arXiv:2102.1 112102211550 11582, 2021.
Preetum Nakkiran and Yamini Bansal. Distributional generalization: A new kind of generalization. arXiv preprint ar (xXi::2009.0809,, 2020.
Jeremy Nixon, Michael W Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In CVPR Workshops, 2019.
Yaniy Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. arXiv preprint ar Xiv: 1906. 02530, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca. Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc, E. Fox, and R. Garnett (ods.), Advances in Newral Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019. URL http://papers. hwtppppppp....ppss..pppp cttpps.cccppppp//ppp wwptorch--n-iipprattee wtw.----yy----iggg--ett-aaa---a- deep- www.rrrrbbpp... .df.
Shira. Schneider. Algorithmic Bias: A New Age of Racism. PhD thesis, 2021.
13

---

Published in Transactions on Machine Learning Research (10/2022)
Lewis Smith and Yarin Gal. Understanding measures of uncertainty for adversarial example detection. arXiv preprinl Xiv: 100::803.00 08533, 2018.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient. image transformers distillation through attention, 2020.
Ross Wightman. Pytorch image models. http:://ggthuub..c//rr/gggaaaaa htwptiaawaaytttrcomm-e-----eeeee 2019.
Andrew Gordon Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generalization, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. ar preprint
2016.
14

---

Published in Transactions on Machine Learning Research (10/2022)
A Equivalent Definitions
Jiang et al. (2022) defines a hypothesis space H. In the literature, this is also sometimes called a version space. The hypothesis space induced by a stochastic training algorithm A is named HAA
We can identify each hypothesis : X' + [K] with itself as parameter wh = h and define p(wn) as a uniform in HA. This has the advantage of formalizing the distribution
distribution over all srrammeters/yypohheeee
from which hypothesis are drawn (h Ha), which not made explicit in Jiang al. (2022). h(x) k then becomes arg max; p( z.wh) = k. Moreover, as p(g. x)) satisfies TOP, have?
"I(h(x) = k3" = p(8 = k. 28, wh).
(54)
Thus, "TestErrg [g(1(((X) * YJ] is equivalent. to:
" TresErrg (h) E=] # YII"
(55)
= Ex.rlp(Y # Y X, wi)l
(56)
Ex[PI& + Y X wall
(57)
= PlL * Y wn.
(58)
TestErrorwuh).
(59)
Similarly, "Disø (h, h) & Eg (1(x) # " is equivalent to:
" DDig (h, h² A Egg [1 (M(X) + (X)]]"
(60)
Ev [Pl + Y' wh. whll
(61)
Dis(wn, wh).
(62)
Further, (x) 2 (1(((((((x) = k|]" equivalent to:
"hx(x) A Evea (1(((x) k)]""
(63)
= =Er = k (1,
(64)
=p(8 k |r).
(65)
For the GDE, "Ep, NA Diig (h,, N(] = ENneE (TestErr(h)) is equivalent to:
"E, "E [Dis (h, h')] Een [TestEr(h)]] < En,ov Dis S2') Es (rso
(66)
For the class-wise calibration, "p(Y = hx()) q) q" is equivalent to:
"p(Y = 11h((X)) =9) q + p(Y = k 11p(8 = * x) = =q) =
(67)
(68)
p(Y p(y=t,h, q" (and note in Jiang et al. (2022), class indices
For the class-aggregated calibration, run from O..K - 1) is equivalent to:
Lnd ((hx(X)=m)
E6 p(Y k, hx 9"
(69)
EK- (X) =
We put definitions expressions written using the notation and variables from Jiang (2022) inside quotation marks ent avoid ambiguities.
15

---

Published in Transactions on Machine Learning Research (10/2022)
ply = k, k, = x) q) K k=
(70)
2k=1 p(p(p =k -k|X)= q)
k
p(Y k. k.p(P = =k 11 9)
k=1
K
Σ p(p(8 = - q).
(71)
Finally, for the class-aggregated calibration error, the definition is equivalent to:
*CACEs(h)
/ Ieela.11
=,p(Y=h,h((x)=
Exp(ha(X)=0) o Σ (bx((())= =q) dg
(72)
/ 1Ejo.11
1 p k. k,,x(X) = 9) - Ep(hh(e = q) ldg"
(73)
k
Σp(Y= p(Y = kk (X) = - EP(P(E = k 1| dd
(74)
qE(0.11
B Comparison of CACE and CWCE with calibration metrics with 'adaptive calibration error' and 'static calibration error'
Nixon et al. (2019) examine shortcomings of the ECE metric and identify a lack of class conditionality, adaptivity and the focus on the maximum probability (argmax class) as issues. They suggest an adaptive calibration error which uses adaptive binning and averages of the calibration error separately for each class, thus equivalent to the class-wise calibration error and class-wise calibration (up to adaptive vs, even binning). In the paper, the static calibration crror is defined as ACE with even instead of adaptive binning. However, in the widely used implemeetattin 10, SCE is defined as equivalent to the class-aggregated calibration error.
C Expanded Discussion
Here, we discuss connections to Bayesian model disagreement and epistemic uncertainty, as well as connections to information theory, the bias-variance trade-off, and prior literature.
C.1 Bayesian Model Disagreement
From a Bayesian perspective, as the epistemic uncertainty increases, we expect the model to become less reliable in its predictions. The predicted error of the model a measure of the overall uncertainty of the model which is the total of aleatoric and epistemic uncertainty, and thus correlated with epistemic uncertainty. Thus, we can hypothesize that as the predicted error increases, the model should become less reliable, which will be reflected in increasing calibration metrics. This is what we have empirically validated in the previous section. We can expand on the connection to the Bayesian perspective. In particular, we can connect the statements of Jiang al. (2022) to well-known Bayesian measure of model disagreement.
We follow notation introduced in Kirsch & Gal (2021). In particular, for entropy, we use an implicit or explicit notation, H[] or H(p(X)), and use a notation for the cross-entropy H(p [] which is follows the Kullback-Leible divergence (KI(p || q). We base these definitions on Shannon's information content h(p)):
Information Theory.
h(p)) = log p.
(75)
com/ http://///gtttaaao/////eeeeeeere otg. robustness haaustess...tttseeeeerr//aaaaaat py8L1585, added in April 2021
16

---

Published in Transactions on Machine Learning Research (10/2022)
H(p(X)) (X)) = p(a) [x(()))
(76)
H[ := H(p(X)) = H(p(X) IP(X),
(77)
(xx(p(X) ((X)) == H(p(X) [[(X)) - H(p(X)),
(78)
where and q are probabilities distributions. Conditional and joint entropies are defined as usual. For completeness, for random variables X and Y:
H[x |Y] := Ep(x.w) log p( v)]
(79)
Finally; the mutual information T[ for random variables X, Y is defined expected uncertainty reduction also sometimes called expected information gain (Lindley, 1956):
[[ H[X] H
(80)
as the entropy H[X] can be seen as quantifying the 'uncertainty" about the random variable X, and H[X as expected uncertainty about X after observing Y.
YY]
Bayesian Model Disagrcement. The mutual information 1 xr] between predictions Ý and model parameters Ω for given sample is well-known quantity that measures the model disagreement for that sample. It can be computed without having to perform Bayesian inference:
1 []] H[y
x²
H[ 2x S2]
(81)
= EE y x - p XX)
(82)
Eal- p( XX]
To see that 1[Ý; S2 zi] measures model disagreement, observe that if each different one-hot class prediction Y, the second term would be 0, while the first term H[Y æ] would be maximal = log K as the predictive distribution would be uniform. Equally, the predictions for each w agreed, we would have p(ğ œ) = =p | x, w) for all w, the first and second term would be equal, and the mutual information would be 0 (Kirsch al., 2019).
equally likely to obtain a
Model disagreement proxy of epistemic uncertainty. Epistemic uncertainty measures model-dependent uncertainty (i. reliability) and is also known as reducible uncertainty: if we were to update the model with a label for using Bayesian inference, the updated model's uncertainly for x would reduce. For this reason, [[ æ] is often used in Bayesian active learning, where it is known as BALD (Bayesian Active Learning by Disagreement) (Houlsby et al., 2011), or in Bayesian optimal experiment design, where it is known as Expected Information Gain (Lindley, 1956). In non-Bayesian active learning, the very same term is known as Query-by- by- Committee (MeCallum & Nigam, 1998) and computed via Kullback-Leilb divergence:
1 2 x] = H[Ý x] - H[ [r
( Ix, Ω) || p( ==)).
(83)
Taking an expectation, we obtain the expected model disagreement over the data T[; X] X]].
Ex T
In $E.2, we also report empirical results for rejection plots based on Bayesian model disagreement instead of predicted error.
C.2 Connection to Information Theory
At first sight, Jiang et al. (2022) seems disconnected from information theory. However, we can recover statements by using hi(p) p as a lincar approximation for Shannon's information content h(p):
hi(p) - p - logp h(p).
(84)
hi(p) is just the first-order Taylor expansion of h(p) logp around 1. Semantically, both Shannon's information content and this approximation quantify surprise. Both are 0 for certain events. For unlikely events, the former tends to too while the latter tends to 1.
17

---

Published in Transactions on Machine Learning Research (10/2022)
We can define an approzimate entropy H[X] using hn:
F[ = E[(pp(()) Elp(x)]= =1 Σ p(x)2
(85)
and an approzimate mutual information [[x; Y]:
i[x; [i[x] fi[x Y] [[[]] [ex]-Epoo Ep(r) H[X 18).
(86)
following the semantic notion of mutual information expected information gain in $2.
12 æ] as Covariance Trace. This approximate mutual information has a surprisingly nice property, which was detailed Smith & Gal (2018) originally:
Proposition C.1. The approzimate mutual information i[; a] is equal the sum of the variances of x, S2 over all y:
1
[p 2. [2] 2 0.
(87)
We present proof in $1) in the appendix. The sum of variances of the predictive probabilities (or trace of the respective covariance matrix) is a common proxy for epistemic uncertainty (Gal et al., 2017), and here the mutual information i æi] using h is just that. This gives evidence that these definitions are sensible and connects them to other prior Bayesian literature. Importantly, this also shows that H(y x] 2 H( x. S2).
Connection to Jiang et al. (2022). As random variable of X and Y, AD = Y X] is the Lest error:
H =Y X] = =1-ppp p(f = =
()) TestError.
(88)
Thus, the approximate cross-cntropy
f((p(Y () I = Y = (ic = - X))]
(89)
is the expected lest error ErTes.Error).
Similarly, when TOP is fulfilled, the mutual information 1[;;9/ [X] is the expected disagreement rate D
That is, when Y X, Ω is onc-hot, we have:
f x,9] 1 - Ex ExEypp( (X O.
(90)
and thus:
1|::0| A|L X] - ÅIL X
(91)
=ipp X
(92)
Ex Ip(P |X]
(93)
= E[Diss]
(94)
Lemma C.2. When the model p(g w) satisfies TOP, the GDE is equivalent to:
x x)) H(p(Y X) p Y
(95)
This relates the approximate cross-entropy loss (test error) to the approximate Bayesian model disagreement.
Without TOP. TOP does not hold, the actual expected disagrocment i Ω x]] lower-bounds the "expected disagreement rate" E[Dis), which then equals the expected predicted error Ex, [p(( |X) when have GDE. We have the following general equivalence to GDE:
18

---

Published in Transactions on Machine Learning Research (10/2022)
Lemma C.3. For model p(g [)), the GDE is equivalent to:
A X) ll p( Y xX) Mip x] [1
(96)
The other statements and proofs translate likewise, and intuitively seem sensible from an information-thcoreti perspective. We can go further and directly establish analogous properties using information theory in the next subsection.
C.3 Information-Thearr Version
Here, we derive information-theerrteee version of the GDE both under the assumption of TOP and without. Importantly, we will not require Bayesian model for any of the main statements they hold for any model P(0 z). We show that we can artificially introduce a connection to disagreement using TOP.
Enformation-Theorele GDE. We have already introduced the BALD equation eq. (81), which connects expected disagreement and predictive uncertainty:
T Ω x] H| [] HY 2 Ω]
The expected disagreement is measured by the mutual information 1[8; Ω2 X], and the prediction error is measured by the cross-cntropy of the predictive distribution under the true data generating distribution H(p(Y () p( = Y X)). ())).. Indeed, the test error is bounded by it (Kirsch et al., 2020):
P(Y + 8) S 1 e- ((0(X1X)))((+=XIX))
(97)
When our model fulfills TOP, we have H[Y X, 0, and thus 1[; X] H[ | The expected disagrcement then equals the predicted label uncertainty H[ X]. Generally, we can define an 'entropic GDE:: Definition C.4. model p(O x) satifies entropic GDE, when:
Y X)) = =H x]. H(P(Y X) P( = Y Lemma C.5. When Bayesion model p(ù. x:) satisfies TOP, entropic GDE is equivalent to H(p(Y XX) || p(Ý Y X)) 1[
(98)
(99)
The latter is close GDE, especially when comparing the previous section.
We formulate entropic class-aggregated calibration by connecting Híğ æx] with H[y xx), where we define Híp xx] := hp(0 x)) = log P0 æ) (Kirsch & Gal, 2021). That is, instead of using probabilitics, we use Shannon's 's information-comlent.
Definition C.6. The model p(O satisfies entropic class aggbessaggeeeeedd calibration when for any q 2
PHH = Y = -) - - g).
(100)
Similarly, we can define the entropic class calss--ggregated calibration ervor (ECACE):
ECACE: := /
p(H[} = X] = g)
(101)
(0 -P(HlL y x| 4) dg.
As - log p is strictly monotonic and thus invertible for non-negative p, entropic class-aggregated calibration and class-aggregated calibration are equivalent. ECACE and CACE are not, though.
The expectation of the transformed random variable H( = Y X] (in Y and X) is just the cross-cntropy:
Exyy H[Ý Y | Ep(ra) H[Ý X] H(p(Y (X) || p( Y 12)).
(102)
Using this notation, and analogous Theorem 3.6, show:
19

---

Published in Transactions on Machine Learning Research (10/2022)
Theorem C.7. When Hςğ [] log x) is uppper-bounded by L for all i and x, we have:
ECACE 2 - HI [ = - -
H[8 [
(103)
and when the model satisfies TOP, equivalently:
1 = L (1)00XX))PP(XXXI11 =Y 1
(104)
There might be better conditions than the upper-bound above but this bound is the spirit of Jiang et al. (2022). Indeed, the proof of Theorem 3.6 is the same, except that we use q L instead of q 1. Finally, when the model satisfies entropic class-aggregated calibration, ECACE 0, cross-entropy (or negative expected log likelihood) equals disagreement (respectively, predicted label uncertainty when TOP does not hold). Thus, we have:
Theorem C.8. When the model p(ù x)) satisfies entropic class-aggregated calibration, it trivially also satisfies entropic GDE:
H(p(Y X) || p( = Y 1 (x)) = H[ x 2 1["; $2 X],
(105)
and when TOP holds:
H(p(Y (X) =Y Y ()) = 1[; X].
(106)
Without TOP. Again, if we do not expect one-hot predictions for our ensemble members, the analogy put forward in Jiang et al. (2022) breaks down because the Bayesian disagreement [2 X] only lower bounds the predicted label uncertainty H[y X] and can not be connected ECACE the same way. But this also breaks down in the regular version in Jiang et al. (2022).
D Additional Proofs
Lemma D.1. For model p(O x), we have for all [K] and g E [0, 1]:
p(ř = k 1pfp & x) 9) = q.
(107)
when the left-hand side well-defined.
Proof. This is equivalent to
p(ś
= q) =9 p(p8 x| = =q),
(108)
as the conditional probability is either defined or p(p(Ý = k XX) = 9) = 0. Assume the former. Let
p(p(Ý = k X) = q) > O. Introducing the auxiliary random variable T = p(p =k| as a transformed random variable of X, we have
= pTT =q)-
(109)
We can write the probability as an expectation over an indicator function
p(Ý = Kr =
(110)
Ex,9/1(f = k,Tx(X) 4}l
(111)
= -Ex.llf88 = k} (X) = =q}
(112)
=Ex(1[Ti(XX) = 9) Ey s1[1 = k X
(113)
- =Ex[1(Zi(X) = p(ř = k x
(114)
20

---

Published in Transactions on Machine Learning Research (10/2022)
Importantly, if (z) g) = 1 for an I, we have (x) = p(y = k x) 9. and otherwise, we multiply with O. Thus, this is equivalent to
= -x([((T(X)) = 2}
(115)
= =XE = gHl
(116)
[X)(Tx(X)= = q).
(117)
Lemma 4.2. The model p(ğ x) satisfies class-wise calibration when for any g E [o, 1] and any class [K]:
p(Y = k. =
kk =) - x) x) q)
=
(36)
Similarly, the model p(ù x) satisfies class-aggregated calibration when for any q E [0, 1]:
- (x) = 4)
(37)
and class-wise colibration implies class-aggregate calibration. Proof. Beginning from we expand the conditional probability to and substitute eq. (35) into the outer y, obtaining the first equivalence
p(Y = k 1p( = x| = = =4
(118)
+ p(Y = k p(P = k X) = -) =k|x)= =)) 4),
= x\
(119)
+ p(V = k. p(p k| x) = g) = p(δ = k. k( q)-
(120)
For the sccond equivalence, we follow the same approach. Beginning from
k. k, = k x) = ②) = Σ p(p(f = = x = =).
(121)
we pull the outer into the sum and expand using (35)
C k k x) = 4) = Σ qplp(p = x) = 9) = = x = 4)- (122)
In the inner expression, k is tied to Y on the left-and side and & on the right-hand side, so we have
= = Y x) = =q) =) =
x) = =).
(123)
k
Summing over k, marginalizes out Y and Ý respectively, yiclding the sccond equivalence
p(p( = Y X) q) p(p(Y X) = q).
(124)
Finally, class- wise calibration implies class-aggregated calibration as summing over different is in (120), which is equivalent to class-wise calibration, yields (122), which equivalent to class-aggregated calibration. J
Proposition C.1. The approzimate mutual information i[; æ] is equal the sum of the variances of x, Ω over all g:
ip: Ω s]]
Vara (p(0 xr, 2)] 2 0.
(87)
21

---

Published in Transactions on Machine Learning Research (10/2022)
Proof. We show that both sides are equal:
f 2 x|| = H[ x]] H[ x
(125)
1y p(8 x)] Ey Ev - p(Y 22,0)
(126)
- 8 a, [n] - 1 |x)]
(127)
En (x [p(0 Ir, [x, Ep((at) p(0 (x)
(128)
K
En
Σ 5=1
p(0 2., 22)
Σ
En PP0 x. 2)}
(129)
8=1
K
En [p( 22 0]
En(p(0 [,
(130)
f=1
[arg|p(0 [x,
(131)
$=1
2 0,
(132)
where we have used that Ep(2)a) P(D [x) Σg P(0 x>2
E
Experimental Validation of Calibration Deterioration under Increasing Disagreement
Here, we discuss additional details to allow for reproduction and present results on additional datascts. In addition to the experiments on CIFAR-10 (Krizhevsky et al., 2009) and CINIC-10 (Darlow et al., 2018), we report results for ImageNet (Deng et al., 2009) (in-distribution) using an ensemble pretrained models and PACS (Li et al., 2017) (distribution shift) where we fine-tune ImageNet models on PACS 'photo' domain, which is close to ImageNet as source domain, and evaluate on PACS 'art painting', 'sketch', and 'cartoon' domains. We use all three domains together for distribution shift evaluation to have more samples for the rejection plots.
E.1 Experiment Setup
We use PyTorch (Paszke et al, 2019) for all experiments.
CIFAR-10 and CINIC-10. We follow the training setup from Mukhoti et al. (2021): we train 25 WideRessNet-28-11 models (Zagoruyko & Komodakis, 2016) for 350 epochs on CIFAR-10. We use SGD with a learning rate of 0.1 and momentum of 0.9. We use a learning rate schedule with a decay of 10 at 150 and 250 epochs.
ImageNet and PACS. We use pretrained models with various architectures (specifically: ResNet-1522D (He et al., 2018), BEIT-L/16 (Bao et al., 2021), ConvNext-L (Liu et al., 2022), DeiT3-L//16 (Touvron et al., 2020), and ViT-B/16 (Dosovitskiy et al., 2020)) from the timm package (Wightman, 2019) as base models. We freeze all weights except for the final linear layer, which we fine-tune on PACS' 'photo' domain using Adam (Kingma & Ba, 2014) with learning rate 5 x 10-3 and batch size 128 for 1000 steps. We then build an ensemble using these different modcls.
E.2 Additional Results
In Figure 2, we see that for ImageNet and PACS, the calibration metrics behave like for CIFAR-10 and CINIC-10, matching the described bohavior in the main text. We use models from cach of the enumcrated architectures to build an ensemble of 25 models. Individual architectures also behave expected we ablate in Figure
22

---

Published in Transactions on Machine Learning Research (10/2022)
Measure
CWCE
CACE
Test Error
ECE
Test Error Predicted Error
ImageNet (In-Distribution)
0.20
Value
0.10
0.05
0.00
0.8
0.6
Predicted Error (Disagreement Rate)
Predicted Error (Disagreement Rate)
PACS (Distribution Shift)
0.35
0.30
Value
0.20
0.15
0.10
0.05
0.00
0.1
0.2
0.3
0.4
0.5
0.7
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Predicted Error (Disagreement Rate)
Predicted Error (Disagreement Rate)
(a) Ensemble with TOP
(b) Ensemble without TOP (same underlying models)
Figure 2: Rejection Plot of Calibration Metrics for Increasing Disagreement In-Distribution (ImageNet) and Under Distribution Shift (PACS "pholo" domain olher domains). Different calibration metrics (ECE, CWCE, CACE) vary across ImageNet and PACS 'art painting', 'cartoon', and 'sketch' domains across ensemble of 5 models trained on ImageNet and 25 models fine-tuned on PACS' 'photo' domain, depending on the rejection threshold of the predicted error (disagreemmntt rate). Again, calibration cannot be assumed constant for in-distribution data or under distribution shift. The mean predicted error (disagreement rate) is shown on the x-axis. (a) shows results for an ensemble using TOP (following Jiang et al. (2022)), and (b) for regular deep ensemble without TOP. Details in $E.2.
23

---

Published in Transactions on Machine Learning Research (10/2022)
Additionally, in Figure 3 and Figure 4, we also show rejection plots using the Expected Information Gain/BAle for thresholding. We observe similar trajectories. Comparing these results with Figure 1 and Figure 2, we see that both the predicted error and the Bayesian metric behave similarly. We hypothesize that this could be because the datasets only contain few samples with high alcatoric uncertainty (c.g. noise), which would of herwise act as confounder (Mukhoti al., 2021). See also the discussion $6.
24

---

Published in Transactions on Machine Learning Research (10/2022)
Measure
CWCE
CACE
Test Error
ECE
Test Error Predicted Error
CIFAR-10 (In-distribution)
0.04
0.03
Value
0.02
0.01
0.00
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Expected Information Gain/BALD (Bayesian Disagreement)
Expected Information Gain/BALD (Bayeslan Disagreement)
CINIC-10 (Distribution Shift)
0.25
0.20
Value
0.15
0.10
0.05
0.0 0.2
0.4
0.6
0.8
1.0
0.0
0.4
0.6
Expected Information Gain/BALD (Bayesian Disagreement)
Expected Information Gain/BALD (Bayesian Disagreement)
(a) Ensemble with TOP
(b) Ensemble without TOP (same underlying models)
Figure 3: Rejection Plot of Calibration Metrics for Increasing Bayesian Disagreement In-Distribution (CIFAR- 10) and Under Distribution Shift (CINIC-10). Different calibration metrics (ECE, CWCE, CACE) vary across CIFAR-10 and CINIC-10, depending on the rejection threshold of Bayesian disagreement (Expected Information Gain/BALD) /BAL)). The trajectory matches the one for prediction disagreement. We hypothesize this is because there few noisy samples in the dataset which would act confounder for prediction disagreement otherwise. Details in $E.2.
25

---

Published in Transactions on Machine Learning Research (10/2022)
Measure
CWCE
CACE
Test Error
ECE
Test Error Predicted Error
ImageNet (In-distribution)
0.225
0.200
0.175
Value
0.150
0.125
0.100
0.075
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
Expected Information Gain/BALD (Bayesian Disagreement)
Expected Information Gain/BALD (Bayeslan Disagreement)
PACS (Distribution Shift)
0.25
0.20
Value
0.15
0.10
0.05
0.00
1.2
0.0
0.8
Expected Information Gain/BALD (Bayesian Disagreement)
Expected Information Gain/BALD (Bayesian Disagreement)
(a) Ensemble with TOP
(b) Ensemble without TOP (same underlying models)
Figure 4: Rejection Plot of Calibration Metrics for Increasing Bayesian Disagreement In-Distribution (CIFAR- 10) and Under Distribution Shift (CINIC-10). Different calibration metrics (ECE, CWCE, CACE) vary across CIFAR-10 and CINIC-10, depending on the rejection threshold of Bayesian disagreement (Expected Information
The trajectory matches the one for prediction disagreement. We hypothesize this because there are few noisy samples in the dataset which would act as a coufounder for prediction disagreement otherwise. Details in $E.2.
26

---

Published in Transactions on Machine Learning Research (10/2022)
Measure
CWCE
CACE
Test Error
ECE
Test Error Predicted Error
PACS, BeiT-L/16
0.3
Value
0.2
0.1
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
Predicted Error (Disagreement Rate)
Predicted Error (Disagreement Rate)
PACS, ResNet-152-D
1.0
Value
0.2
0.0
0.1
0.2
0.4
0.5
0.6
0.7
0.8
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Predicted Error (Disagreement Rate)
Predicted Error (Disagreement Rate)
(a) Ensemble with TOP
(b) Ensemble without TOP (same underlying models)
Figure 5: Rejection Plot of Colibration Metrics for Increasing Disagreement Under Distribution Shift (PACS "photo" domain other domains) for Specific Model Archilectures. We use the same encoder weights and evaluate ensemble of 5 models, which were last-layer fine-tuned PACS, We show 152--D and BeiT-L/16. Details in 2..2.
27


Reviews:
[{"review_id": "PJqrsngNH3", "reviewer": "Reviewer_dFis", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- The paper is well written and nice to read. It states its conclusions clearly (see Sec. 6) and provides evidence for them.\n- The Bayesian formulation of Jing et al.'s results is more concise and clearer than the hypothesis space view.\n- The experiments clearly support the paper's main argument.\n- The proofs are straightforward and self-contained in the main text.\n\n\n**Weaknesses**\n- I found some of the notation unnecessarily confusing. This is mainly due to the use of round and square parentheses for the p operator. I feel that either only p[.]  should be used, or the way p(.) is used should have a single consistent notation. The notation p(.) is used to denote a p.m.f. or p.d.f. The expression like Eqs. (2) and (3) use lowercase versions of the random variables to indicate which distribution is being referred to. Whereas Eqs. (4) and (5) use an equals sign to indicate the which r.v.s the density comes from. This can be unclear for expressions like p(\\hat{Y} = Y | X), which evaluates the conditional p.m.f. of \\hat{Y} conditional on X at the random variable Y. The issue is further confused by expressions like p(\\hat{Y} |X) in Eq. (23). It seems like this is supposed to be the conditional p.m.f. of \\hat{Y} conditional on X evaluated at the r.v. \\hat{Y}.\n- There could be experiments on more datasets and model types. In general, I believe the main empirical claims of the paper, but the experiments are somewhat limited.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a response to Jing et al. (2022). First, it rederives the results from Jing et al. using a Bayesian instead of a hypothesis space approach. This simplifies many of the proofs. Second, it questions the key conclusions of Jing et al. The authors demonstrate empirically that a model's calibration can deteriorate as prediction disagreement increases---especially under distribution shift. This means that to apply the bounds, calibration must be measured on the validation data, which requires labels. \n\nOverall I like the paper and feel it makes a good contribution and should be accepted. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "*Major*\n- I would recommend using \\mathbb{P} for the operator p[E] that is equivalent to \\mathbb{E}[1_E].\n- For p(.) that is used to denote a p.m.f. or p.d.f., I think it should be clearer what distribution is being used by settling on a single notation, i.e. use a subscript to denote the distribution: Eq. (3) would be p_{\\hat{Y}|X=x}(\\hat{y}) = \\mathbb{E}_\\Omega[ p_{\\hat{Y}|X=x,\\Omega}(\\hat{y}) ] and p(\\hat{Y}=Y|X) would become p_{\\hat{Y}|X)(Y). This would also resolve the issue with p(\\hat{Y} |X) as p_{\\hat{Y} |X}(\\hat{Y})\n- I'm happy to be outvoted on this if the other reviewers disagree.\n\n*Minor*\n-The first sentence of the introduction is incorrect. I understand it's just preamble, but there is significant research on the reliability of machine learning models and the field does not \"trust them blindly.\"", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mvKrJGqfoQ", "reviewer": "Reviewer_RrY3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The main strength of the paper is that it makes several contributions that shed light on understanding generalization via disagreement, as summarized above. The exposition of the paper is nice, the equivalent definitions to (Jiang et al 2022) are more intuitive, and the proofs are straightforward and understandable.The flow of the paper is generally clear, while I have a suggestion for improving it. \n\nThe main weakness of the paper is that the paper is written as if this is a response to (Jiang et al. 2022). My suggestion would be for the authors to make this a standalone paper, i.e., put down the definitions and assumptions, and do all the derivations and proofs; then in a separate section connect everything back to (Jiang et al. 2022). In particular, this would increase the archival value of the paper, and improve its readability.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This short paper builds closely on and around the findings of Jiang et al. (2022), and makes several contributions: (1) The authors show that the results of Jiang et al. (2022) could be re-derived in a more straightforward manner through a Bayesian view. (2) The authors argue that a lot of the heavy lifting in the results are indeed due to the calibration assumption, which itself is not verifiable without labels leading to a circular argument; (3) The authors make a case that one should be cautious about interpreting the results when dealing with arbitrary distribution shifts, as the calibration properties may not generalize when distributions shift (and provide numerical evidence for it). Overall, I enjoyed reading the paper and liked the exposition of it. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Please reformat the paper for the technical derivations/exposition to become standalone, and then make connections with (Jiang et al 2022) once all technical results are derived/presented.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nd9ZstyErt", "reviewer": "Reviewer_Tz6j", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is clearly written and easy to follow. I particularly liked how this perspective allowed for more simplifications of (Jiang et al. 2022).\n\nWeaknesses:\n- Perhaps this isn’t the point of the submission (since it seems to be more of an explanatory paper), but do the empirical findings on CIFAR-10 hold for some of the other datasets or architectures explored in (Jiang et al. 2022)? Say, SVHN or PACS. I don’t think it’s necessary to reproduce all combinations of them, but it’d be more compelling to have more than one (dataset, architecture) example in the submission to drive the point home.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides an alternative perspective to (Jiang et al. 2022), revisiting their theoretical and empirical results through a probabilistic lens. In particular, they simplify their results (which claim that looking at model disagreements can provide an upper bound on their test errors) by examining various definitions of (Bayesian) model calibration, propose new connections to prior works that emerge as a consequence, and demonstrate through experiments on CIFAR-10/CINIC-10 that the empirical findings in (Jiang et al. 2022) should be interpreted with care.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Major:\n- At least one more verification of a (dataset, architecture) combination to ensure that the empirical findings in the submission generalize to different settings.\n\nMinor:\n- In the introduction, there are several notions of uncertainty that would be helpful to be unpacked a bit further. For example, on page 2, what does “overall uncertainty” refer to in the context of epistemic/aleatoric uncertainty?\nNitpick: Notation section is a bit verbose.\n\nTypos:\n- “And thus includes a distribution [shift]” on page 10\n", "broader_impact_concerns": "The authors did not include a broader impacts statement, though there do not seem to be any immediate ethical concerns or implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FgNE2RGbuV", "reviewer": "Reviewer_AWck", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I like the work a lot. I think it does a great job at presenting its argument clearly and offering a counter point to the original work. \nStrengths: \n * very interesting results, offering significant context to the original work \n * well written, easy to follow\n * to the extent I followed the proofs (and I can't say I properly internalized them) I do not see any error with them\n\nWeaknesses: \n  * while the authors did a great job with the write-up, I think some parts of the proof can still be a bit hard to follow. I don't know if this really qualifies as a weakness, but personally I found that I can't just read the paper, but I have to go back and forth over the derivations to make sure I understand them correctly. I think this was partially because I was not familiar with the original work, and some of the quantities used in the work (e.g. predictive accuracy or error, which initially I thought I understood, but as going through the proofs I realized I had to go back to the definition and digest what they mean). That said, for a theoretical work, I do not know if this can be avoidable. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The work is a direct follow-up on Jiang et al. work, adding two important contributions. One is providing a Bayesian formulation of the original work, which allows to simplify the proofs. The other contribution is to argue that disagreement becomes unreliable as the value increases (or more exactly calibration becomes worse as disagreement increases) and calibration requires labels, making the use a disagreement instead of test error difficult.  ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Overall I recommend acceptance. I personally did not see any technical issues with the proofs. To a certain extent most of the theoretical work is just presenting the results of the original work from a Bayesian perspective, so the results themselves are not proposing something new, so in some sense we expect these theorems and lemmas to hold.\nThat said I cannot say I fully internalized the proofs, but I'm fairly confident they are ok. \n\nRegarding the deterioration of the calibration, this result is mostly empirical. I think this observation is very useful, and I'm happy the authors have emphasized it in their work. \nI think there is one change I would condition the acceptance on, which is for the authors to provide in the final version a bit more details on the experimental setup. I think the description provided is not sufficient for reproducing their results, and I can easily see an appendix where they can just enumerate all the details of their setup. While I think this requires little work, and the results agree with my intuitions so I have no doubt to distrust the empirical evidence, I think it is highly important to be fully transparent when it comes to empirical results.\n\nAdditionally it would be great if the authors can provide any kind of intuition of why this deterioration happens (from a more theoretical stand point). But of course I agree this might be a big ask, and formally it might be much harder to make this argument. I just felt at the end, even if informal, providing more intuition of what is happening under the hood would be great. ", "broader_impact_concerns": "I do not see any ethical concerns with this work. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors built on a previous work from Jiang et al. (2022), adopting a simplifying viewpoint which makes the previous results clearer while demonstrating the empirical limit (e.g., under distribution shift) of this previous work statement. The authors have followed the reviewers feedback and improved notations to make the paper more readable and added experiments to further support their claim. 
Decision: Accept (poster)