A statistical theory of cold posteriors in deep neural networks
Authors: Laurence Aitchison
Venue: ICLR 2021 Poster (2021)
Keywords: Bayesian inference
cold posteriors
sgld

Abstract: To get Bayesian neural networks to perform comparably to standard neural networks it is usually necessary to artificially reduce uncertainty using a tempered or cold posterior. This is extremely concerning: if the prior is accurate, Bayes inference/decision theory is optimal, and any artificial changes to the posterior should harm performance. While this suggests that the prior may be at fault, here we argue that in fact, BNNs for image classification use the wrong likelihood. In particular, standard image benchmark datasets such as CIFAR-10 are carefully curated. We develop a generative model describing curation which gives a principled Bayesian account of cold posteriors, because the likelihood under this new generative model closely matches the tempered likelihoods used in past work.

Published as a conference paper at ICLR 2021
A STATISTICAL THEORY OF COLD POSTERIORS IN DEEP NEURAL NETWORKS
Laurence Aitchison Department of Computer Science, University of Bristol, Bristol, UK, F94W 9Q laurence wet.iiiicccccccc
ABSTRACT
To get Bayesian neural networks to perform comparably to standard neural net- works it is usually necessary to artificially reduce uncertainty using "tempered" or "cold" posterior. This is extremely concerning if the generative model is accurate, Bayesian inference/decision theory is optimal, and any artificial changes posterior should harm performance. While this suggests that the prior may be at fault, here argue that in fact, BNNs for image classification use the wrong likelihood. In particular, standard image benchmark datasets such as CIFAR- 10 are carefully curated. We develop a generative model describing curation which gives a principled Bayesian account of cold posteriors, because the likelihood under this new generative model closely matches the tempered likelihoods used in past work.
the
INTRODUCTION
Recent work has highlighted that Bayesian neural networks (BNNs) typically have better predictive performance when we "sharpen" the posterior (Wenzel et al.) 2020). In stochastic gradient Langevin dynamics (SGLD) (Welling Teh, 201T), this can be achieved by multiplying the log-posterior by is smaller than 1 (Wenzel et al. 2020). Broadly the same effect can
1/T, where the "temperature" be achieved in variational inference by "tempering" i.e. downweighting the KL term. As noted in Wenzel et al. (2020), this approach has been used in many recent papers to obtain good performance, albeit without always emphasising the importance of this factor (Zhang et al. 2017 Bae et al.. 2018
Osawa et al. 2019 Ashukha et al. 2020).
These results are puzzling if we take the usual Bayesian viewpoint, which says that the Bayesian posterior, used with the right prior, and in combination with Bayes decision theory should give optimal performance (Jaynes] (003). Thus, these results may suggest we are using the wrong prior. While new priors have been suggested (e.g. Ober & Aitchison 2020), they give only minor improvements in performance - certainly nothing like enough to close the gap to carefully trained non- -on-Bayesian networks. In contrast, tempered posteriors directly give performance comparable carefully trained finite network.
The failure develop effective prior suggests that should consider alternative explanations for the effectiveness of tempering. Here, we consider the possibility that it is predominantly (but not entirely) the likelihood, and not the prior that is fault. In particular, benchmark datasets such as ImageNet and CIFAR- 10 are carefully curated, and that it is important consider this curation as part of our generative model. We develop a simplified generative model describing dataset curation which assumes that datapoint is included in the dataset only if there is unanimous agreement on the class amongst multiple labellers. This model naturally multiplies the effect of each datapoint, and hence gives posteriors that closely match tempered or cold posteriors. We show that toy data drawn from our generative model of curation can give rise to optimal temperatures being smaller than 1. Our model predicts that cold posteriors will not be helpful when the original underlying labels from all labellers are available. While these are not available for standard datasets such CIFAR- 10, found a good proxy: the CIFAR-105 dataset (Peterson et al. 2019). in which ~ 50 humans annotators labelled the CIFAR-10 test-set (we use these as our training set, and use the standard CIFAR-10 10 training set for test-data). As expected, we find strong cold-posterioe effects
note that standard image

---

Published as a conference paper at ICLR 2021
when using the original single-label, which are almost entirely eliminated when using the 50 labels from CIFAR-1OH, In addition, curation implies that each label is almost certain to be correct, which one way to understand the statistical patterns exploited by cold posteriors. As such, if we destroy this pattern by adding noise to the labels, the cold posterior effect should disappear. We confirmed that with increasing label noise, the cold posterior effect disappears and eventually reverses (giving better performance at temperatures close to 1).
2 BACKGROUND: COLD AND TEMPERED POSTERIORS
Tempered (e.g. Zhang et al., 2017 and cold (Wenzel et al. 2020 posteriors differ slightly in how they apply the temperature parameter. For cold posteriors, scale the whole posterior, whereas
tempering is a method typically applied in variational inference, and corresponds to scaling the likelihood but not the prior,
log P Poold (0 V) = + log (X, 010) + + log P (0) + const
(1)
log P rempered (0 Y) = log P (X, (0) + log (0) + const.
(2)
While cold posteriors are typically used in SGLD, tempered posteriors are usually targeted by variational methods. In particular, variational methods apply temperature scaling to the KL-divergence between the approximate posterior, Q (0) and prior,
L = EQ( [log P (X, [ A (Q (#) (#))
(3)
Note that the only difference between cold and tempered posteriors is whether we scale the prior, and if we have Gaussian priors over the parameters (the usual case in Bayesian neural networks), this scaling can be absorbed into the prior variance,
T log Poold (0)
Σ of + const =
Σ 0 + const log P cold (0).
(4)
2T colL
2 kayare
in which case, aadld of immpreed/ T, so the tempered posteriors we discuss are equivalent to cold posteriors with rescaled prior variances.
METHODS: A GENERATIVE MODEL FOR CURATED DATASETS
Standard image datasets such as CIFAR-10 10 and ImageNet are carefully curated to include only unambiguous examples of each class. For instance, in CIFAR-10 10. student labellers were paid per hour (rather than per image), were instructed that "It's worse to include one that shouldn't be included than exclude one", and then Krizhevsky (2009) "personally verified every label submitted by the labellers". For ImageNet, Deng et al. (2009) required the consensus of a number of Amazon Mechanical Turk labellers before including an image the dataset.
To understand the statistical patterns that might emerge in these curated datasets, we consider a highly simplified generative model of formatins-oormaiion. In particular, draw random image X from some underlying distribution over images. P (X), and ask S humans to assign a label, {Y.}.. [= (c.g. using Mechanical Turk). We force every labeller label every image and if the image ambiguous they are instructed to give a random label. If all the labellers agree, Y1 YY= reached and we include the datapoint in the dataset. If any of the labellers disagree consensus not reached, and we exclude the datapoint (Fig. D Formally, the observed random variable, Y, is taken to be the usual label if consensus was reached and None if consensus was not reached (Fig. B
=Ys. consensus is
Yi Y YIYYJ = None
ffy.YY=....Y. =Ys
(5)
otherwise
Taking the human labels, Y., to come from the set y, so Y, E y, the consensus label, Y, could be any of the underlying labels in V or None if no consensus is reached, so Y E y U (None]. When consensus was reached, the likelihood is,
P PY = ((Y, = (X,)
- PYY==/XX00) = (Y. (Y,= (6)
2

---

Published as a conference paper at ICLR 2021
Y1 = 2 = Y/3 = cat
Vi = 1/2 = Y33 = dog
Yi Y/2 = cat;
Y/3 = dog
= cat
Y = dog
Y None
Figure simple example of our generative process describing dataset curation, with Labellers have the task of classifying images cats dogs. The first image is unambiguously cat, so the three labellers agree, consensus is reached, and the image included the dataset. The second image is unambiguously a dog, the labellers agree, consensus is reached, and the image is included the dataset. However, the third image ambiguous: the labellers disagree, consensus is not reached, and the image may be excluded from the dataset.
where we have assumed labellers are IID. This would appear to directly give an account of tempering, we have taken the single-labeller likelihood to the power S, which is equivalent setting À 1/S. However, to see how the full generative model functions we need to go on to consider the case in which consensus not reached,
P PY=None|XX 0)
P((==)|X,)) =1 PY.
yEY
wEY
To understand the impact of our model of consensus formation, note that the probability of particular class-label can separated into two terms, probability of consensus, and the probability of particular class given that consensus was reached.
P(-=y = (Y
X, (
(8)
Remarkably, knowing there was consensus gives us information about the weights, even in the absence of the class label,
(Y (X,0)
pX) DE
in essence, telling us that the output probability was close to for one of the class labels, without telling us which one. Schematically (Fig. 3). see that the datapoints with consensus class-label, "cat"" or "dog", lie far from the decision boundary where the class is unambiguous, and consensus easily reached. In contrast, in regions close to the decision boundary the inputs ambiguous, which tends to produce disagreement in the labellers, Icading to noconsensus. Thus, the existence of one more consensus points region implies that decision boundaries do not go through that region, giving us information about the decision boundary location, even if the label is not known. Concurrent work has shown that this likelihood can used explain classical semi-supervised likelihoods (Aitchison) 2020), this term really does give information about the neural network parameters. Finally, the label probability, conditioned on consensus.
P(Y=y(XX0)
PY,
P PYY=yly X,,,,, PYY
(10)
Egey P (Y,=y(X,0)
simply represents "reparameterised" softmax.
In datasets where the noconsensus inputs are known it is clear we should use the full likelihood. (Eq, 6 and 77. The question is: in real-world datasets, where we only know the consensus inputs and the noconsensus inputs have been thrown away (Fig. 2D, can we use Eq. (10), a reparameteriisatoon of the softmax- cateoooiiaa probabilities, for the known consensus points? The answer no because in Bayesian inference, we do not get to just pick a sensible- lloking conditional probability distribution, such Eq. 10 as the likelihood. Instead, need write down the full generative model,
3

---

Published as a conference paper at ICLR 2021
A
X
B X
C
X
Z
Y
(Y,)s
[Y,]s=1
Y
0
0
Figure 2: A The standard generative model for supervised tasks, assuming no data curation. B Generative model of data curation, where the consensus and noconsensus images are known. C Generative model of data curation, where the noconsensus images are unknown. The underlying images, X are no longer observed. Instead, we observe Z, where X if consensus is reached (Y = Y22 = Ys) and Z None otherwise.
and marginalise over unknown latents. To write down the generative model in the case of unknown noconsensus images (Fig.2) (C), we need to take X to be an unobserved latent variable, and take Z to be an observed random variable,
[X
ZJX, (Y,)
(11)
None otherwise
which the underlying image, X, consensus is reached, and None otherwise. As depends on A (Fig. CC), cannot take the usual shorteut of using P (Y)Z, 0); we must instead use the full likelihood, (Y, (0),
/ (dXP
P PY,
E
210)
(Y-15
/ (x P PYYXX)=))))))))))))))>)))))))))) (z|x, (r.)sss)
Σ (1:)2
(12)
for y not None,
P -Y = =(0
[1 ifY, = y for all s fl.....S))
(13)
otherwise
and,
P (ZIX (8(2-X) if 1/2 ...
(14)
Iz=None otherwise
where 8 is the Dirac-delta function, and where the indicator function, Tzz is 1 if Z = None and
otherwise. Substituting Eq. [3) and Eq, (14) into Eq. 12p
S
P(Y = y. 2 (X) P = x/X,0) PY y/x,))s
(15)
where the proportionality arises because P (X) does not depend the parameters of interest, A. Note that this is proportional to Eq. (6) above, and not Eq. (IO), so this does not just represent reparameterisation of the softmax.
Finally, this explanation for the cold posterior effect would suggest that we would see cold posteriors in maximum a- a-posteriori (MAP) inference, it is "common knowledge" (but we do not know of a good reference) that while weight-decay is closely related to MAP inference with Gaussian priors, the best performing value of the weight decay coefficient tend to be lower than those suggested by untempered MAP inference.
confirm in Appendix A This is expected as
4 DIFFERENCES BETWEEN COLD POSTERIOR AND DATASET CURATION SETUPS
Our model data curation provides a direct explanation for the effectiveness of cold posteriors, as Eq. (6) takes the underlying likelihoods, P (Y.|X,0) to the power S, which has exactly the same
4

---

Published as a conference paper at ICLR 2021
Hseal dectsion boundery relectd,
Figure 3: Illustrative example of artificial clustering induced in the dataset by rejection of ambiguous no-consensus points. The input points for "cat" and "dog" classes are generated from separate 2D Gaussian distributions, and the classifier (and decision boundary) comes from the ratio of the Gaussian probability density functions. For the consensus processes, we used S (we used relatively large value to make the effects unambiguous).
effect as 1/A in tempered posteriors (Eq. 2.
However, it is important note two differences between the cold posterior setup and the ideal setup for our generative model. First, our generative model assumes that the noconsensus points are known, or they are unknown, we can compute the integral,
P PY==None, Z=None|0) = = d (X)
1- E.(y,_-ux,0)
(16)
DEY
which is obtained by evaluating Eq. 122 in the case where Y=none, and substituting Eq. 77. Of course, in reality it is not possible to compute this integral because we do not know the underlying (X) (and usually, we do not even have samples from this distribution). In contrast, the standard cold-posterior setup entirely ignores these terms. Second, in the usual setting, the test data is also subject to the same consensus- formation process, in which case, we should use Eq. (TO) for prediction. use the single-labeller distribution, P (Y,|X, 0).
In contrast, in the standard oold-posterior setting, (Note that if the test-set was drawn "from the wild", without dataset curation, and labelled by a single labeller, then we should use P (Y, X, ff) for prediction). Overall then, while our model of data curation offers a potential explanation of the benefits of tempering, the differences in setup imply that we cannot expect X 1/S. to hold exactly, where X is the optimal temperature, and this is confirmed in our results on toy data below.
5 RESULTS
5.1 DATA SAMPLED FROM A KNOWN GAUSSIAN PROCESS MODEL
In the introduction, we took it as given that if the prior is correct, the optimal approach is to use the true Bayesian posterior, avoiding either tempering or cold posteriors. To check that this is indeed the case, tested the performance, measured as data generated from a known Gaussian process model. We uniformly generated 50 input points on the 1D interval [-10, 10], and used Gaussian process with squared exponential kernel with standard deviation of 4, and kernel bandwidth of 1. For inference, we used reparameterised VI, with an approximate posterior given by multiplying the prior by single Gaussian factor for each datapoint
for tempered posteriors using
Ober & Aitchison 2020
In particular, we used Gaussian process prior for the function values, u, at the training inputs (Williams & Rasmussen 2006).
P (f|x) = AV (x)))
(17)
and we use an approximate posterior (Hensman et al. 2015 Matthews et al. 2016 Ober & Aitchison 2020) defined by multiplying the prior by Gaussian with diagonal covariance, where we treat v and
5

---

Published as a conference paper at ICLR 2021
A
B
C
2
0.25
0.50
1/X
5
1est
0.50
0.75
0.75
0
10
10
10
10P
10
10"
10
10
0
true S
F
S
0.0
test LL
0.1
-02
10
10P
10 100
10
10
10
109
10
10
Figure 4: Tempering with data generated from toy Gaussian process model. A Performance of tempered posteriors on GP regression. The green dashed line highlights À 1. The black line represents the mean of 20 with our model of consensus formation. We train and test using our exact log-likelihood, and we include knowledge of the no-consensus input datapoints in the datasets. D Data generated from our model of consensus formation, with S = 4. Training uses P (Y,) XX, 0) log-likelihood (excluding noconsensus points), but testing uses the exact lest-lo--lkkelihen line lies at À 1/S 1/4. E Data generated from our model of consensus formation, with S = 4. Training and testing follow the standard cold-posterior setup, excluding noconsensus points and using P (Y.IX, 0). The x-axis gives the number of labellers in the underlying generative process, and the y-axis gives the reciprocal of the optimal temperature. Plots underlying E. The red dashed lines indicate 1/S.
(translucent blue lines). B GP classification. C GP classification
(Eq. and 77. The red dashed
A as variational parameters.
Q (f) x P (f(x)
where Σ= 3 (x) A)- (18)
Note that this captures the true posterior in the case of regression (where v is set to and A is output noise covariance: Ober Aitchison) 2020). then optimize the tempered ELBO (e.g.
Zhang et al. 2017),
C= EQ(F) [log (y(f) A (log (f[x) - log Q (f))).
(19)
And used the standard GP approach for prediction test points (Williams & Rasmussen 2006).
Initially, we tried using standard regression and classification generative models, without including model of consensus formation. Unsurprisingly. À corresponding the Bayesian posterior, optimal for GP regression (Fig. (FA) and classification (Fig. B). For GP regression, we use a Gaussian likelihood with standard deviation I, and for classification, we use a standard sigmoid probability with Bernoulli likelihood.
Next, confirmed that even under our new, more complex generative model of dataset curation = was still optimal (Fig. [CC, if we trained and tested using the correct form for the log- likelihood. particular, we considered no-consensus inputs known and used Eq. to incorporate their likelihood. However, in standard benchmarks, only the consensus inputs are known. As such, next we generated curated data, trained using (Y||X, 0) and ignoring noconsensus points, but tested on the correct likelihood, including noconsensus points (Fig. 4D); the optimal temperature remains around 1. Finally, we considered the standard cold/tempered posterior setup, where we use P (Y,|X, 0), excluding noconsensus points, for training and testing. The optimal À indeed fell with S (Fig. 4EF), giving potential explanation for the cold posterior effect. As expected, we have S 1/X*, but the relation does not appear to hold exactly due to the mismatches discussed in Sec. 4 Finally, we show in Appendix B that the same patterns emerge in a Bayesian neural network, using Langevin sampling for inference.

---

Published as a conference paper at ICLR 2021
A
A test log ggg-llallhooo 0
100
lest accuracy (2))
test log-lkellhood
-0.2-
0.2
accuracy
80
3-
uncurated
-0.4-
0.1
80
curated
-0.6
0.0
10
10"
10
10
10
10"
10-2
10
Figure 5: A Test log- ledelike with tempering. B Change in test log-likelihoods from the baseline A 1. We see standard cold posterior effects at low noise levels, which reverse at higher noies levels. C, D As A, B but for test accuracy.
5.2 CURATED AND UNCURATED GALAXY ZOO DATASETS
The most direct test of our theory is to evaluate the cold-posterior effect real-world curated and uncurated datasets. To this end, we used the Galazy Zoo dataset (Willett et al. 2013), as the original dataset is not "curated"" in our sense (the criteria for inclusion were e.g. brightness) and the dataset gives classifications from ~ 50 labellers. We used a reduced label set for simplicity (see Appendix ( For the uncurated dataset, we selected 12500 points at random from the full dataset, and for the curated dataset, we selected the 12500 most confident points (defined in terms of labeller agreement), such that the correct class-balance was maintained. These datasets training points and 10000 test points. Note that using our generative model directly would lead to drastically different class-balance in the curated and uncurated datasets, due to different levels of certainty for different classes. To perform probabilistic inference, we used SGLD with code adapted from (Zhang al. 2019). particular, we used their standard settings of a ResNet18, momentum of 0.9, and cyclic cosine learning rate schedule. Due to the smaller size of our training set, we used longer cycles (600 epochs rather than 50), and more cycles (8 rather than 4).
split at random into 2500
As expected, we found that performance curated data was far better than that uncurated data, both terms of test log-likelihood and test accuracy (Fig-5AC) (AC). We found some cold-posterior effects in uncurated data, which is not surprising because there may be other causes of cold- posterotsss such as modell-mismatee effects for curated data than for uncurated data (Fig. [BD).. Moreover, these plots tend to understate the differences between curated and uncurated data. In particular, the proportional changes to the test-log- eest-log-likeihoodd likelihood (which has an upper bound of zero) and the test error is for curated data is very large, with both changing by a factor of ~ 3 as temperature falls. In contrast, proportional changes for dest-log-likelihood and test-error are much smaller (only 20%).
or biases in SGLD. Critically though, we found much stronger cold ood-pooteriorr
5.3 CIFAR-10H
One prediction made by framework is that if we had access the original underlying human labels on the full dataset, including images that were rejected because consensus was not reached, then tempering should not be necessary. Obviously, all this additional information is not available for standard datasets such as CIFAR-10. However, we are able to get close by considering the CIFAR-IOH dataset (Peterson et al. 2019). The authors of this work asked around 50 human labellers to label the CIFAR- test-set. As we might expect given the careful curation that went in to creating the original CIFAR- 10 dataset (Krizhessyy 2009), for almost half the datapoints, all 50 labellers agreed, and more than three-quarters of images had 2 or fewer disagreements (corresponding to 4%) of labellers (Fig. AA) While not possible to estimate without having information about the unknown image distribution, the high level of agreement would indicate that the effective value of S is large- - potentially even larger than 10.
Next, we trained neural network the 50 labels provided by the CIFAR-IOH dataset. As these labels are provided for the test-set with 10,000 images, this required us to swap the identities of the test and training sets (so that our training set consisted of 10,000 points, each with around 50 labels, and the test set consists of 50,000 points, with the single label from the original CIFAR- 10 dataset). We compared against training with the standard CIFAR-10 10 test set, containing the same images but
7

---

Published as a conference paper at ICLR 2021
4000
90
Test log Bkellkeeeddd
lest accuracy (%)
-0.4
frequency
-0.5
2000
-0.6
CIFAR-10
80
00.7
CIFAR-10H
15
10
10
10
10
100
number of errors (out of ~ 50 labels)
A
Figure A The number errors (defined as labellers who disagree with the most popular category) out of around 50 labels. B The test log-likelihood for different values of À for training with the standard single-labels provided by CIFAR- 10 and the 50 labels provided by CIFAR-IOH Note that here we have swapped the identities of the train and test set. C As in B, but for accuracy.
only with a single label. To perform probabilistic inference, we used SGLD with code adapted from (Zhang al., 2019). In particular, we used their standard settings of ResNet18, momentum of 0.9, and cyclic cosine learning rate schedule. Due to the smaller size of our training set, we used longer cycles (150 epochs rather than 50), and more cycles rather than 4). Importantly, we kept all the parameters of the learning algorithm the same for CIFAR-101 10H and our CIFAR-10 comparison. The only complication was that to keep the same effective learning rate for CIFAR-10H we need to take into account the number of labellers per datapoint (if we have 50 labels for a datapoint, the loss and the gradients of the loss are 50 times larger, resulting in step-sizes that are also 50 times larger). As such, keep the same step-sizes, we divided the actual learning rate by 50. An alternative way to look at this is that we use the average log- likelihood per labeller (rather than per per-image). Importantly, this change in learning rate, leaves the stationary distribution unchanged, as changing the learning rate is nol equivalent to tempering. All other parameters were left at the values specified by Zhang al. 2019).
As expected, when training on the single-label from the original CIFAR-10 testset (blue lines Fig. (BC). there are very large tempering effects. In contrast, when training on the 50 labels provided CIFAR-10H (orange), the effects of tempering are far smaller. In a sense, this is not surprising - using 50 labellers in effect makes the likelihood 50 times stronger, which very similar to applying tempering. But this simplicity is the point: in the Bayesian setting we need to condition on all data in our case all the labels, and once we do that, the cold posterior effect disappears.
Note that while ceapering-effectte are dramatically reduced, they have not been eliminated entirely. This is expected in our setting, both because of the mismatch between this setup and the exact setup (Sec. E. but also because the inference method, SGLD, becomes more accurate as the minibatch-size increases, but we can never use full-batch in practical settings due to the large size of datasets such as CIFAR-10 (Welling& & Teh 201T). In SGLD minibatch gradients are used as proxy for the gradient for the full dataset, but these minibatch gradients contain additional noise, and there is a potential that reducing the temperature may partially compensate for this additional noise. This is particularly evident if we consider (Wenzel et al.) 2020) Fig. 6). which showed that cold-posterior effects can be amplified by using very small minibatch sizes smaller minibatches imply larger variance in gradient estimates, and hence more potential for lower temperatures to compensate for that additional noise. That said, these effects do not appear to be significant for CIFAR- 10 (Wenzel al. 2020 Fig. 5), so we are in agreement with Wenzel et al. (2020) that minibatch noise are unlikely to be the primary source of tempering effects.
5.4 CIFAR-10 WITH NOISY LABELS
Curated labels are almost certain to represent the true class, and this is one way to understand the statistical patterns induced by curation that exploited by tempered posteriors. We therefore considered disrupting this property by adding noise to the labels. In the standard classification setting, this should have very little effect (except shrink the logit outputs somewhat give more
8

---

Published as a conference paper at ICLR 2021
test llog gg-llkelhhodd
lest log sggllkellhoodd
test accuracy
acouracy
2
-1-
0.0
0.05
0.1
-0.1-
60
0.2
-2-
10
10"
10
10
10
10
10 10"
A
A
A
Figure 7: A Test log- log-likelihood with tempering for different probabilities of using a noisy label, Noiser labels mean sistllg-likeelhoode are lower. Change in log- lieelioosss from the baseline at À = 1. We see standard cold posterior effects at low noise levels, which reverse at higher noies levels. C, As A, B but for test aceuracy.
uncertain outputs). However, and as expected under our theory, we find that adding noise destroys and eventually reverses the cold-posterior effect (Fig. [ This confirms that cold-costedd aod--posteriors are exploiting specific properties of the labels that are destroyed by adding noise. As such, cold posteriors are not likely to arise e-g. from failing to capture the prior over neural network weights (in which case adding noise to the outputs should have little effect). We suspect that the small improvement in performance at the first value of below might be due to partial compensation for additional noise introduced by minibatch estimates of the gradient.
RELATED WORK
Wenzel al. 2020) introduced the cold-posterior effect in the context of neural networks, and proposed and dismissed multiple potential explanations (although none like the one we propose here).
Other past work, though not in the neural network context, argues that tempering may be important in the context of model misspecification (Grünwald) [012] Grünwald et al. (017).. Critically, we helieve that there may be many causes of cold-pon aol-trostr like effects, including but not limited to curation, model misspecificatioon and artifacts from SGLD. Ultimately, the contribution of each of these factors any given setting will depend on the exact dataset and inference method in question. Importantly, this also means we do not necessarily expect there to be no tempering in uncurated data, merely that we should see less tempering in the case of uncurated data.
Finally, the closest paper is concurrent work (Adlam et al. 2020) raising the possibilitty that BNNs overestimate aleatoric uncertainty, in part because high-quality labels available benchmark datasets. However, they concluded that while cold posteriors might help to capture our priors, they do not correspond to an exact inference procedure. In contrast, here we give a generative model of dataset curation in which tempered likelihoods emerge naturally under exact inference methods.
7 DISCUSSION AND CONCLUSIONS
We showed that modelling the process of data-curation can explain the improved performance of tempering cold posteriors in Bayesian neural networks. While this work does provide a justification for future practitioners to use tempering in their Bayesian neural networks, we would urge caution. Importantly, and as confirmed Fig. [ there may other causes of the oold-rosterior effect, e.g. because inference is inaccurate (compensating for additional noise added in SGLD), or may compensate for issues arising from model mispecification (Grünwald 2012 Grünwald et al.) 2017). As such, we urge practitioners to regard tempering with caution: if a very large amount of tempering necessary to achieve good performance, may indicate issues with either inference or the prior, and fixing these issues is of the utmost importance to obtaining accurate uncertainty estimation. More importantly, we hope that our work will prompt more careful dataset design, and further study of how data curation might impact downstream analyses in machine learning. Indeed, there are initial suggestions that semi--supeeeeee aemi-supervised learning methods are also exploiting the artificial clustering (Fig. induced by data curation (Aitchison 2020).

---

Published as a conference paper at ICLR 2021
ACKNOWLEDGEMENTS
I would like to thank Adrià Garriga-Alonoo and Sebastian Ober for insightful discussions, Stoil Ganev for the GZ2 analyses which sadly came only during the ICLR review period which point author changes are banned (which is odd, given that revising the paper during the review period is allowed) and Mike Walmsley and Sotiria Fotopoulou for getting us set up with G72. would also like to thank Bristol's Advanced Computing Research Centre (ACRC) for providing invaluable compute infrastructure that was used for all the experiments in this paper.
REFERENCES
Ben Adlam, Jasper Snoek, and Samuel L Smith. Cold posteriors and aleatoric uncertainty, arXiv preprint arXiv:2008. 00009,, 2020.
Laurence Aitchison. A statistical theory of semi-supervised learning. arXiv preprint, 2020.
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling in deep learning. arXiv preprini arXiv:2002.06470, 2020.
Juhan Bae, Guodong Zhang. and Roger Grosse. Eigenvalue corrected noisy natural gradient. arXiv preprint arXiv:1811.12555 2018.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. Icee, 2009.
Peter Grünwald. The safe bayesian. In International Conference on Algorithmic Learning Theory, pp. 169-183. Springer, 2012.
Peter Grünwald, Thijs Van Ommen, et al. Inconsistency of bayesian inference for misspecified linear models, and a proposal for repairing Bayesian Analysis, 1264):1069-111 2017.
James Hensman, Alexander Matthews, and Zoubin Ghahramani. Scalable variational gaussian process classification. JMLR, 2015.
Edwin T Jaynes. Probability theory: The logic of science. Cambridge university press, 2003.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Toronto, 2009.
Alexander G de G Matthews, James Hensman, Richard Turner, and Zoubin Ghahramani. On sparse variational methods and the -eibllr divergence between stochastic processes. Journal of Machine Learning Research, 51:231-239 2016.
Sebastian W Ober and Laurence Aitchison. Global inducing point variational posteriors for bayesian neural networks and deep gaussian processes. arXiv preprint arXiv:2005.081 08140, 2020.
Kazuki Osawa, Siddharth Swaroop, Mohammad Emtiyaz E Khan, Anirudh Jain, Runa Eschenhagen, Richard E Turner, and Rio Yokota. Practical deep learning with bayesian principles. In Advances in Neural Information Processing Systems, pp. 4289-4301, 2019.
Joshua Peterson, Ruairidh M Battleday, Thomas Griffiths, and Olga Russakovsky. Human uncertainty makes classification more robust. In Proceedings of the IEEE International Conference on Computer Vision, pp. 9617-9626. 2019.
Twan Van Laarhoven. L2 regularization versus batch and weight normalization. arXiv preprint /xXiv::706.053500 2017.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-1I), pp. 681-688, 2011.
10

---

Published as a conference paper at ICLR 2021
Florian Wenzel, Kevin Roth, Bastiaan S Veeling, Jakub Swiętkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good the bayes posterior in deep neural networks really? arXiv preprint S0t1::2002.02400.. 2020.
Kyle W Willett Chris Lintott, Steven P Bamford, Karen L Masters, Brooke D Simmons, Kevin RV Casteels, Edward M Edmondson, Lucy F Fortson, Sugata Kaviraj, William C Keel, et al. Galaxy zoo 2: detailed morphological classifications for 304 122 galaxies from the sloan digital sky survey. Monthly Notices of the Royal Astronomical Society, (35(4))2835-286)) 2013.
Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning. MIT press Cambridge, MA, 2006.
Guodong Zhang, Shengyang Sun, David Duvenaud, and Roger Grosse. Noisy natural gradient as variational inference. arXiv preprint arXiv: 2017.
Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, and Andrew Gordon Wilson. Cyclical stochastic gradient meme for bayesian deep learning, arXiv preprint 1922::190..033322 2019.

---

Published as a conference paper at ICLR 2021
B
test likelhood
-0.50
test accuracy (%)
-0.75
70
-1.00
60
-1.25
10
10
10
10
10
10
10
10
Figure SGD a ResNer18 with batchnorm removed with a tempered maximum a-posteriori loss. A Test log-likelihood. B Accuracy.
A COLD POSTERIORS IN MAXIMUM A-POSTERIOR INFERENCE
Our explanation for the cold posterior effect suggests that it is not just due to the stochasticity of the posterior, but should also arise in e.g. maximum a- posottriorii (MAP) inference. However, MAP inference causes serious issues in networks with batchnorm. In particular, with batchnorm, the outputs become invariant to the scale of the weights, so the weights can continually decay towards zero (Van Laarhoven 2017). Note that these considerations full approximate inference, as the product of the prior and likelihood a static quantity with a well-defined scale that cannot e.g. decay towards zero. As such, considered tempering in our usual R8sNet18, but where we have deleted the batchnorm layers. We used standard protocol for these types of network: SGD with momentum of 0.9 and an initial learning rate of 1, followed by decays to 0.01 and 0.001 at epochs 150 and 200. We read off performance at the final epoch (250) and used batch size of 128. We use standard prior over the weights, with variance 2/ 2/fan-in, to keep the scale of the inputs and outputs similar.
relevant when are doing
Remarkably, we found extremely strong cold-postern posterior effects (Fig. 8. These are perhaps expected as it is related to MAP inference with a Gaussian prior, good settings for the weight-decay coefficient are much smaller than those suggested by MAP inference with sensible prior. Interestingly, these results suggest that the robustness of standard models to very low temperatures might arise due to batchnorm, and not be fundamental property (e.g. having very large amounts of data).
(but difficult to find a reference for) that while weight-decay is closely
12

---

Published as a conference paper at ICLR 2021
B
test tast log-lkkelhood
=
-0.4
S
S
S-2
4
S=1
2
-0.8
0
10
10
10
10'
0
2
4
$
Figure 9: Toy BNN model. Test log-likelihood vs for different settings of S. The optimal X, plotted against S.
B TOY BAYESIAN NEURAL NETWORK EXAMPLE
To confirm the results in the Gaussian processes toy example in the main text, did the same exercise with toy Bayesian neural network. In particular, we considered 100 training examples, drawn IID from standard Gaussians, each with 5 input dimensions and class outputs. We generated the "true" or target outputs from a one hidden layer ReLU network, with 30 hidden units, drawn from the prior. We trained the same network, using Langevin dynamics (full batch, but rejection step; initialized randomly from the prior). As in the GP example, we applied our generative model of dataset curation with different settings for S in the generative model. For test and training, used the standard single-labeller log-likelihood, which ignores rejected noconsensus inputs. We implemented highly parallelised Langevin sampling algorithm. which allowed us draw 500 different datasets, and run one chain for each dataset (and each setting of X in parallel). The large number of chains allowed us to obtain very accurate results, but did mean that it was not possible to plot lines for individual datasets. We used learning rate of 0.01 and burn in of 4,000 steps; to compute performance, we averaged over 50 samples of neural-network parameters, which were taken from chain that was 20,000 steps long in total.
In Fig. A we plot the test-log-likelnng likelihod for different settings of A. There a clear trend that as increases, the optimal temperature falls, and at the same time, the curves become flatter. Notably, these curves become flatter at the top for larger settings S, making it difficult to identify the optimal setting for À at higher values of S. We then plotted the optimal setting of À against S (Fig ) Initially, S = 1 is equivalent to the standard classification setup, as consensus is always reached, and such, find that the optimal temperature was 1. As in Fig. 4 approximately. Nonetheless, these results suggest that we might need more tempering than suggested by X 1/S. which supports our proposed explanation of the cold posterior effect. Finally, should be noted that these results depend sensitively on the accuracy of the inference algorithm, so it is difficult say as yet the degree to which mismatch between X and 1/S arises due inaccuracy in inference or would emerge in the intractable -osterter
found that X 1/S held only
13

---

Published as a conference paper at ICLR 2021
C GALAXY ZoO 2 DETAILS
The Galaxy Zoo data set (Willett et al. to volunteers, who were asked to answer a questionnaire about the morphological features of the presented galaxy. These questions are not independent from each other but are part of a decision tree. As such, the questions volunteer gets asked depends on their answers earlier questions. For each galaxy, answers from multiple volunteers were collect. The raw data is recorded as counts of how many times each answer was picked across all volunteers.
2013) was constructed by presenting images galaxies
From this data we wanted to derive classes in such a way that the classification performed by each volunteer was equivalent to a vote towards one of our classes. The most straightforward approach is to consider each unique path in the decision tree to be unique class. Given that volunteer fills in single vote towards a specific class.
the questionnaire only once, their classification equivalent The problem with this approach is that there are 1265 possible paths in the original decision tree. The number of volunteers per galaxy less than a hundred, which means that the majority of these paths would receive 0 votes.
In order to address this, we simplified the tree to one where there are only possible paths. The simplifications performed by manually pruning branches of the decision tree. This way each path in the original tree could be mapped to path in the simplified onc, while maintaining specific aspects of its meaning. The simplified tree can be seen in (modified graph.png). m The following pruning operations were performed in order to create this simplified tree. Here we refer to each question using its task number, as given in the original paper.
Task 02, Answer 'no": leads directly to Task 05, skipping Task 03, Task 04, Task 10 and Task 11
Task 05, Answers 'dominant" and' "obvious"; merged together becoming combined category labelled 'obvious'
Task 05, all answers: lead directly to the end of the graph, becoming exit points
Task 07, all answers: lead directly to the end of the graph, becoming exit points
Task 09, Answers 'rounded' and 'boxy": merged together becoming a combined category labelled 'with bulge'
Task 09, all answers: lead directly to the end of the graph, becoming exit points
As a consequence of the above pathing changes Task 03, Task 04, Task 06, Task 08, Task 10 and Task 11 are no longer reachable and thus removed from the graph
If we are to take each of the paths in the simplified tree to form a class, then the resulting 9 classes would have the following meanings:
smooth galaxy, completely round shape
smooth galaxy, in-between round and cigar shaped
smooth galaxy, cigar shaped
edge-on disk galaxy, with central bulge
edge-on disk galaxy, no central bulge
face-on disk galaxy, obvious central bulge
face-on disk galaxy. just noticeable central bulge
face-on disk galaxy, no central bulge
star or artifact
Using the simplified decision tree, we can decompose the data into votes for each path. The Galaxy Zoo data was recorded as counts for each specific answer, thus losing the information about which route specific volunteer took in the tree. However, with the help of network flow algorithms it is possible to translate the data into counts of how many volunteers took a specific path. This way, by taking each path be class and the number of volunteers that followed path be the number of votes toward a class, we have a data set where each data point has received multiple classification over fixed range of classes.
14

---

Published as a conference paper at ICLR 2021
to the galaxy simply smooth and rounded, with sign disk?
Could edge-on?
How sounded is it?
Is there sign of bar fea through gaaxy?
Does the galaayy have bulge at its centre?
so, what shape?
anything odd?
End
X
Is there any sign of upiral arm pallerm?
How tighlly wound the spiral arms appear?
feature ring, the galaxy disturbed or irregular?
How there?
How prominent is the central bulge, compared galaxy?
Figure 10: Modified decision from Willett al. (2013) showing how truncated the decision tree. Images in the same box are viewed as the same class.
For each galaxy in the Galaxy Zoo data volunteers. As such, we can use our derived classes in an image classification task. In order to do this, performed certain pre-processing. The original images have resolution of 424x424 with the galaxy being at the center of the image. Thus, we performed center crop of size 212x212 and then resized the images down 32x32. For data augmentation, we used full 360 degree continuous rotations.
also have the image that was presented to the
15


Reviews:
[{"review_id": "m48thS5gXB3", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose the idea that cold posteriors in Bayesian neural networks could be caused by the likelihood instead of the prior. They argue theoretically that the curation process of popular benchmark data sets would lead to a different weighting of the likelihood in the posterior. They show in some experiments that the cold posterior effect can be reduced when accounting for this.\n\nMajor comments:\n- The paper title suggests that it is about cold posteriors, and it quite prominently references [1] in the introduction. However, in Sec. 2, it is then clarified that the paper is in fact not about cold posteriors, but about tempered ones. It is just briefly mentioned in passing that the results should transfer, but this is never tested. I think an experiment on actual cold posteriors, similar to the one in [1], would be warranted to support such a statement and the usage of the current paper title.\n- The theory suggests that the optimal posterior performance should be achieved at lambda=S, that is, cooling down beyond that point should deteriorate performance again. This is an interesting prediction, since it does not seem to fit the observations in [1]. It would be nice to see this confirmed on an actual BNN experiment, similarly to what can be seen in the toy GP experiment.\n- The related work section seems awfully short. It does not even mention [1] (although it is cited heavily elsewhere). Moreover, for a paper that is proposing a statistical theory of tempered posteriors, works such as [2] and [3] should probably be mentioned.\n\nMinor comments:\n- Sec. 4.2: \"we when\" -> \"when\"\n\nSummary:\nThe idea that cold/tempered posterior effects can be caused by the data set curation instead of by misspecified priors is very interesting and definitely deserves a theoretical and empirical investigation. However, the investigation at hand seems a bit incomplete in some places, especially the related work and experiments sections. Also the title does not currently seem to fit the experiments. Given that the current manuscript is comfortably within the ICLR page limit, I'm hopeful that these points can be addressed in a revised version during the discussion phase. \n\nUpdate: I increased my score following the clarifications and addition of the BNN experiment during the discussion phase.\n\n[1] Wenzel, F., Roth, K., Veeling, B. S., Świątkowski, J., Tran, L., Mandt, S., ... & Nowozin, S. (2020). How good is the bayes posterior in deep neural networks really?. arXiv preprint arXiv:2002.02405.\n[2] Grünwald, P. (2012, October). The safe bayesian. In International Conference on Algorithmic Learning Theory (pp. 169-183). Springer, Berlin, Heidelberg.\n[3] Grünwald, P., & Van Ommen, T. (2017). Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it. Bayesian Analysis, 12(4), 1069-1103.", "extra_scores": {}, "extra_text": {}}, {"review_id": "j7zT_Q6h5W", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Apologies for the late review!\n\n## Summary of the Paper\n\nThe paper provides a potential theoretical explanation of the known empirical observation that cold (or tempered) posteriors improve predictive performance of deep Bayesian neural networks. The provided explanation is simple and it leads to additional predictions, which the authors check empirically as far as possible with existing data sets. The empirical results agree with the predictions.\n\n## Main Strengths\n\nI believe the main message of the paper is so relevant and seems so simple (at least in hindsight) that it has the potential of becoming a kind of \"common knowledge\" in Bayesian neural networks community (caveat: I can't judge if these findings had already been informally known to a larger group of researchers, but unless someone has explicitly written them down somewhere I wouldn't hold this against the paper).\n\nResearchers have been trying to increase predictive performance of deep neural networks by applying scalable Bayesian methods to deep learning for a while, but even replicating the performance of point estimated models with Bayesian neural networks has proven surprisingly difficult. To my knowledge, it was found out only recently that Bayesian neural networks systematically outperform their point estimated counterparts if the prior is made artificially sharper than what probability theory would predict (i.e., by \"lowering the temperature\"). This empirical result has been puzzling from a theoretical perspective, but the present paper provides a simple potential explanation for this effect.\n\nI believe the findings in this paper go beyond a theoretical justification of an empirically known fact. The findings may also have implications on model robustness: the authors argue that the \"tempering effect\" is a result of curation of the training set. Validations and tests sets are typically curated in the same way as the training set in the machine learning community. However, when models are deployed in the field, they typically see uncurated data points. I would be curious to know if explicitly modeling the curation process, as the authors do in this paper, would also address this issue.\n\n## Potential Weaknesses\n\nThere's one caveat to my review: I am not an expert on Bayesian neural networks and, as stated above, the argument made in this paper seems so simple in hindsight that I cannot say with absolute certainty that it hasn't been made before. I personally haven't heard this argument before, but if other reviewers can point to a reference that already made this argument, then that would probably be the only thing that could convince me to lower my rating. Otherwise, I would consider the simplicity of the authors' argument a strength of the paper.\n\n## Questions to the Authors\n\n- What do the authors mean with the phrase \"finite networks\" in the first paragraph? Is it the same as networks with point estimated parameters (as opposed to Bayesian neural networks)?\n- As mentioned above, the paper models the curation of the training set, but I didn't understand how or if curation of the test set is modeled. Could the authors clarify this? Specifically, what changes for a model trained on curated data when it is either (a) tested on an equally curated test set or (b) applied to uncurated data in the wild? Would the optimal $\\lambda$ during training differ between cases (a) and (b) or would the posterior have to be changed after training?\n\n## Minor Issues\n\n- I like the short Section 2 (which compares \"cold\" and \"tempered\" posteriors) very much! I think it could even be improved by adding one reference each for \"cold\" and \"tempered\" posteriors, respectively. More importantly, as far as I understand, the two are really essentially the same if one uses, e.g., a Gaussian prior. Unless I'm mistaken, the missing factor of $\\frac{1}{T}$ in Eq. 2 could then be absorbed into a rescaling of the prior covariance (unless the prior covariance itself is learned with expectation maximization). If this is correct then I'd add a corresponding statement at the end of Section 2 (it would make the paper's claims more widely applicable).\n- I think Figure 2 is never discussed in the paper (but there should be enough space left to discuss it). The figure caption says \"Schematic diagram\". Could the authors clarify what this means? Do the points come from some toy model with a 2-d parameter space (or does the figure show a 2-d PCA of the parameter space) or were the points really just drawn manually to visualize the idea? I think both would be fine but I would be very curious to know how the figure looks with real data.\n- In Figure 4, the last panel is labelled \"F\" but referred to as \"E\". Also, in the last two panels, the left dashed vertical bar is not discussed. Is it the theoretically expected optimal value of $\\lambda$ (i.e., $\\lambda=\\frac{1}{4}$) or the empirically found optimal value?", "extra_scores": {}, "extra_text": {}}, {"review_id": "NSDkHPKZbp0", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The work propose a theory suggesting that the cold posterior phenomena arises solely due the the curated nature of image benchmarks. A generative model is proposed where multiple annotators label datapoints, and only unanimously labeled datapoints are accepted into a dataset. This theory is studied under a toy-problem using VI and a relabelled version of the CIFAR-10 test set with SGLD. \n\nHowever, many questions remain unanswered and the proposed theory is not sufficiently studied.\n\n- Q: The cold posterior problem was highlighted in the SGMCMC case, but this work's main toy problem only explores tempered posteriors as prevalent in VI. It would be beneficial if the work highlights why these results should extend to the cold posterior, or better yet, run experiments in this scenario.\n- Q: 4.1 strongly suggests that there is a relationship between between \\lambda and S in the toy problem. It should be an easy addition to study this connection for a range of values for S to explore if this holds.\n- Q: The work claims that the consensus protocol for standard datasets is not available, but it would appear that this is a a simple manner of reaching out to the authors of the datasets.\n- Q: The main experiment presented in figure 5 is missing some important ablations: what happens when the CIFAR-10 baseline is trained under the same conditions (learning rate) as CIFAR-10H?\n- Q: Why is it acceptable to use the original CIFAR-10 training-set as the test-set for CIFAR-10H? This seems like a problematic shift in data distribution.\n- Q: the theory of dataset curation is interesting but makes a broad claim. More datasets should be explored from varying modalities. The sole focus here on CIFAR-10 provides too little evidence. As a suggestion: curation processes are different for e.g. medical imaging datasets, and typically well documented. \n- Q: It's unclear to me why it is acceptable to increase the training set size by the number of annotators. An increase of factor 50 is effectively setting the temperature to ~ 1e-2. At this temperature the baseline performs just as well. Does this large gap still hold if just a much smaller subset of annotators is taken from the CIFAR-10H dataset?\n \nThe core idea proposed in this work is thought provoking and contributes to the discussion on this topic. The work is relatively short, which is not a problem in its own right, but the experiment section needs to provide more evidence and analysis. I vote for reject.\n\nNitpicks:\n- F instead of E in figure 4.\n- \"As expected, we when\"\n\nUpdate: I've increased my score.", "extra_scores": {}, "extra_text": {}}, {"review_id": "DSud5vd8qzO", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses the perplexing issue of cold posterior having better predictive performance than the ideal Bayesian posterior in Bayesian deep learning (Wenzel et al., 2020), and offers a possible explanation in terms of a mis-specified likelihood function that deviates from the true generative process of the data. By considering the data curation process and augmenting the likelihood model accordingly, the effect of cold posterior is shown to diminish significantly, and the ideal posterior is again optimal. Empirical results on both a toy problem and image classification support the theory.\n\n\n------------------\n\nPros:\n1. Given the prevalence of Bayesian deep learning and the issue of cold posterior, this paper offers a timely contribution that bridges theory and practice.\n1. The paper is well written and motivated, the method appears sound (but see questions below), and concepts are explained in a clear and pedagogical manner.\n2. The experiments are well thought out and offer clear empirical support of the proposed hypothesis.\n\n------------------\n\nCons:\nThis might be due to my limited understanding of the paper, but I think there are still some limitations to the paper's proposed theory, e.g., it doesn't explain the observation that extremely cold posterior (λ -> 0) doesn't seem to hurt the performance of BNN (which should, according to the proposed theory, as there is only one optimal temperature λ = 1 / S, where S is the true number of underlying labelers), and more below.\n\n------------------\n\nQuestions and Comments:\n1. My biggest confusion is this: the paper argues that it's incorrect to assume a simple categorical likelihood p(y|x) as it doesn't take into account the data curation process; however, under the extended likelihood model as proposed, when conditioning on the event that y!=None and x!=None (as we do when training on standard datasets), and after marginalizing out the intermediate variables and renormalizing, isn't the conditional distribution p(y|x) still just a categorical distribution (except parameterized in a different way now)?  If so, then the difference between the two likelihoods is really just a different parameterization, and I'm no longer sure what to make of the suggested theory and the supporting results.  I find it very surprising that the more complex parameterization significantly reduced the tempering effect.  And if the we take the ground truth likelihood p(y|x) to be as in the standard (curated) dataset, which the paper argues is in some sense artificially \"tempered\", then why can't a well-spcified BNN just adapt to this (still categorical) likelihood and learn an optimal posterior under it?  I'm happy to raise my score if the authors can clarify these issues for me.\n2. Since point estimation with SGD optimizes the same likelihood function, why don't we observe the tempering effect in SGD? Perhaps there is some effect but rather minimal; in any case, some experiments on SGD (with / without the corrected likelihood) would be interesting.\n3. Related to my comment about \"extremely cold posterior\": in the GP experiment, when trained and tested on the corrected likelihood (considering curation), the test performance seems to really prefer the optimal λ = 1, whereas on the image experiment, more tempering (λ -> 0) doesn't seem to affect test performance. Is there an explanation for this?\n4. Finally, does the proposed theory explain the observation that the cold posterior effect is more prominent in BNN with higher capacity (Wenzel et al. 2020)?\n\n\n------------------\n\nPossible typos and minor mistakes:\n1. p.3, under eq (7), \" This likelihood is equivalent to labeling each datapoint S times with the same label, and therefore has exactly the effect of setting λ = S in a tempered posterior\". Should be \"λ = 1/S\" instead.\n2. The right-most subfigure in Figure 4 should be labeled \"E\" instead of \"F\" to match the caption below.\n\n\n------------------\nUpdate:\n\nI've raised my score in light of author response and new results. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This is an interesting, controversial paper that contributes to an ongoing debate in Bayesian deep learning.

Bayesian inference with artificially “cooled” posteriors (e.g., trained with Langevin dynamics with down-weighted noise) was recently found to outperform over both point estimation and fully-Bayesian treatments (Wenzel et al., 2020). This paper proposes a new explanation for these observed phenomena in terms of a data curation mechanism that popular benchmark data sets such as CIFAR underwent. The analysis boils down to an evidence overcounting/undercounting argument and takes into account that curated data sets only contain data points for which all labelers agreed on a label. The authors claim that, when modeling the true generative process of the data, the cold posterior effect (partially) vanishes.

The paper is well-written and provides a consistent analysis by modeling the data curation mechanism in terms of an underlying probabilistic graphical model of the labeling mechanism. Unfortunately, several observed phenomena of (Wenzel et al., 2020) remain unexplained by the theoretical arguments, e.g., the fact that “very cold” (T --> 0) posteriors don’t hurt performance, or the observation that the optimal temperature seems to depend on the model capacity. While the proposed explanation doesn’t capture the full picture (upon which both authors and reviewers agree), the paper’s focus on the data curation process, supported extensive experiments, gives a partial explanation and provides an interesting perspective that will spur further discussion and should be of broad interest to the Bayesian deep learning community. 

Decision: Accept (Poster)