On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification
Authors: Sanyam Kapoor
Wesley Maddox
Pavel Izmailov
Andrew Gordon Wilson
Venue: NeurIPS 2022 Accept (2022)
Keywords: probabilistic methods
deep learning
Bayesian deep learning

Abstract: Aleatoric uncertainty captures the inherent randomness of the data, such as measurement noise. In Bayesian regression, we often use a Gaussian observation model, where we control the level of aleatoric uncertainty with a noise variance parameter. By contrast, for Bayesian classification we use a categorical distribution with no mechanism to represent our beliefs about aleatoric uncertainty. Our work shows that explicitly accounting for aleatoric uncertainty significantly improves the performance of Bayesian neural networks. We note that many standard benchmarks, such as CIFAR-10, have essentially no aleatoric uncertainty. Moreover, we show that data augmentation in approximate inference softens the likelihood, leading to underconfidence and misrepresenting our beliefs about aleatoric uncertainty. Accordingly, we find that a cold posterior, tempered by a power greater than one, often more honestly reflects our beliefs about aleatoric uncertainty than no tempering --- providing an explicit link between data augmentation and cold posteriors. We further show that we can match or exceed the performance of posterior tempering by using a Dirichlet observation model, where we explicitly control the level of aleatoric uncertainty, without any need for tempering.

On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification
Sanyam Kapoor* New York University
Wesley Maddox New York University
Pavel Izmailov* New York University
Andrew Gordon Wilson New York University
Abstract
Aleatoric uncertainty captures the inherent randomness of the data, such as mea- surement noise. In Bayesian regression, we often use a Gaussian observation model, where we control the level of aleatoric uncertainty with noise variance pa- rameter. By contrast, for Bayesian classification with no mechanism to represent our belicfs about alcatoric uncertainty. Our work shows that explicitly accounting for alcatoric uncertainty significantly improves the performance of Bayesian neural networks. We note that many standard bench- marks, such CIFAR-10, have essentially aleatoric uncertainty. Moreover, we show that data augmentation in approximate inference softens the likelihood, leading underconfidence and misrepresenting tainty. Accordingly, we find that a cold posterior, tempered by a power greater than one, often no tempering - providing an explicit link between data augmentation and cold posteriors. We further show that we can match or exceed the performance of posterior tempering by using Dirichlet observation model, where we explicitly control the level of alcatoric uncertainty, without any need for tempering.
categorical distribution
beliefs about aleatoric uncer-
honestly reflects our beliefs about aleatoric uncertainty than
Introduction
Uncertainty is often compartmentalized into epistemic and aleatoric uncertainty [27, 33, 41]. Epis- temic uncertainty, sometimes called model uncertainty, is the reducible uncertainty about which solution correct given limited information. Bayesian methods naturally represent epistemic uncer- tainty through distribution over model parameters, leading posterior distribution over functions that are consistent with data. As we observe more data, this posterior concentrates around a single solution. Aleatoric uncertainty the irreducible uncertainty in data, often representing measurement noise in regression, or mislabeled training points in classification [e.g. 4]. Although measurement noise be reduced, for example, with better instrumentation, it is often fixed property of the data we are given. Correctly expressing our assumptions about alcatoric uncertainty is crucial for achieving good predictive performance, both with Bayesian and non-Bayesian models [56].
In particular, our assumptions about alcatoric uncertainty profoundly affect predictive accuracy, not only predictive uncertainty.' In Figure ((a,b) show the predictive distributions of two Gaussian process (GP) regression models [54] trained on the same data and using the same RBF kernel func- tion. The only difference between these models is their assumptions about the aleatoric uncertainty. The model in Figure I(a) assumes a high observation noise (each point is corrupted by A (1 o²),
"quual contribution.
"Eiistemic uncertainty also has significant effect predictive accuracy, Wilson and Izmailov [62].
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

-2
0.0 0.5 1.0
0.0 0.5
1.0
Glass 0 Class 2
x
Scumarlo
(a) GP regression,
=1 (b) GP regression, o² = 10
(c) Classification
Figure/ 1: Effect of aleatoric uncertainty. In regression, we can express our assumptions about aleatoric uncertainty by setting the observation noise parameter o². (a): GP regression with high observation noise o² explains many of the observations with noise. The datapoints are shown with purple circles, the dashed line shows the predictive mean, and the shaded region shows one standard deviation of the predictive distribution. (b): GP regression on the same data, but with low obser- vation noise o² fits the training data nearly perfectly. (c): In classification, we do not have direct way to specify our assumptions about aleatoric uncertainty. We might use the same Bayesian neural network model if we know the data contains label noise (Scenario A) and if know that there is no label noise (Scenario B), leading to poor performance in at least one of these scenarios.
with x² 1). Consequently, this model explains many of the observations with noise such that the predictive mean does not closely fit the training datapoints. The model in Figure 1(b) assumes a low observation noise (o² 10 3), and consequently the predictive mean runs through the training data, leading to very different predictions compared to the model in Figure (a)).
Our assumptions about the aleatoric uncertainty are just important in classification. We illustrate this point in Figure I(c), where we fix a dataset and consider two possible scenarios. Scenario A: suppose know that the data contains label noise, with some incorrectly labeled training examples. In this case, it is reasonable to believe that the data points shown in red in Figure ((c) are highly likely be incorrectly labeled, and should opt for solution with decision boundary shown with dotted red line. Scenario B: now, suppose that we know that the data are correctly labeled. In this case, we want our model to perfectly classify all the training data and would prefer solution shown by the dashed blue line. Both of these solutions are reasonable descriptions of the data, and we can only make an informed choice between them by incorporating our assumptions about the aleatoric uncertainty. While in regression problems we can easily express our assumptions about aleatoric uncertainty, such as through a noise variance parameter we do not have direct way to express these beliefs. Indeed, in practice we might use the same Bayesian neural network (BNN) with the same likelihood and prior in both scenarios A and B above, necessarily leading poor performance in least one of these scenarios.
Gaussian likelihood, in classification
In this paper, we explicitly characterize aleatoric uncertainty BNN classification. We show that cold posteriors [61] more honestly reflect our beliefs about aleatoric uncertainty than standard Bayesian posteriors. Further building on our analysis of aleatoric uncertainty in Bayesian classi- fication, we describe an alternative to cold posteriors called the noisy Dirichlet model. The noisy Dirichlet model can be viewed as simply modifying the prior over the parameters of BNN to put more mass on the solutions that make highly confident predictions on the training data, but only in scenarios with low aleatoric uncertainty. Unlike standard BNN classifiers, we get control over the aleatoric uncertainty through the prior. This approach achieves competitive performance on image classification problems with a valid Bayesian model and removes the cold posterior effect.
Finally, we show theoretically how data augmentation counterintuitively softens the likelihood, lead- ing solutions that underfit the training data. These results precisely characterize the empirical link between data augmentation and the success of cold posteriors observed by Izmailov et al. [31] and Fortuin al. [14], and show that cold posteriors fact serve to correct for the effects of data augmentation on aleatoric uncertainty.
2
Related Work
Early work on Bayesian neural networks focused on hyperparameter learning and mitigating over- fitting, using Laplace approximations, variational methods, and MCMC [e.g. 37, 25, 47). More

---

recently, Wilson and Izmailov [62] show how Bayesian model averaging is especially compelling for modemn deep neural networks. A number of recent works have focused on making Bayesian deep learning practical [e.g., 63, 38, 49, 65, 12, 11], with improved performance, and essentially no overhead.
Modeling label noise. In the context of label noise for classification, prior works focus on mod- ifying the loss function. For example, Collier et al. [9, 10] propose input-dependent noise losses for label noise in classification to improve accuracy, but do not focus on the uncertainty estimates. Similarly, Zhang and Sabuncu [66] and Feng et al. [13] propose modifications of the standard cross- entropy loss to provide increased robustness to label noise, but do not focus on modeling the inherent uncertainty in label noise settings.
Dirichlet-based uncertainty (DBU). Recent literature proposes neural networks where the output is Dirichlet distribution, capturing both the epistemic and aleatoric uncertainty [41, 57, 40, 42, 7, 67, 46, 59, 58, 34]. These models are generally proposed as efficient alternatives to ensembles and Bayesian models, and targeted for improved uncertainty quantification, anomaly detection and adversarial robustness. Joo et al. [32] reformulate the problem of minimizing the softmax cross- entropy loss over the data into a collection of distribution matching problems. In this paper, we study the Dirichlet distributions associated with the posterior over the predictive class probabilities, which arise naturally in standard Bayesian classification models. Contrary the methods in the DBU family, we do not modify the observation model or the structure of the network.
Cold posterior effect. Wenzel et al. [61] demonstrated with several examples that raising the pos- terior to a power 1/T, with T < 1, in conjunction with SGLD inference, improves performance over classical training, but T 1 can provide worse results than classical training, in what has been termed the cold posterior effect. However, in study using HMC inference, Izmailov et al. [31] showed that for all cases considered in Wenzel al. [61] there is no cold posterior effect when data augmentation is removed. The sufficiency of data augmentation to observe the cold posterior effect has been confirmed by several works using SG-MCMC inference [14, 48, 45]. Several works have suggested that misspecified priors explain cold posteriors, arguing that the current default choice of isotropic Gaussian priors BNNs are inadequate [61, 64, 14]. However, Fortuin al. [14] find that the cold posterior effect cannot be alleviated for convolutional neural networks by using heavy tailed or correlated priors under data augmentation. With HMC inference and no data augmentation, Izmailov et al. [31] also show that standard Gaussian priors perform similarly to a range of other pri- ors, such as heavy-tailed logistic or mixtures of Gaussian priors, and generally are high performing- Noci et al. [48] argue that many different factors- likelihoods, priors, data augmentation cause cold posteriors, and that there may be no one cause in general. Aitchison [2] argue that BNN likelihoods are misspecified since many benchmark datasets have been carefully curated by human labelers. They also show that posterior tempering is less helpful in the presence of label noise. Ad- lam et al. [1] also consider model misspecification, showing a cold posterior effect can arise with exact inference in GP regression when the aleatoric uncertainty is misestimated. Nabarro et al. [45] modify the likelihood to accommodate data augmentation, using the observation model the average of the neural network outputs over augmentations, but still find a cold posterior effect. In concurrent work, Bachmann et al. [3] provide an alternative argument how data augmentations introduce misspecification due to correlation between errors which cold posteriors correct for.
can
Our paper makes several distinctive contributions in the context of prior work: (1) we argue that standard likelihoods do not represent our beliefs about aleatoric uncertainty, and that standard bench- marks have essentially no aleatoric uncertainty; (2) we show how tempering and data augmentation specify alcatoric uncertainty, beyond curation; (3) we show the precise way in which data augmen- tation with SGLD leads to underconfidence in the likelihood, counterintuitive result which finally resolves the empirical connection between data augmentation and cold posteriors; (4) we show that a
T < 1, particularly with data augmentation, is a more honest reflection of our beliefs about aleatoric uncertainty than 7 = 1; (5) we show how the noisy Dirichlet likelihood, originally used for tractable GP classification [43], can naturally reflect our beliefs about aleatoric uncertainty, and for the first time remove the cold posterior effect in the presence of data augmentation.
3

---

3 Background
Bayesian Model Averaging. With Bayesian inference, we aim infer the posterior distribu- tion over parameters having observed a dataset 'D {(x,, yi) ] of input-output pairs, given by p(w] x w)p(w) for given observation likelihood under the i..d. assumption p(D w) = (( Ply æ, w), and prior over parameters p(w). For any novel input 2.. we compute the posterior predictive distribution via Bayesian model averaging (BMA) given by
ply. x.) = p(y. x.,, w) D)dw.
(1)
The BMA integral in Eq. (1) cannot generally be expressed in closed form for Bayesian neural network, and is typically approximated with variational inference (VI), the Laplace approximation, or Markov Chain Monte Carlo (MCMC) [e.g.. 62, 44].
Bayesian Classification. For a C-class classification problem, standard choice of observation likelihood is the categorical distribution p(y w) Cat([ TTl), where each class proba- bility is computed using a softmax transform of logits g( w) [g (z: w).... gC (æ; w)] E RC as Te & fe( w) = e (æ; w)), and hence called the softmax likelihood. The prior over parameters p(w) is often chosen to be an isotropic Gaussian N (0. o² o²1) with some fixed variance a². Approx- imate posterior samples are often obtained via SGLD [60]. and used for a Monte Carlo estimator of the BMA in Eq. (1).
Cold Posteriors. Let U(w) log P(D w) log p(w) denote the posterior energy function of a BNN. Following Wenzel et al. [61], we define cold posterior for T < 1 as
1 (w (D) x exp { 1-JUG U(w)
(2)
which effectively raises both the likelihood and the prior to a power 1/T. T recovers the standard Bayes' posterior. A tempered likelihood posterior (c.g. [17]) only raises the likelihood to power 1/T as
Premp (w D) x p(D w) p(w))
Aleatoric Uncertainty in Bayesian Classification
We will now discuss how we represent aleatoric uncertainty in classification (Section 4.1), how tempering reduces aleatoric uncertainty (Section 4.2), and how we can explicitly represent aleatoric uncertainty without tempering with noisy Dirichlet observation model (Section 4.3).
4.1 How do we represent aleatoric uncertainty in classification?
Let us consider the Bayesian neural network posterior over parameters in classification problem: p(w D) xx p(w) Il So (2; w), where we denote the output of the softmax layer of the model corresponding to class y on input x as (x;
think of the class probability f(x) & fx; [S (2; w), fo(æ; w)] latent variables? in C simplex A C-1 such that Ee= fa(a;; w) = 1. A prior distribution p(w) over the parameters of the network implies joint prior distribution over f (f(z) 2EE.. We can then do inference directly in function space over the variables f. yielding a posterior p(f (D) equal to the distribution over implied by the the parameter space posterior p( (D) [see c.g. 54, Ch. 2.2]. We will use this function space posterior to analyze the assumptions on alcatoric uncertainty encoded by the Bayesian classification models.
Alternatively,
can
vectors
Let us consider the likelihood pvu f(x)) for single observation (æ, v):
p(u fx)) fy(x) (()))) C!. fz
= Dir. 1, 2 1,............... ..... 1)(((z)),
position
We use f(z; w) and f(x) interchangeably to denote the latent variable corresponding to the class proba- bility vector the input z. We use fx;; w) emphasize the dependence the weights w.

---

where the left hand side is equal to the right hand side of Eq. (4) up to a scalar multiplier C (normalizing constant of the Dirichlet distribution), which does not depend on f(x). We note that the algebra in Eq. (4) is the same as the algebra used to show the conjugacy of the Dirichlet prior and the categorical likelihood [e.g. 6, Ch. 2.2]. Eq. (4) describes the distribution induced by the observation on the predicted class probabilities for the corresponding input by the likelihood. Consequently, the joint posterior over the class probability vectors f = {f(=)) with the full dataset can then be written as
p(f D) os p(f)
II (2.9)ED
(1.. 1, 2 1, 1)((z)).
(5)
position y
Eq. (5) provides intuition for how BNNs estimate aleatoric uncertainty in classification. we further assume that the implied prior p(f) over f(x:) for each is uniform A and independent across different inputs æ, then for an observation (æ, y) the posterior over f(x) is Dir. Dir.11.... (1..... ...... .. 1) and the posterior mean for the probability of the correct class is (Iff())) 2/(C' + 1). For example, with a dataset containing 100 classes (e.g. CIFAR-100), the model on average will only be less than 2% confident in the correct label on the training data!
In practice, however, the prior p(w) will imply a non-trivial joint distribution p(f) over {f(;; w)EED, so the actual posterior may be more (or less) confident than suggested by the analysis above. Moreover, we have a very limited understanding of the implied distribution p(f) [see 61, 62, for empirical analysis]. In particular, most practitioners use simple p(w) (N a²1) priors, re- gardless of the amount of label noise in the data. Explicitly constructing a prior in the parameter space that leads highly confident predictive distributions over f(x) is challenging, but will show how modifying the likelihood can be viewed as providing an input-dependent prior that more accurately reflects our beliefs about aleatoric uncertainty in Section 4.3.
Eq. (5) suggests two natural ways of modifying the posterior account for the aleatoric uncertainty: (i) increasing the Dirichlet concentration for the observed class (equivalent to likelihood tempering), or (ii) decreasing the concentration for the unobserved classes (noisy Dirichlet model). Next, we describe both of these approaches in detail.
4.2 Likelihood tempering reduces aleatoric uncertainty
The tempered likelihood posterior over the parameters w in Eq. (3) for BNN classification can be written as (w 'D) xw. p(w) (I (2;;; w))//², where T the temperature. Analogous to the derivation above, we can rewrite the corresponding function space posterior as
Premp (1I os p()) II (w.y)D
Dir. (ir.
......1.1+11 1. ..... ()(())))
(6)
position g/
In other words, the tempered posterior corresponds to the same model as the regular posterior, but assumes that observed 1/T counts of class for each input in the training data. Assuming the prior p(w) implies a uniform distribution over f (z; w), the confidence in the correct labels on the train data under the tempered posterior [z)] (1 1/T) + 1/T). For dataset with 100 classes, at T 10-2 we get much higher average confidence of more than 50% on the training data, much higher than the 2% for the standard observation model.
Tempering the Likelihood vs Posterior. Prior work has mostly considered tempering the full Bayesian posterior as opposed just the likelihood (see Section 3). In Appendix G.2 we show that tempering the full Bayesian posterior is almost always equivalent to changing the prior distribution, and tempering the likelihood, and that likelihood tempering recovers the cold posterior effect.
How should we think about tempering? Prior work has asserted that posterior tempering sharply deviates from the Bayesian paradigm and the cold posterior effect is highly problematic (61,48, 14). We, on the other hand, argue that likelihood tempering is in fact a practical way to incorporate our assumptions about aleatoric uncertainty. Relatedly, Aitchison [2] argue that the cold posterior effect can be caused by data curation. For example, CIFAR- 10 and ImageNet datasets have relatively very little label noise and are carefully curated [c.g. 53]. Thus, we may expect that tempered likelihoods with low temperatures will lead to optimal performance.
We use notation &e denote proportionality with respect to 0.
5

---

Is Tempered Likelihood Valid Likelihood? The observation model is a distribution p(y | over the labels conditioned on the input. The tempered softmax likelihood is in general not valid likelihood because it does not sum to 1 over classes. But Wenzel et al. [61] argue that the tempered softmax likelihood with < 1 can be interpreted as valid likelihood if we introduce new class which is not observed in the training data. Moreover, from Eq. (6), we can naturally interpret the tempered likelihood as using the multinomial observation model, assuming 1 1/T counts of the label observed for each of the training datapoints, which is uncontroversial and perfectly valid.
Smoothed softmax model. Guo et al. [19] show that softmax temperature scaling can be used to fix the uncertainty calibration of trained neural networks. Analogously, we can simply divide the logits of the model by scalar temperature T, before computing the softmax likelihood, resulting in the smoothed sofmmax model, valid Bayesian model with a tunable temperature parameter. In Appendix we show that the smoothed softmax behaves differently than the tempered softmax likelihood, and is insufficient to model the aleatoric uncertainty in Bayesian classification. In par- ticular, Figure 10 shows that the smoothed softmax likelihood does not address the cold posterior effect.
4.3 Noisy Dirichlet model: changing the prior over class probabilities
As we have seen in Eq. (6), likelihood tempering increases the posterior confidence by increasing the Dirichlet concentration for the observed class from 2 to 1 + 1/T'. We can achieve a similar effect by decreasing the concentration of the unobserved classes". Indeed, consider the distribution
PND[ 1D) xf p())
II (2,9)ED
(ar...........
Otc. Cc. 1.t t............. (.)(()))
(7)
position y
where ND in PND stands for noisy Dirichlet and a is a tunable parameter. The noisy Dirichlet model was originally proposed by Milios et al. [43] in the context of Gaussian process classification, where they designed a tractable approximation to this model. Using our running example, if p(w) induces uniform distribution over fa; w), for problem with 100 classes and a, 10 have expected confidence [x]] = (a, + 1)/(Ca, + 1), slightly more than 50%, similar to the tempered likelihood with = 10-2.
Going back to the parameter space, we can write the posterior distribution over w corresponding to the noisy Dirichlet model as
(ND(W] D) Oae p(w)
Dir. (aee.... Okey +1, ac. (f) (Z,0)ED
II
(8)
position y
The noisy Dirichlet model corresponds different prior. Using the conjugacy of the Dirichlet and multinomial distributions [e.g., 6, Ch. 2.2.1I, we can rewrite PND from Eq. (8) as follows:
PND(W] D) Xw p(w) II (an. (a.laa....,,.) (f))((( (x;
(2,9)E
(ND(W)
II (x.y)ED
(x;
We can interpret (ND _ p(w) I LEED Dir.(a....., (.) (x; w)) as prior over the parameters w of the model. Indeed, (ND(W) does not depend on the labels and simply forces the predicted class probabilities to be confident in any one of the classes for the training data. Then, the only difference between the noisy Dirichlet model in Eq. (9) and the standard Bayesian neural network for classification the choice of the prior over the parameters: instead of using a generic prior p( w) such as the standard Gaussian distribution, we use the noisy Dirichlet prior IN See the EmpCov prior in Izmailov et al. [30] for another example of a prior that depends on the training inputs but not training labels.
The noisy Dirichlet prior intuitively appealing: in many practical settings priori we believe that the aleatoric uncertainty on the training data is low, and the model should be confident in one of the
'While we discuss increasing the confidence of the model here, we can equivalently decrease the confidence of the model by increasing the concentration parameters for the unobserved classes.

---

classes. At the same time, we would not want to enforce low aleatoric uncertainty everywhere in the input space, as we do not expect that all the possible inputs to the model should be classified with high confidence.
The noisy Dirichlet model is valid Bayesian model. In short, the noisy Dirichlet model corre- sponds to using a valid likelihood - the softmax likelihood - with a prior that explicitly enforces the model predictions be confident on the training data. In Section 6.2 we will see that the noisy Dirichlet model removes the cold posterior effect.
Gaussian approximation. Milios al. [43] considered the distribution in Eq. (9) in the context of Gaussian process classification, but with a different goal: they aimed to create a regression likelihood which would approximate the softmax likelihood. We describe their Gaussian approximation which amounts to solving a regression problem in the space of logits =(x) with a Gaussian observation model in Appendix D. While theoretically we do not need use the Gaussian approximation for
Bayesian neural networks and can directly use Eq. (9), we found that in practice the approximation
much more stable numerically. We use the Gaussian approximation in all experiments.
The Effect of Data Augmentation
Data augmentation is a key ingredient of modern deep learning. Surprisingly, however, data aug- mentation has not been studied in the context of Bayesian methods until recently. Several works have shown that data augmentation often causes the cold posterior effect in BNNs [31. 64, 14, 45]. We show that data augmentation, as applied in practice'. is closely connected to likelihood tempering with T 1. this case, the cold posteriors counterbalance the effect of data augmentation.
First, we consider the interaction between data augmentation and SGLD (or, equivalently, any other SGMCMC method). With SGLD, we aim construct an unbiased estimator of the gradient of the posterior log-density (see Appendix C). In prior work, e.g. in Wenzel et al. [61], the stochastic gradient is estimated using a randomly augmented mini-batch of the data. For a minibatch of the full dataset D,, (T,, Mi J C D, and finite set of augmentations T (4,,.... 'x) }. this stochastic gradient is given by (ū(w)=
m (E.. V w log p(y (z,)) + V.. w log p(w), where the transformations tj are sampled uniformly from T. There two sources of randomness in (Ü(w) the choice of the mini-batch D.,, and the choice of augmentation i, used for each z;. The limiting distribution of SGLD is determined by the expectation of the stochastic gradient above, given by
N
N
K
E2 V w log ((x)(()((( V logp(w) =
V. (I))
+ V log p(w),
(10)
where lj T are augmentations sampled uniformly from T. Therefore, we conclude that the limiting distribution of SGLD under data augmentation is given by
K
Pang (w D) Xen
p(w)
IIII plu: (t((
(11)
We can interpret the limiting distribution in Eq. (1I) as a tempered likelihood posterior for a new dataset D' = ((t)( which contains all augmentations of every data point from the original dataset D. Furthermore, the likelihood is tempered with a temperature K > 1, corresponding to warm posterior. In other words, the limiting distribution of SGLD with data augmentation corresponds to posterior over the augmented dataset with a softened likelihood, leading to less confidence about the labels. By applying a cold temperature T = 1/K to the posterior in Eq. (11), we can recover the standard Bayesian posterior on the augmented dataset D'. In Section 6.2, we explore the effect of data augmentation aleatoric uncertainty in practice and show that it does indeed soften the likelihood and lead to increased aleatoric uncertainty.
$ DDaa augmentation a practical procedure, where at each iteration of optimization or sampling, we draw random augmentation for each object in each mini-batch of the data. Consequently, we can only reason about the effect of the data augmentation on the limiting distributions of approximate inference procedures: data augmentation is not defined for true Bayesian neural networks.
7

---

Augmentations
4 Augmentations
Augmentations
W
M
Train Data
>
-2
1 3
2
0 1
3
Figure 2: Effeec of data augmentation GP regression. As we increase the number K of augmentations the dataset, a GP model fit becomes more diffuse. Here the original training data is shown with black stars, and the augmented datapoints 1 (z) Ty are not shown. Both the predictive mean and confidence change with the number of augmentations.
What about Other Inference Procedures? While the derivation in Eq. (11) comes directly from equivalently derive the same poste-
studying the stochastic gradient evaluated in SGMCMC, rior distribution in stochastic variational inference [26] as we show Appendix I.
Should we always use temperature T 1/K? While using the temperature T = 1/K recovers the standard Bayesian posterior on the augmented dataset D', is not necessarily correct approach to modeling data augmentation. Indeed, the standard posterior p(w D') x p(w) I II p(u:
t, (z;)) assumes independence across both augmentations and data points. In practice, however, we share the same label y/ across all the augmented versions of the image; treating the observations as independent then leads to underestimating the aleatoric uncertainty. Indeed, consider the extreme scenario, where all the augmentations tz()) simply return x. this case, treating the observations (1,(),) u) as independent will simply raise the likelihood to a power of K, while in reality we have not received any additional information from the augmentation policy. Correcting the likelihood by
factor of 1/K /K is the correct approach only when the predictions on (z())) are completely independent from each other (see Section 6.1). See also Nabarro et al. [45] for a related discussion.
Experiments
In this section we provide empirical support for the observations presented in Sections and 5. First, in Section 6.1 we visualize the effect of data augmentation on the limiting distribution of SGMCMC using a Gaussian process regression model. Next, in Section 6.2 we report the results for BNNs on image classification problems. In Appendix we additionally illustrate the effects of tempering, noisy Dirichlet model and data augmentation using a Bayesian neural network on a synthetic 2-D) classification problem. Appendix K provides detailed numerical results for reference. All results report the mean and one standard deviation over 3 trials. The code to reproduce experiments is bayesian--llssiffcaiinn
available at
6.1 Visualizing the Effect of Data Augmentation
To illustrate our analysis in Section 5 we show the posterior in Eq. (11) under data augmentation in a Gaussian process (GP) regression model. We use a GP with RBF kernel, noting that data points where k(x, x²) 2o 0 are effectively independent from each other, which occurs when || - x'1l > 6. Consequently, the implied correlations between inputs beyond the distance are zero. We use augmented samples that are given by (z)= assuming that ((z,t((z)) of the GP on the training data are independent from the outputs on the augmented datapoints.
+ Tj. for vectors {T,} such that ||T > 5, 0 for all datapoints z, x and all augmentations ty. Thus, the outputs
We show the posterior near the original data in Eq. (11) for K = 1,4 and 10 augmentations per datapoint Figure 2. As the predictions on the train data are independent from the predictions on the augmented datapoints, the posterior in Eq. (11) corresponds to tempering the posterior on the original training data with a warm temperature equal to the number K of augmentations. As a result, increasing K softens the likelihood, leading to a less confident fit on the training data.
8

---

6.2 Image Classification with Bayesian Neural Networks
In this section we experimentally verify the results of the analysis in Sections 4 and for Bayesian neural networks in image classification problems. First, we show that both the noisy Dirichlet model and tempered softmax likelihood can be successfully used to express our beliefs about the amount of label noise in the data. Then, we show that the noisy Dirichlet model does not require tempering to achieve optimal performance. Finally, we show that data augmentation softens the likelihood for BNNs, and that the optimal temperature depends on the complexity of the data augmentation policy. For all experiments we use ResNet-18 model [20] and the CIFAR-10 [35] and Tiny Imagenet [36] datasets. We use the SGLD sampler with cyclical learning rate schedule [60, 65] to sample from the posterior. We provide details on the hyper-parameters in Appendix L.
0.9
-4
2.0
X10
Tempored Softmax
Softmax 3)
0.6
Je 0d
K- 63
X30
++-
Temppred Softmax
Softmax (T
Test Accuracy
0.8
0.7
1.55
-+-
Nakey Dirichet
0.4
1=01
Nelry Ditchh et
Tect
0.6
8.22
0.5
0.4
0.
0.2
0.0
0.2
0.6
0.8
0.8
0.2
0.6 0.8
Label
Labe
Label
(a) CIFAR-10
(b) Tiny Imagenet
Figure 3: Label noise in image classification. BMA test accuracy and average sample NLL for the softmax, tempered softmax and noisy Dirichlet model (a) on CIFAR-IO and (b) Tiny Imagenet. Accounting for the label noise via the noisy Dirichlet model or the tempered softmax likelihood significantly improves accuracy accross the board. Moreover, the best performance is achieved by different values of temperature T in the tempered softmax likelihood or noise at in the noisy Dirichlet model, i.c. no one model describes the data optimally across all levels of label noise.
Modeling label noise. In classification problems the aleatoric uncertainty corresponds to the amount of label noise in the data. Throughout the paper we argued that alcatoric uncertainty is misrepresented by standard Bayesian neural networks, and that likelihood tempering and the noisy Dirichlet model are compelling approaches to incorporate information about the amount of label noise. In Figure we show the BMA test accuracy and average sample negative log- lokelikeld (NLL) for the standard softmax likelihood, tempered softmax likelihood and the noisy Dirichlet model on CIFAR-10 and Tiny Imagenet under varying amounts of label noise. We plot the results for the best performing temperature T E [10-5, 10] or noise a, E [10-6, 10 for the noisy Dirich- let model. On both datasets and across label noises, we can significantly improve performance over the softmax likelihood by explicitly modeling the aleatoric uncertainty with either tempering or the noisy Dirichlet model.
No cold posterior effect in the noisy Dirichlet model. We ex- plore the effect of posterior tempering on the standard softmax classification likelihood and the noisy Dirichlet model with noise parameter a, 1 10-6. In Figure we show the results on the CIFAR- 10 dataset. As reported by Wenzel et al. [61], for the stan- dard softmax likelihood, tempering required achieve optimal performance, with 1-10-3 10-3 providing the best results. For the noisy Dirichlet model on the other hand, tempering does not sig- nificantly improve the results with roughly constant performance across the temperature values. In particular, the noisy Dirichlet model achieves near-optimal results T= These results agree with our analysis in Section 4: both tempering and the noisy Dirich- let model are alternative ways of expressing our beliefs about the aleatoric uncertainty in the data; with the noisy Dirichlet model, we can achieve strong results without any need for tempering.
0.94
Test Acquracy
2..9
0.90
0.88
Softmax
0.86
Noisy Dirich
10a
10-2
109
Figure 4: BMA test accuracy for the noisy Dirichlet model with a, =0=10-6 and the soft- max likelihood as function of temperature on CIFAR- 10. The noisy Dirichlet model shows no cold posterior effect.
Data augmentation leads to underfitting on train data. In Fig- ure 5a we show the average sample train NLL for the models trained with and without data augmentation. Across temperatures for the tempered softmax and a, parameters for the noisy Dirichlet model, adding data augmentations reduces the quality of the

---

10"
10
10
0.550
Aug.
Aug
0.810
10 10 10P 10a1
10"4
10
Tempprat..r
Dr
Aughtte
Firs Orga
Flps
Softmax noisy Dirichlet
(b) Augmentations Tempering
Figure 5: Effect of data augmentation on BNN image classification. For all plots we use ResNet- 18 on the CIFAR-I0 dataset. (a): NLL on the train data for the tempered softmax model across different temperatures T and the noisy Dirichlet model across different values of noise at., with and without data augmentation. As predicted in Section 5, data augmentation softens the likelihood, leading to more diffuse fit the train data. Tempering or reducing the noise parameter a, is needed to counteract the effect data augmentation.
(b) BMA test accuracy and NLL for various augmentation policies. As predicted in Section 5, more complex policies corresponding to higher effective number of augmentations require lower temperatures.
fit on the original training data. This result is analogous to the GP results in Section 6.1 and our analysis in Section 5: data augmentation softens the likelihood, leading a more diffuse fit.
Complex augmentations require lower temperatures. In Section 5 we showed that under data augmentation the likelihood effectively tempered with temperature K, assuming the predic- tions on the augmented datapoints are completely independent from the original datapoints. In practice however, the augmentations dictions can be highly correlated. In Figure 5b, we consider five separate types of augmentations for ResNet-18 on CIFAR-10 at various posterior temperatures: horizontal and vertical flips, random crops, combinations of flips and crops, and AugMix [23] - an augmentation policy employing very diverse of transformations. We find that, as predicted by the analysis Section 5, the optimal temperatures are different for different policies: in terms of test NLL, the simple policies (vertical and horizontal flips) corresponding to K 2 work best at warmer temperatures T 1, intermediate policies (crops and crops frips) corresponding to K z 100 require lower temperatures E [10-2, 2, 1. and the most complex Aug policy requires the lowest temperature Z 10-1
be close the original images, and the corresponding pre-
Robustness to corruptions. Finally, we consider robustness to dataset corruptions by considering the CIFAR-10C dataset [22], evaluating our models with both tempered softmax (TS) and noisy Dirichlet models. A full plot is shown in Appendix K; summarizing those results, we see that TS has the best performance T 10-3 with an accuracy of 93.2 (with T 1 again under-performing), while again the noisy Dirichlet model performs well, with an accuracy of 92.6 (at a = 10-4).
7 Discussion
A correct representation of aleatoric uncertainty is crucial to achieve strong performance with Bayesian neural networks. Standard Bayesian classifiers have no explicit mechanism represent our beliefs about alcatoric uncertainty, which can lead to inadequate fits to the training data. This effect exaggerated by data augmentation, which softens the likelihood, making the models under- confident on the training data. Posterior tempering is a simple and practical way to correct for this misspecification and express our beliefs about the aleatoric uncertainty: most benchmark datasets have very low levels of label noise, and we can express this belief by tempering the softmax likeli- hood. We also showed that we can achieve similar effect without tempering, by using a prior that forces the model to be confident on the training datapoints with the noisy Dirichlet model.
Acknowledgements
We would like to thank Micah Goldblum, Wanqian Yang, and anonymous referees for very help- ful comments. This research is supported by an Amazon Research Award, NSF I-DISRE 193471, NIH R01DA048764401A7 A1, NSF 195-19,, and NSF 1922658 NRT-HDR: FUTURE Founda- tions, Translation, and Responsibility for Data Science. This work supported part through the NYU IT High Performance Computing resources, services, and staff expertise.
10

---

References
[1] Ben Adlam, Jasper Snock, and Samuel L Smith. Cold posteriors and alcatoric uncertainty. arXiv preprint arXiv:2008.00029 2020. 3, 22
[2] Laurence Aitchison. A statistical theory of cold posteriors in deep neural networks. In Inter-
national Conference on Learning Representations, 2020. 3, 5, 21, 23
[3] Gregor Bachmann, Lorenzo Noci, and Thomas Hofmann. How tempering fixes data augmen- tation in bayesian neural networks. arXiv preprint 22000:22005.39000, 2022. 3
[4] Lucas Beyer, Olivier J Hénaff, Alexander Kolesnikov, Xiaohua Zhai, and Aäron van den Oord. Are we done with imagenet? arXiv preprint arXiv:2006.077 07159, 2020. 1
[5] Lukas Biewald. Experiment tracking with weights and biases, 2020. URL https://wrrt wandb.com/. Software available from wandh.com. 29
[6] Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006. 6
[7] Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncer- tainty estimation without ood samples via density-based pseudo-counts. Advances in Neural Information Processing Systems, 33:1356-1367, 2020. 3
[8] Tianqi Chen, Emily Fox, and Carlos Guestrin. Stochastic gradient hamiltonian monte carlo. In Eric P. Xing and Tony Jebara, editors. Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 1683-
1691, Bejing, China, 22 24 Jun 2014. PMLR. 18
[9] Mark Collier, Basil Mustafa, Ef Kokiopoulou, Rodolphe Jenatton, and Jesse Berent. A simple probabilistic method for deep classification under input-dependent label noise. arXiv preprint arXiv:2003.06778 2020. 3
[10] Mark Collier, Basil Mustafa, Efi Kokiopoulou, Rodolphe Jenatton, and Jesse Berent. Corre- lated input-dependent label noise in large-scale image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1551-1560, 2021. 3
[11] Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless bayesian deep learning. Advances in Neural Information Processing Systems, 34, 2021.
[12] Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yian Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, and Dustin Tran. Efficient and scalable bayesian neural nets with rank-1 1 factors. International conference on machine learning. pages 2782-2792. 2020. 3
[13] Lei Feng, Senlin Shu, Zhuoyi Lin, Fengmao Lv, Li Li, and Bo An. Can cross entropy loss be robust to label noise? Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 2212, 2021.
[14] Vincent Fortuin, Adrià Garriga-Alonso, Florian Wenzel, Gunnar Rätsch, Richard Turner, Mark van der Wilk, and Laurence Aitchison. Bayesian neural network priors revisited. arXiv preprint arXiv:2 20XKv:2/02.000 06276657, 2021. 2, 3, 5, 7, 17, 22, 23
[15] Jacob R Gardner, Geoff Pleiss, David Bindel, Kilian Q Weinberger, and Andrew Gordon Wil- son. Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. Advances in Neural Information Processing Systems, 2018:7576-7555 -586, 2018. 29
[16] Joyee Ghosh, Yingbo Li, and Robin Mitra. On the use of cauchy prior distributions for bayesian logistic regression. Bayesian Analysis, 13(2):359-383, 2018. 23
[17] Peter Grünwald. The safe bayesian. In International Conference on Algorithmic Learning Theory, pages 169-183. Springer, 2012. 4
[18] Peter Grünwald and Thijs Van Ommen. Inconsistency of bayesian inference for misspecified linear models, and proposal for repairing it. Bayesian Analysis, 12(4):169--113, 2017. 21
[19] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pages 1321-1330. PMIR, 2017. 6, 20
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference computer vision, pages 630-645. Springer, 2016. 9, 29
11

---

[21] Jonathan Heek and Nal Kalchbrenner. Bayesian inference for large scale image classification. arXiv preprint arXiv: 1908.9908.000 03491, 2019. 22
[22] Dan Hendryeks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv: 1903. /2261, 2019. 10, 27
[23] Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint 1rXivv1912.027811 2019. 10
[24] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network.
arXiv preprint arXiv: 503.02533 2015. 20
[25] Geoffrey E Hinton and Drew Van Camp. Keeping the neural networks simple by minimiz- ing the description length of the weights. In Proceedings of the sixth annual conference Computational learning theory, pages 5-13, 1993.
[26] Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational inference. Journal of Machine Learning Research, 2013. 8
[27] Stephen C. Hora. Aleatory and epistemic uncertainty in probability elicitation with example from hazardous waste management. Reliability Engineering & System Safety, 54:217-223, 1996.
[28] W Ronny Huang, Zeyad Emam, Micah Goldblum, Liam Fowl, Justin K Terry, Furong Huang. and Tom Goldstein. Understanding generalization through visualizations. 2020. 20
[29] Alexander Immer, Matthias Bauer. Vincent Fortuin, Gunnar Rätsch, and Khan Mohammad Emtiyaz. Scalable marginal likelihood estimation for model selection in deep learning. In International Conference on Machine Learning, pages 4563-4573. PMLR, 2021. 25
[30] Pavel Izmailov, Patrick Nicholson, Sanae Lotfi, and Andrew Gordon Wilson. Dangers of bayesian model averaging under covariate shift. arXiv preprint arXiv:2 / 1905, 2021. 6,
17, 18, 23
[31] Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Wilson. What are bayesian neural network posteriors really like? arXiv preprint vxXiv:2104114221, 2021. 2, 7, 16. 17, 20, 24.30
[32] Taejong Joo, Uijung Chung, and Minji Seo. Being bayesian about categorical probability, In ICML, 2020. 3
[33] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In NIPS, 2017. 1, 16
[34] Anna-Kathrin Kopetzki, Bertrand Charpentier, Daniel Zügner, Sandhya Giri, and Stephan Günnemann. Evaluating robustness of predictive uncertainty estimation: Are dirichlet-based models reliable? In International ConeffrrtnncCCmfeeeenee Machine Learning, pages 5707-5718. PMLR, 2021.
[35] Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. 9, 14
[36] Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge. 2015. 9, 14
[37] David J. C. MacKay. A practical bayesian framework for backpropagation networks. Neural Computation, 4:448-472, 1992. 2
[38] Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. Advances in Neural Information Processing Systems, 32, 2019.
[39] Wesley J Maddox, Gregory Benton, and Andrew Gordon Wilson. Rethinking parameter count- ing in deep models: Effective dimensionality revisited. arXiv preprint arXiv:2003.02139, 2020. 19, 29
[40] Andrey Malinin and Mark Gales. Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness. Advances in Neural Information Processing Systems, 32, 2019.
[41] Andrey Malinin and Mark John Francis Gales. Predictive uncertainty estimation via prior networks. In NeurlPS, 2018. 1,3
12

---

[42] Andrey Malinin, Bruno Mlodozeniec, and Mark Gales. Ensemble distribution distillation. In International Conference on Learning Representations, 2020. URL https://openrerier. net/forum? id=ByggPeVtvr. 3
[43] Dimitrios Milios, Raffaello Camoriano, Pietro Michiardi, Lorenzo Rosasco, and Maurizio Filippone. Dirichlet-based gaussian processes for large-scale calibrated classification. In NeurlPS, 2018. 3, 6, 7, 16, 18
[44] Kevin P. Murphy. Probabilistic Machine Learning: Advanced Topics. MIT Press, 2023. URL
probml. ai. 4
[45] Seth Nabarro, Stoil Ganev, Adrià Garriga-Alono, Vincent Fortuin, Mark van der Wilk, and Laurence Aitchison. Data augmentation in bayesian neural networks and the cold posterior effect. arXiv preprint wwxxiv22/06005566 2021. 3, 7, 8, 17
[46] Jay Nandy, Wynne Hsu, and Mong Li Lee. Towards maximizing the representation gap be- tween in-domain & out-of-distribution examples. Advances in Neural Information Processing Systems, 33:9239-9250, 2020. 3
[47] Radford M. Neal. Bayesian learning for neural networks. 1995.
[48] Lorenzo Noci, Kevin Roth, Gregor Bachmann, Sebastian Nowozin, and Thomas Hofmann. Disentangling the roles of curation, data-augmentation and the prior in the cold posterior effect.
ArXiv, abs/2106.06596, 2021. 3, 5, 17
[49] Kazuki Osawa, Siddharth Swaroop. Mohammad Emtiyaz E. Khan, Anirudh Jain, Runa Eschen- hagen, Richard E Turner, and Rio Yokota. Practical deep learning with bayesian principles. Advances in neural information processing systems, 32, 2019. 3
[50] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, ierf-peeformance deep learning library. In Advances in neural information processing systems, 2019. 29
[51] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit- learn: Machine learning in python. the Journal of machine Learning research, 12:2825-2830, 2011. 29
[52] John Platt et al. Probabilistic outputs for support vector machines and comparisons to regular- ized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999. 20
[53] Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinberger. Identifying mislabeled data using the area under the margin ranking. Advances in Neural Information Processing Systems, 13:17044-17056 17056, 2020. 5
[54] Carl Edward Rasmussen and Christopher K Williams. Gaussian processes for machine learn- ing, volume 2. MIT press Cambridge, MA, 2006. 1,4
[55] Simo Särkkä and A. Solin. Applied stochastic differential equations. 2019. 18
[56] Robin Senge, Stefan Bösner, Krzysztof Dembczynski. Jörg Haasenritter, Oliver Hirsch, Nor- bert Banzhoff, and Eyke Hüllermeier. Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty. Inf. Sci., 255:16-29 2014. 1
[57] Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify clas- sification uncertainty. Advances in neural information processing systems, 31, 2018. 3
[58] Murat Sensoy, Lance Kaplan, Federico Cerutti, and Maryam Saleki. Uncertainty-awan deep classifiers using generative models. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 34, pages 5620-5627, 2020. 3
[59] Weishi Shi, Xujiang Zhao, Feng Chen, and Qi Yu. Multifaceted uncertainty estimation for label-efficient deep learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 17247- 17257. Curran Associates, Inc., 2020. URL https: http:/eeeddinngn..uu..... cc/ppprr/ .df.
[60] Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-II), pages
681-688, Citeseer, 2011. 4, 9,
13

---

[61] Florian Wenzel, Kevin Roth, Bastiaan Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the bayes posterior in deep neural networks really? In International Conference on Machine Learning, pages 10248-10259. PMIR, 2020. 2, 3, 4, 5, 6, 7, 9, 17, 20, 22, 24,
[62] Andrew Gordon Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic per- spective of generalization. arXiv preprint arXiv:2002.
2020. 1, 3, 4, 5, 17, 24
[63] Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P Xing. Deep kernel learning. In Artificial intelligence and statistics, 2016.
[64] Chen Zeno, Itay Golan, Ari Pakman, and Daniel Soudry. Why cold posterions? the subop- timal generalization of optimal bayes estimates. In Third Symposium on Advances in Approxi- mate Bayesian Inference, 2020. 3, 7.
[65] Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, and Andrew Gordon Wilson, Cycli- cal stochastic gradient MCMC for bayesian deep learning. arXiv preprint arXiv:/902.0::: 2019. 3, 18. 22
[66] Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural net- works with noisy labels. Advances neural information processing systems, 31, 2018.
[67] Xujiang Zhao, Feng Chen, Shu Hu, and Jin-Hee Cho. Uncertainty aware semi-supervised learning on graph data. Advances in Neural Information Processing Systems, 33: 12827-1283,, 2020.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Appendix A.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Ap- pendix A.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Eq. (10), Appendices G and I.
(b) Did you include complete proofs of all theoretical results? [Yes] See Eq. (10), Ap- pendices G and
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL.? [Yes]
(b) Did you specify all the training details (e.g., data splits, byperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] Unless otherwise stated, mean and standard deviation
over 3 trials.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Appendix L.
4. If you using existing assets (e-g., code, data, models) curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] CIFAR-10 [35], Tinylmagenet [36].
(b) Did you mention the license of the assets? [Yes] CIFAR-10, MIT License, Tinylma- genet, MIT License [36]
(c) Did you include any new assets either in the supplemental material or as URL.? (Yss]
14

---

(d) Did you discuss whether and how consent was obtained from people whose data you re using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifi- able information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "YQ5DVvaeKy1", "reviewer": "Reviewer_JCLG", "summary": "This work closely examines aleatoric uncertainty in Bayesian neural networks for classification. It demonstrates how tempered softmax (cross-entropy likelihood), standard softmax and their proposed method, the Dirichlet observation model, behave under varying degrees of aleatoric uncertainties. They show that via the Dirichlet model or by tempering the likelihood function, the prior belief about aleatoric uncertainty can be expressed in BNNs for classification. In this way, the authors provide an important understanding of why BNNs fit poorly to the training data when using data augmentation, as well as tools for addressing the issue. ", "questions": "Regarding the results in Fig. 3. In lines 305-306: \"We plot the results for the best performing ... $T$ ... or ... $\\alpha_\\epsilon$...\". To me, the effect of varying these values wrt the results in Fig. 3 is clearly missing. It would help the community and/or practitioners a lot to know more about the robustness of these parameters wrt to the label noise. E.g. for CIFAR's BMA test accuracy, $\\alpha_\\epsilon=1e-04$ is the best performing hyperparam when label noise = 0.0, while $\\alpha_\\epsilon=1e-02$ is the way to go when labe noise = 0.2. This is a big jump. \n\n1. What happened at label noise = 0.2 when $\\alpha_\\epsilon \\in [1e-04, 1e-02)$? Or for larger values? If you tested $T\\in[10^-5, 10]$ and $\\alpha_\\epsilon\\in[10^-6,10^-1]$, why not include these? A clarification of this would be very helpful and interesting.\n\n2. Smaller comment on Fig. 3. What noise values and temperatures did you use to produce the intermediate points? Such as label noise = 0.5 in Fig. 3a? Please explicitize. \n\nRegarding the paragraph \"Data augmentation leads to underfitting on train data.\" \n\n3. It seems unsurprising that a model that does not use data augmention is more overfitted to the train data than one that uses no augmentation. Can't the underfitting on the training data in part be explained by data augmentation giving more samples for the model to fit? In other words, to which extent would you say that the reduced quality of the fit on the original training data is due to the increased cardinality of the augmented dataset ($\\mathcal{D}'$), and not the softened likelihood? \n\n", "limitations": "Yes; found in the appendix. Above I have addressed a small concern about the transparency of the robustness of the tempered softmax and Dirichlet model wrt their parameters. Constructive suggestions for improvement are given above.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "***Strengths***\n\nThe paper is exemplarily **clearly** written and is easy to follow. The useful explanation of how to combine tempering with data augmention between lines 246-256 is an example of the pedagogical nature of the work. \n\nExperiments are conducted in order to back up claims in the text. For instance, the augmentation vs. tempering experiment neatly supports the ideas in the paragraph between lines 246-256. Also, I find the paper to be technically sound. Hence the **quality** is good. I have some thoughts on improvement outlined below and in the \"Questions\" field.\n\nThe Dirichlet observation model appears **original**, but especially the insights regarding the effect of data augmentation in BNNs are novel and should be impactful. The related work seems OK and carefully written.\n\n***Weaknesses***\n\n**Quality:** Limitations of the proposed method and the tempered softmax can be more elaborated in Fig. 3. See Questions for more explicit feedback.\n\n**Clarity:** You are referencing Fig. 4 (line 287) before you reference Fig. 3. (line 303). You should be able to switch the order of the two corresponding paragraphs as they seem to be the same length.\n\nIn the current form of the work, I find the use of the word \"honest\" when referring to prior beliefs about aleatoric uncertainty slightly awkward as I am unsure what is meant by it. I suggest to add an explanation, for instance in lines 8-11 \"... misrepresenting our honest beliefs about aleatoric uncertainty, as [insert explanation of how the honest belief is misrepresented]\" or elsewhere. Alternatively the term could be dropped as it is not being used much in the later parts of the work.\n\nTypos: \n* line 9: we show [that] data augmentation\n* missing punctuation in bold-fonted title in line 328.\n* the sentence in lines 185-187 could be less tediously written. For instance \"However, Wenzel et al. [51] show, the tempered\" --> \"However, Wenzel et al. [51] show that the tempered\".\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "RApvfNHnIkV", "reviewer": "Reviewer_fpwD", "summary": "The authors discuss the relationship between tempering and data augmentation and build a Dirichlet distribution-based approach that allows them to explain this relationship and how to use it in a more principled manner.\n", "questions": "- Can the authors comment on why the finding on the softened likelihood is 'counterintuitive' (l57)? I would have assumed that data augmentation, giving us essentially more noisy covariates, would unsurprisingly lead to a less confident model. ", "limitations": "Limitations and broader impact have been discussed. However, only in the appendix.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- The task is well motivated and discussed.\n- The proposed solution gives a clear and structured approach to the relationship between tempering and augmentation.\n\n## Weaknesses\n- The writing has several weaknesses, not in the style, where it is well written, but rather in the formulations and presentation which make common knowledge sound too special and new. E.g. l28 makes it sound, especially with the typographical emphasis on 'accuracy' as if the dependency of the accuracy on the aleatoric uncertainty were a surprise. If we tell our models not to trust the data too much, i.e., assume high aleatoric uncertainty, then of course this will lead the GP towards a smoother fit, changing its predictions as well. The same goes for the scenario in Figure 1(c). If we know that we have different assumptions about noise in the data and therefore the smoothness of our decision boundary we would and should incorporate that in our BNN by stronger/weaker regularization instead of using the same likelihood and prior. \nAnother example is the paragraph around l150. Here again, the finding is accompanied by an emphasis on 'training data'. But if we have a strong prior with a weak probability of observing any specific class we will still have a rather weak posterior after only a single observation. This is not a surprising statement worth an exclamation mark. Repeated around l170, where a different prior gives us different confidence in the posterior. Similarly to the $\\alpha_\\epsilon$ approach around eq (5), this is not a surprise, but simply the statement that our prior influences our posterior.\nNote: I do see why the authors use these examples and similar parts as building blocks to develop their storyline. They would just read a lot better if presented less as surprises and more as _here are some well-known facts about our Bayesian models. Now let us tell you, dear reader, how they motivate the approach we propose_. \n- A major weakness is the complete lack of any discussion on prior work using Dirichlet distributions as part of a deep (Bayesian) classification model. Malinin et al. (2018) are cited, but only with respect to the uncertainty decomposition. A discussion of this work and follow-up work (Malinin et al., 2019) is missing. Similarly, Sensoy et al. (2018), Charpentier et al. (2020), etc. all tackle the deficiencies of relying on a simple softmax-based categorical likelihood by introducing Dirichlet priors in different ways.\n\n\n## Other\n- Fig 2: Please show the full training data, not just the unaugmented one either here or in the appendix, for the reader to get a clear picture of what data the model actually sees during training.\n- l253: Technically we have not only not received new information in _our_ reality. In the model's reality, it has received new information through these repetitions of the same observations, leading to the behaviour you correctly describe. \n- Figure 3 and Fig 5 (a) lack the error bars of the remaining plots. \n- Similarly, the number of runs the error bars in the experiments correspond to are currently only available in the checklist and the appendix. They should be mentioned somewhere in the main paper as well (beginning of Sec 6 and/or figure caption).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GJuVZyRmH_W", "reviewer": "Reviewer_1PhB", "summary": "This work investigates the impact of explicitly modeling aleatoric uncertainty in Bayesian Neural Networks (BNNs). They demonstrate empirically and theoretically that it equates to tempering the likelihood function and link it to the recently observed cold posterior effect under data augmentation. Moreover, this work proposes to model aleatoric uncertainty by parameterizing the models output using a noisy Dirichtlet distribution in order to eradicate the cold posterior effect for data augmentation in BNNs.", "questions": "- Can the authors comment on why they equate input-agnostic tempering of the likelihood with modeling aleatoric uncertainty appropriately? I would expect that aleatoric uncertainty is an input dependent quantity.\n- Can the authors comment on W1. I would consider voting for acceptance if this concern is adequately addressed.\n- The effect observed in this work seems to be connected to the more general phenomenon that NNs are poorly calibrated (over/under confident) [1]. Do the authors think that the observation for BNNs (i.e. cold posterior effect for data augmentation) is simply a symptom of this more general problem?\n", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The theoretical link between data augmentation and the cold posterior effect is very interesting, although entirely this papers novelty\n- The effect of modeling aleatoric uncertainty on BNNs is an underexplored area\n - This work provides an empirical investigation of great depth\n\nWeaknesses:\n- W1 I am not entirely convinced by the finding that the noisy Dirichlet parameterization eliminates the necessity for tempering the likelihood (Fig. 4). If I am not mistaken the noise parameter in the noisy Dirichlet parameterization is equivalent to changing the temperature of the resulting model. Thus, for a fairer comparison in Fig. 4 the authors would have to plot the test accuracy of their proposed method over a larger range of Ts. With the given range of Ts it cannot be judged whether the drawn conclusions are true. In particular, these experiments are conducted on CIFAR10 which does not contain aleatoric uncertainty as mentioned by the authors. Thus by setting the noise parameter very small the authors implicitly offset their temperature towards lower values.\n- I think the usage of the term aleatoric uncertainty in this work is confusing. The general trend seems to be that aleatoric uncertainty is a quantity that depends on the input data which seems to be not the case in this work.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DlC9A8rNCgk", "reviewer": "Reviewer_stWW", "summary": "The authors propose a novel strategy for representing aleatoric uncertainty in classification problems. They connect their method to the use of cold posteriors and data augmentation, and empirically demonstrate that it provides an effective alternative to tempering.", "questions": "Where does the representation of aleatoric uncertainty come from? Is there a well-defined posterior over some quantity that corresponds to aleatoric uncertainty?\n\nHow do you see the relationship between this work and the existing literature on Dirichlet-based uncertainty for modeling aleatoric and epistemic uncertainty jointly (see e.g. Kopetzki, …, Gunnemann, Evaluating Robustness of Predictive Uncertainty: Are Dirichlet-Based Models Reliable?, 2020 and the work cited therein in the third paragraph)", "limitations": "It’s unclear what effects the method has on uncertainty quantification, a key use of Bayesian inference in general.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper offers a nice, clearly-explained overview of the cold posterior effect and its possible interpretations. It also offers a nice explanation of its relationship to data augmentation.\n\nThe method itself, however, I cannot understand as a Bayesian procedure, a serious weakness in a paper that seems to aim to provide a “valid” Bayesian method for representing uncertainty. In particular, in the proposed method, the normalizing constant of the likelihood p(y | f(x)) will in general depend on f(x) (since sum_{y = 1}^C p(y | f(x)) does not necessarily equal 1 when using the likelihoods in equations 4 and 5). Therefore, the proportionality given in line 164 — the central justification for the method, as a tempered likelihood approach — does not seem to hold. (Note that in general, the reason why tempered likelihoods are difficult to justify from a purely Bayesian perspective is because they directly imply that the likelihood does not sum to 1; ideas such as coarsening or adding an unobserved class are used to get around this problem).\n\nAlso, the use of proportionality notation everywhere is generally very confusing. I’d appreciate a clear statement of the generative process behind the model somewhere, including its prior and likelihood. Lines 135-139 are extremely confusing as they make it seem as though a Dirichlet prior is being introduced into the model, when in fact the authors are only constructing a likelihood function in a particular form (if I understand correctly). \n\n---- UPDATE ----\n\nThe authors have explained their proposed method more clearly in our discussion. With the understanding that they will rewrite the section of the paper introducing the proposed method to increase the clarity substantially, and that they will add discussion on (a) the softmax temperature prior approach and (b) previous work on modeling aleatoric uncertainty in classifiers with Dirichlet distributions, I am happy to raise my score. I sincerely thank the authors for their committed engagement, and am optimistic that once explained concisely and clearly the ideas will be a valuable contribution to the literature.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept