Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a novel strategy for representing aleatoric uncertainty in classification problems. They connect their method to the use of cold posteriors and data augmentation, and empirically demonstrate that it provides an effective alternative to tempering.
strengths_and_weaknesses: The paper offers a nice, clearly-explained overview of the cold posterior effect and its possible interpretations. It also offers a nice explanation of its relationship to data augmentation.

The method itself, however, I cannot understand as a Bayesian procedure, a serious weakness in a paper that seems to aim to provide a “valid” Bayesian method for representing uncertainty. In particular, in the proposed method, the normalizing constant of the likelihood p(y | f(x)) will in general depend on f(x) (since sum_{y = 1}^C p(y | f(x)) does not necessarily equal 1 when using the likelihoods in equations 4 and 5). Therefore, the proportionality given in line 164 — the central justification for the method, as a tempered likelihood approach — does not seem to hold. (Note that in general, the reason why tempered likelihoods are difficult to justify from a purely Bayesian perspective is because they directly imply that the likelihood does not sum to 1; ideas such as coarsening or adding an unobserved class are used to get around this problem).

Also, the use of proportionality notation everywhere is generally very confusing. I’d appreciate a clear statement of the generative process behind the model somewhere, including its prior and likelihood. Lines 135-139 are extremely confusing as they make it seem as though a Dirichlet prior is being introduced into the model, when in fact the authors are only constructing a likelihood function in a particular form (if I understand correctly). 

---- UPDATE ----

The authors have explained their proposed method more clearly in our discussion. With the understanding that they will rewrite the section of the paper introducing the proposed method to increase the clarity substantially, and that they will add discussion on (a) the softmax temperature prior approach and (b) previous work on modeling aleatoric uncertainty in classifiers with Dirichlet distributions, I am happy to raise my score. I sincerely thank the authors for their committed engagement, and am optimistic that once explained concisely and clearly the ideas will be a valuable contribution to the literature.
questions: Where does the representation of aleatoric uncertainty come from? Is there a well-defined posterior over some quantity that corresponds to aleatoric uncertainty?

How do you see the relationship between this work and the existing literature on Dirichlet-based uncertainty for modeling aleatoric and epistemic uncertainty jointly (see e.g. Kopetzki, …, Gunnemann, Evaluating Robustness of Predictive Uncertainty: Are Dirichlet-Based Models Reliable?, 2020 and the work cited therein in the third paragraph)
limitations: It’s unclear what effects the method has on uncertainty quantification, a key use of Bayesian inference in general.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work closely examines aleatoric uncertainty in Bayesian neural networks for classification. It demonstrates how tempered softmax (cross-entropy likelihood), standard softmax and their proposed method, the Dirichlet observation model, behave under varying degrees of aleatoric uncertainties. They show that via the Dirichlet model or by tempering the likelihood function, the prior belief about aleatoric uncertainty can be expressed in BNNs for classification. In this way, the authors provide an important understanding of why BNNs fit poorly to the training data when using data augmentation, as well as tools for addressing the issue. 
strengths_and_weaknesses: ***Strengths***

The paper is exemplarily **clearly** written and is easy to follow. The useful explanation of how to combine tempering with data augmention between lines 246-256 is an example of the pedagogical nature of the work. 

Experiments are conducted in order to back up claims in the text. For instance, the augmentation vs. tempering experiment neatly supports the ideas in the paragraph between lines 246-256. Also, I find the paper to be technically sound. Hence the **quality** is good. I have some thoughts on improvement outlined below and in the "Questions" field.

The Dirichlet observation model appears **original**, but especially the insights regarding the effect of data augmentation in BNNs are novel and should be impactful. The related work seems OK and carefully written.

***Weaknesses***

**Quality:** Limitations of the proposed method and the tempered softmax can be more elaborated in Fig. 3. See Questions for more explicit feedback.

**Clarity:** You are referencing Fig. 4 (line 287) before you reference Fig. 3. (line 303). You should be able to switch the order of the two corresponding paragraphs as they seem to be the same length.

In the current form of the work, I find the use of the word "honest" when referring to prior beliefs about aleatoric uncertainty slightly awkward as I am unsure what is meant by it. I suggest to add an explanation, for instance in lines 8-11 "... misrepresenting our honest beliefs about aleatoric uncertainty, as [insert explanation of how the honest belief is misrepresented]" or elsewhere. Alternatively the term could be dropped as it is not being used much in the later parts of the work.

Typos: 
* line 9: we show [that] data augmentation
* missing punctuation in bold-fonted title in line 328.
* the sentence in lines 185-187 could be less tediously written. For instance "However, Wenzel et al. [51] show, the tempered" --> "However, Wenzel et al. [51] show that the tempered".

questions: Regarding the results in Fig. 3. In lines 305-306: "We plot the results for the best performing ... $T$ ... or ... $\alpha_\epsilon$...". To me, the effect of varying these values wrt the results in Fig. 3 is clearly missing. It would help the community and/or practitioners a lot to know more about the robustness of these parameters wrt to the label noise. E.g. for CIFAR's BMA test accuracy, $\alpha_\epsilon=1e-04$ is the best performing hyperparam when label noise = 0.0, while $\alpha_\epsilon=1e-02$ is the way to go when labe noise = 0.2. This is a big jump. 

1. What happened at label noise = 0.2 when $\alpha_\epsilon \in [1e-04, 1e-02)$? Or for larger values? If you tested $T\in[10^-5, 10]$ and $\alpha_\epsilon\in[10^-6,10^-1]$, why not include these? A clarification of this would be very helpful and interesting.

2. Smaller comment on Fig. 3. What noise values and temperatures did you use to produce the intermediate points? Such as label noise = 0.5 in Fig. 3a? Please explicitize. 

Regarding the paragraph "Data augmentation leads to underfitting on train data." 

3. It seems unsurprising that a model that does not use data augmention is more overfitted to the train data than one that uses no augmentation. Can't the underfitting on the training data in part be explained by data augmentation giving more samples for the model to fit? In other words, to which extent would you say that the reduced quality of the fit on the original training data is due to the increased cardinality of the augmented dataset ($\mathcal{D}'$), and not the softened likelihood? 


limitations: Yes; found in the appendix. Above I have addressed a small concern about the transparency of the robustness of the tempered softmax and Dirichlet model wrt their parameters. Constructive suggestions for improvement are given above.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work investigates the impact of explicitly modeling aleatoric uncertainty in Bayesian Neural Networks (BNNs). They demonstrate empirically and theoretically that it equates to tempering the likelihood function and link it to the recently observed cold posterior effect under data augmentation. Moreover, this work proposes to model aleatoric uncertainty by parameterizing the models output using a noisy Dirichtlet distribution in order to eradicate the cold posterior effect for data augmentation in BNNs.
strengths_and_weaknesses: Strengths:
- The theoretical link between data augmentation and the cold posterior effect is very interesting, although entirely this papers novelty
- The effect of modeling aleatoric uncertainty on BNNs is an underexplored area
 - This work provides an empirical investigation of great depth

Weaknesses:
- W1 I am not entirely convinced by the finding that the noisy Dirichlet parameterization eliminates the necessity for tempering the likelihood (Fig. 4). If I am not mistaken the noise parameter in the noisy Dirichlet parameterization is equivalent to changing the temperature of the resulting model. Thus, for a fairer comparison in Fig. 4 the authors would have to plot the test accuracy of their proposed method over a larger range of Ts. With the given range of Ts it cannot be judged whether the drawn conclusions are true. In particular, these experiments are conducted on CIFAR10 which does not contain aleatoric uncertainty as mentioned by the authors. Thus by setting the noise parameter very small the authors implicitly offset their temperature towards lower values.
- I think the usage of the term aleatoric uncertainty in this work is confusing. The general trend seems to be that aleatoric uncertainty is a quantity that depends on the input data which seems to be not the case in this work.

questions: - Can the authors comment on why they equate input-agnostic tempering of the likelihood with modeling aleatoric uncertainty appropriately? I would expect that aleatoric uncertainty is an input dependent quantity.
- Can the authors comment on W1. I would consider voting for acceptance if this concern is adequately addressed.
- The effect observed in this work seems to be connected to the more general phenomenon that NNs are poorly calibrated (over/under confident) [1]. Do the authors think that the observation for BNNs (i.e. cold posterior effect for data augmentation) is simply a symptom of this more general problem?

limitations: N/A
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors discuss the relationship between tempering and data augmentation and build a Dirichlet distribution-based approach that allows them to explain this relationship and how to use it in a more principled manner.

strengths_and_weaknesses: ## Strengths
- The task is well motivated and discussed.
- The proposed solution gives a clear and structured approach to the relationship between tempering and augmentation.

## Weaknesses
- The writing has several weaknesses, not in the style, where it is well written, but rather in the formulations and presentation which make common knowledge sound too special and new. E.g. l28 makes it sound, especially with the typographical emphasis on 'accuracy' as if the dependency of the accuracy on the aleatoric uncertainty were a surprise. If we tell our models not to trust the data too much, i.e., assume high aleatoric uncertainty, then of course this will lead the GP towards a smoother fit, changing its predictions as well. The same goes for the scenario in Figure 1(c). If we know that we have different assumptions about noise in the data and therefore the smoothness of our decision boundary we would and should incorporate that in our BNN by stronger/weaker regularization instead of using the same likelihood and prior. 
Another example is the paragraph around l150. Here again, the finding is accompanied by an emphasis on 'training data'. But if we have a strong prior with a weak probability of observing any specific class we will still have a rather weak posterior after only a single observation. This is not a surprising statement worth an exclamation mark. Repeated around l170, where a different prior gives us different confidence in the posterior. Similarly to the $\alpha_\epsilon$ approach around eq (5), this is not a surprise, but simply the statement that our prior influences our posterior.
Note: I do see why the authors use these examples and similar parts as building blocks to develop their storyline. They would just read a lot better if presented less as surprises and more as _here are some well-known facts about our Bayesian models. Now let us tell you, dear reader, how they motivate the approach we propose_. 
- A major weakness is the complete lack of any discussion on prior work using Dirichlet distributions as part of a deep (Bayesian) classification model. Malinin et al. (2018) are cited, but only with respect to the uncertainty decomposition. A discussion of this work and follow-up work (Malinin et al., 2019) is missing. Similarly, Sensoy et al. (2018), Charpentier et al. (2020), etc. all tackle the deficiencies of relying on a simple softmax-based categorical likelihood by introducing Dirichlet priors in different ways.


## Other
- Fig 2: Please show the full training data, not just the unaugmented one either here or in the appendix, for the reader to get a clear picture of what data the model actually sees during training.
- l253: Technically we have not only not received new information in _our_ reality. In the model's reality, it has received new information through these repetitions of the same observations, leading to the behaviour you correctly describe. 
- Figure 3 and Fig 5 (a) lack the error bars of the remaining plots. 
- Similarly, the number of runs the error bars in the experiments correspond to are currently only available in the checklist and the appendix. They should be mentioned somewhere in the main paper as well (beginning of Sec 6 and/or figure caption).
questions: - Can the authors comment on why the finding on the softened likelihood is 'counterintuitive' (l57)? I would have assumed that data augmentation, giving us essentially more noisy covariates, would unsurprisingly lead to a less confident model. 
limitations: Limitations and broader impact have been discussed. However, only in the appendix.
review_title: null

Meta-review: metareview: This paper studies Bayesian neural networks and tempered posteriors and study the link between data augmentation and cold posteriors. Overall, after the rebuttal and discussion, all reviewers unanimously agreed to accept the paper. Reviewers appreciated the exploration of the link between data augmentation and the cold posterior effect and found the empirical results to be convincing. 

One reviewer had concerns about the method but after discussion was convinced to accept the paper provided that the paper will be rewritten to increase clarity and that "they will add discussion on (a) the softmax temperature prior approach and (b) previous work on modeling aleatoric uncertainty in classifiers with Dirichlet distributions." Another reviewer stated that the discussion of limitations should be expanded in the paper. Please revise the paper to address the remaining concerns of the reviewers in the final version.
confidence: Less certain
Decision: Accept