Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper analysises the loss minimisation in second-order learner for epistemic uncertainty quantification. It shows the proposed loss functions do not incentivise the learner to represent the epistemic uncertainty in a faithful way.  
strengths_and_weaknesses: The arguments are clear and richful. 
questions: Are there any potential solutions to address the gap?
limitations: n/a
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper demonstrates the problem of current approaches while modelling epistemic uncertainty by minimising a given loss function.

The discussion is built upon two common approaches to training a model with epistemic (level-2) uncertainty: (1) minimising averaged prediction loss with random samples from the epistemic uncertainty model. (2) the Bayesian approach, where a given prior distribution controls the epistemic uncertainty.

The authors proposed two requirements for the desired loss function for epistemic (level-2) uncertainty and then theoretically demonstrated that both (1) and (2) above cannot meet the requirements under some circumstances.

strengths_and_weaknesses: (Originality): While modelling aleatoric and epistemic uncertainties has gained much interest in recent years, to the reviewer's knowledge, this paper is the first work demonstrating the problems on objective functions of related modelling approaches. In particular, the authors gave a systematic view of the problems that arise from both Bayesian and non-Bayesian treatments.

(Significance): This paper provides a sounding argument for major approaches that claim to offer reasonable epistemic estimates. Therefore the reviewer expects this paper to change the view of the uncertainty quantification community. The message is relatively straightforward: (1) the averaged (proper) loss minimisation shouldn't be used for epistemic uncertainty. (2) the prior / regularisation setting might also lead to inappropriate epistemic uncertainty for a Bayesian setting.     

(Quality, Clarity): This paper comes with systematic mathematic notations and formalisations. Readers with similar research interests should have no issues following the discussion. The technical statement is motivated by clear definitions as well as working examples.

On the weaknesses:

(Quality) Given that the paper is pointing out the problem of existing methods, it is understandable that the paper doesn't come with a pack of numerical experiments. However, it would still benefit the readers with a few demonstrated experiments. Even if there is no ground truth for the level-2 uncertainty, both requirements in definition-1 should be able to be examined with some synthetic data distributions. Therefore some demonstrative experiments with different data sizes and loss functions should help the community to receive the paper better. 
questions: While the discussion on the (Average Level-1 Loss) is clear, the reviewer found the debate on (Bayesian Losses and Regulations) requires an additional assumption that the uniform posterior gives the maximal uncertainty. I wonder if the authors can comment if such an assumption is too restrictive? 

(1) In a regression setting where the posterior has infinite support, it might be hard to come up with such a uniform distribution without additional transformation. 

(2) Also, for a highly unbalanced classification problem, the uniform distribution might not be the case for maximal uncertainty.


========================================

The reviewer found the additional examples and discussion from the authors informative and provided some further justification for the paper. This paper should lead to some nice discussion at the conference.
limitations: Yes, the author discussed the limitations at the end of the paper.

Following the comments above, I wonder if the authors could also discuss that they are mainly working on the classification scenario where a proper scoring rule is defined, particularly for the Bayesian setting.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper analyzes the recent line of work on uncertainty estimation in neural networks which model distributions over distributions by loss minimization. The authors argue that this paradigm of learning higher order distributions with loss minimization does not lead to meaningful estimates of epistemic uncertainty. The authors support their arguments with theoretical results showing that predictors learned with loss minimization.
strengths_and_weaknesses: **Strengths**
* I believe the paper is quite relevant to the community as it examines a popular class of methods for uncertainty estimation. The theoretical results are an important (albeit limited in scope) contribution to the literature. 
* The paper is generally well written. The authors convey their arguments systematically supplemented with enough running examples to follow the argument. The paper is also relatively self-sufficient with all the relevant quantities defined properly. The results and accompanying assumptions are stated clearly. 

**Weaknesses**
* The analysis is limited in scope, applying only to a certain losses proposed in prior work. It is not reveal a general negative result and neither does it reveal potential remedies. I believe this limits the impact of the work substantially.
* Another aspect which the analysis does not address is the good empirical performance of the analyzed family of methods. Contrary to the results presented here, these methods exhibit strong empirical performance on various tasks evaluating uncertainty estimation. This gap between the theory and empirical performance in critical and should be discussed. 


--- Post Discussion Update ---- 
The response from the authors addressed some of my concerns. I have updated the score to reflect that.
questions: * The results consider methods which learn second-order predictors through loss minimization relying on the input data. But there have been methods proposed in literature [1], which learns a second order predictor relying on features derived from the dataset. How do the theoretical results in this work relate to such approaches? 

[1] Jain, M., Lahlou, S., Nekoei, H., Butoi, V., Bertin, P., Rector-Brooks, J., ... & Bengio, Y. (2021). Deup: Direct epistemic uncertainty prediction. arXiv preprint arXiv:2102.08501.
limitations: * In summary, I think the main limitation of the work is that it is fairly narrow in scope. I believe it is important to critically analyze existing work, and re-examine the validity of approaches, but the impact is limited since the results are restricted to existing losses and the analysis does not reveal any potential fixes. 
* Another drawback of the analysis is that it does not account for the good empirical performance of the methods in discussion. This could certainty be attributed in part to empirical evaluation metrics but this gap is something which should be discussed.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper discusses the difficulty in learning a predictor that exhibits the epistemic uncertainty given labeled examples as a training dataset. 
This is mainly due to the lack of ground truth of epistemic uncertainty in the training set. 
Loss functions to handle the uncertainty are apt to result in an arbitrary amount of uncertainty in the prediction. 

strengths_and_weaknesses: The strengths of this paper are: 
* The topic is relevant and important in the literature. 
* The paper is well-organized and easy to follow with many examples. 

The weakness is
* a limited amount of numerical results may restrict the understanding of readers. 
For example, giving a prediction with various values of $\lambda$ in (15) may improve the clarity even better. 


questions: 1. Can ensemble methods be regarded as a form of averaging level-1 loss with $Q$ being a set of ensemble members in Eq. (8)? 
An example of ensembles is particle variational inference. 
A. Saeedi et al. "Variational Particle Approximations" JMLR 18(69):1--29, 2017.

2. An exact quantification of epistemic uncertainty may be ill-posed, but having the order of amount in the uncertainty can be useful for making a decision from the prediction. 
For example, the uncertainty in $p(y|x_1)$ is greater than that in $p(y|x_2)$. 
Is it still difficult to determine the order of uncertainties due to the lack of ground truth, or any of losses discussed in the paper may handle such relationship of magnitude? 

limitations: The theme of this paper is the limitation of epistemic uncertainty quantification.
review_title: null

Meta-review: metareview: This meta review is based on the reviews, the authors rebuttal and the discussion with the reviewers, and ultimately my own judgement on the paper. There was a consensus that the paper contributes interesting insights on uncertainty quantification, and most reviewers praised several aspects of the submission. I feel this work deserves to be featured at NeurIPS and will attract interest from the community. I would like to personally invite the authors to carefully revise their manuscript to take into account the remarks and suggestions made by reviewers. Congratulations!
confidence: Certain
Decision: Accept