Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a method to ensure calibration of probability outputs during training. They do so by adding an explicit penalty to push the output probability values towards an empirical estimate of the actual probability for the current inferred probability. They compare the method to a number of baseline approaches, showing the benefits on both synthetic and real-world data sets.
main_review: The idea is quite straightforward (which is not bad) and seems to be working well, as the authors compare their method to a number of baselines and show it is competitive. The idea is to a degree non-intuitive (at least to me a bit), but it does show good results. The writing is confusing though, and the authors could have done a much better job on that side. Please see details comments below.
summary_of_the_review: Detailed comments:
- "classification problems without inherent uncertainty", I am not sure that such problem exist. Please rephrase or elaborate further.
- Minor: make sure that all notation is correct (e.g., scalars are not bolded), both in the figures and in the text.
- "of just 75%", can you please clarify how?
- It is not fully clear why MSE_p is declared gold-standard, please clarify why.
- For Fig 4 results, more details are needed. E.g., early stopping is not well explained.
- I suggest moving Fig 5 much later, it seems introduced too early.
- For the kernel function, it is not clear whether the r-window is in the input or output space.
- In Section 6, uniform is not really uniform? Since it still increases with z.
- It is unclear which models were actually used for different data sets. This should be explained to a greater detail.
- In the last paragraph in Section 6, please explain better how each scenario aligns with the data sets, it is unclear as is. In fact, currently the entire section feels hand-wavy.
- Also, the last sentence doesn’t parse well, please rephrase.
- I recommend removing the last sentence in the Conclusion section.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper studies the problem of uncertainty quantification in deep neural nets. It introduces a concept called "probability estimation" and an uncertainty calibration method called "CaPE" based on this new concept. The studies uncertainty calibration in a few new data sets such a histopathology cancer diagnostics.
main_review: The main message of the paper is not clear. What is essentially the mathematical definition of probability estimation? All definitions in Eqs 1 - 6 are well-established terms in the uncertainty calibration literature and the phenomenon in Fig 2 is also well known to the community. It is accounted for simply as reporting prediction accuracy and calibration scores together. I am then missing what is new thing the "probability estimation" notion is telling us and where we see it in the experiments, which are 
designed simply as standard calibration experiments.

I am having hard time to make sense of the calibration loss presented as an alternative to the discrimination loss. Where does p_emp^i come from? How can one define it without giving reference to a sample-based estimation of the confidence score of f(x), which eventually boils down to the calibration we know? Having given a reference to a confidence score estimate, how is it different then from ordinary calibration?

The "function kernel" inside Algorithm 1 appears to be the only part where one can argue about a methodological novelty. This function only replaces the Dirac measure used by standard binning with a kernel density estimator. This is a standard trick to improve numerical stability, but it is hard to say that this much makes a scientific novelty worthwhile to be published as a main-track conference paper.

The paper lacks a clear focus in its core theme and a consistent logical flow. Is the main message how good Algorithm 1 is? Then the justification of the algorithm and explanation of why it works in the way it works is missing. Is it the allegedly new applications where uncertainty calibration is used? Then references to the closest work are missing. I also do not think these are such novel applications of uncertainty calibration. For instance see:

Thagaard et al., Can you trust predictive uncertainty under real dataset shifts in digital pathology?, MICCAI 2020
summary_of_the_review: The paper lacks a clear problem statement, sufficient technical novelty, and a consistent logical flow. Under these conditions, I am not able to recommend an accept.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper tackles the problem of uncertainty calibration in the special case of binary classification. The authors propose the so-called CaPE to iteratively minimize the discrimination loss and the calibration loss. 
main_review: The definition of of ECE and MCE is a special case (for binary classification) of the more general one that handles multi-class classification. Mathematically the definition is accurate, but overly obscure and complicated. I suggest the authors refer to the definition from Guo et al., 2017. Essentially, ECE calculate the difference between confidence and accuracy in each bin. It is therefore a bit confusing to call the accuracy term p_emp. 

In term of methodology, it makes sense to alternate between the two losses. However, it seems CaPE uses the training set to compute p_emp. Would p_emp itself run into the problem of over-confidence? For example, a well trained neural network (NN) is always over-confident and therefore most predictions (probability) will concentrate around 1, meaning that most bins will have nearly zero data point. Could you provide more intuition or theoretical analysis on why over-confidence would not happen? 

Why is the discrimination loss only one-sided? In other words, what happens if the normal cross-entropy loss is used? 

This brings me to another concern which is the extension to multi-class classification. The current discrimination loss will not work probably when there are more classes since it only focuses on the single correct class. Therefore, I would expect that the performance will get worse and worse as the number of classes increases.

The plots in Figure 5 show the benefit of CaPE. However, the drop of training loss and increase of validation loss are quite steep for CE. In practice, this rarely happens for high dimensional data, especially if the network architecture is well chosen. 

The baselines seem comprehensive. Are there results for ECE/MCE/Brier Score for Table 1?

In Table 2, it seems CaPE is worse than Deep Ensemble and MMCE Reg. in Collision Prediction and worse than Deep Ensemble  in Weather Forecasting (in terms of AUC). I was wondering how many models are used for Deep Ensemble, since the performance of Deep Ensemble is also related to the number of individual models. 



Minor:
There are two entries for the same reference Gupta et al., 2021.

Some related work on probabilistic neural network for uncertainty estimation is missing [a,b,c]. 

[a] Natural-Parameter Networks: A Class of Probabilistic Neural Networks, NIPS 2016
[b] Feed-forward Propagation in Probabilistic Neural Networks with Categorical and Max Layers, ICLR 2018
[c] Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagation, CVPR 2019

summary_of_the_review: Overall, the authors proposed a simple and effective method for improving the uncertainty estimation for binary classification. Several drawbacks include: the reason why training directly on the training will not lead to over-confidence is unclear; performance is worse than some baselines in some dataset; the method seems limited to binary classification and it would be interesting to see at least its performance on multi-class classification.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work tackles the problem of probability estimation. Current machine learning models do not fully reflect the uncertainty of the outcome but only of the model. The authors propose a loss that enforces both calibration and discrimination. Additionally, they present a semi-synthetic dataset for further study of this problem: from an image dataset of faces with associated age, they estimate the risk of developing a given disease.
main_review: The authors address an interesting problem. However, it would be interesting to further justify why current models do not offer an accurate estimate of uncertainty, particularly bayesian modelling. It is also important to note that calibration is a key issue in the survival literature and penalties have been developed to tackle this issue [1]. 

An interesting point is the decomposition of the Brier loss, which would benefit from further descriptions.

Finally, it is interesting that the authors use their loss only as a fine-tuning of the neural network. From figure 5, it seems that the introduced loss does not allow to reach a lower minimum but stops the training of the neural network. It would be interesting to see what is the effect of training the neural network from the start with this given loss.

[1] Lee, C., Zame, W.R., Yoon, J. and van der Schaar, M., 2018, April. Deephit: A deep learning approach to survival analysis with competing risks. In Thirty-second AAAI conference on artificial intelligence.
summary_of_the_review: The work is clear and concise. The authors propose to tackle an interesting problem. However, it would benefit from further justifications of the utility and contextualisation with other domains of machine learning.
review_title: null

Meta-review: The paper addresses the problem of uncertainty quantification in deep neural nets. The authors introduces the CaPE calibration loss to deal with the inherent uncertainty in probabilistic prediction, e.g. medical prognosis, weather prediction or collision prediction.

The paper initially received contrasted reviews: two weak acceptance, one weak rejection, and one strong rejection recommendation. The main limitation pointed out by reviewers related to the unclear definition of the problem setting, the limited contributions, and clarifications on experiments (comparison with deep ensembles). After authors' feedback, the reviewers were not convinced by the clarification on the problem setting, and there was a consensus among reviewers to reject the paper. 

The AC's own readings confirmed the concerns raised by the reviewers, and also identifies additional shortcomings of the current submission. The paper addresses the problem of proper quantification of data uncertainty (generally referred as aleatoric uncertainty), and the CaPE calibration loss should be positioned with respect to the literature on the topic. The AC thus recommends rejection, but encourages the authors to re-submit their work after specifying the focus and motivation of their work.
Decision: Reject