Venue: ICLR 2023 Conference

Reviews:

final_score: rating: null
review_content: The area chair raised the question about what exactly are the relationships of this work with the work of Zhang et al 2020 which also involves KDE in the context of calibration. Currently, the paper refers to Zhang et al 2020 but does not elaborate on the exact relationship of the KDE part in that work. Furthermore, the paper currently claims that "Interestingly, no one has used kernel density estimation (KDE) to generate calibrated predictions to the best of our knowledge". Taken literally, I think the claim is valid because Zhang et al 2020 used KDE for evaluating calibration and not for generating calibrated predictions.  But then the question is whether the proposal is a straightforward adaptation of the work of Zhang et al from evaluation of calibration to generation of calibrated predictions. In estimating ECE, Zhang et al. define the quantity $\hat{\pi}(z)$, which is essentially an estimated calibration map that could also be used for generating calibrated predictions. A crucial difference, though, is that the current paper performs KDE in a different and learned space rather than on the classifier's outputs directly. However, I think it would be better if the authors would explicitly acknowledge in the paper that KDE has been used before in a similar situation and explain the differences from the work of Zhang et al. 2020.

Jize Zhang, Bhavya Kailkhura, and T. Yong-Jin Han. Mix-n-match : Ensemble and compositional methods for uncertainty calibration in deep learning. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 11117–11128. PMLR, 13–18 Jul 2020. URL https://proceedings.mlr.press/v119/zhang20k.html.

review_title: Similarity to Zhang et al 2020 to be clarified

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper tackles the question of how to generate “fully calibrated” predictions in multi-class classification problems. It proposes a method, “KCal”, which learns a low-dimensional projection of a DNN’s final-hidden-layer embeddings, and then uses kernel density estimation (KDE) in this projected space to generate the fully-calibrated probability vectors.

Theoretically, this paper shows that as the size of the calibration set increases (where size here is measured in terms of the number of occurrences of the rarest class), the probability vectors generated by KCal get closer to being fully calibrated, and in the limit m -> infinity, they are perfectly calibrated.

Empirically, this paper compares KCal to a large number of other calibration methods, on a variety of computer vision and health monitoring datasets. It shows that KCal generally maintains or improves the accuracy of the uncalibrated model, while generally attaining better calibration scores relative to the baseline methods.

strength_and_weaknesses: Strengths
- The method is relatively simple, well-motivated, and easy to understand. Using KDE for calibration seems like a natural idea, given the connection between KDE and the Bayes classifier mentioned at the end of section 3.1 (on that note, it’s surprising to me that KDE has not been proposed elsewhere to generate calibrated predictions, as is claimed at the end of section 2).
- The method performs well empirically, across a wide range of experiments, and compared to a wide range of baselines.
- The method is theoretically tractable, and provably leads to calibrated predictions under certain conditions/assumptions.

Weaknesses
- Although I didn’t study the proofs in detail, I was curious whether there was anything in the proofs that was specific to the proposed learning algorithm. In particular, would the results hold for any feature space? Or is there something specific about the learned low-dimensional feature space that allows these theorems to hold? Are these standard results based on using KDE on a fixed feature space?
- As far as I can tell, no ablations are performed to better understand what aspects of the algorithm are crucial for attaining good calibrated predictions. For example, what happens if KDE is applied directly on the final-hidden-layer outputs of the DNN?
- I was also curious whether theorems 3.3 and 3.4 gave tight or loose bounds, given the real values of m, B, C, K, d, and alpha on the real tasks (can all of these be estimated?).

clarity,_quality,_novelty_and_reproducibility: - Clarity: The paper is quite clear in most places. The experimental results could perhaps be explained in a bit more detail (for example, reviewing the definitions of the various calibration scores).
- Quality: The work appears to be high-quality throughout.
- Novelty: The method seems novel. However, I am not very up-to-date with the latest calibration methods, and thus this is hard for me to evaluate.
- Reproducibility: Code is provided in the supplementary material.

summary_of_the_review: Overall, I believe the proposed KCal method is a novel and promising approach for generating more calibrated predictions for multi-class classification problems, and thus that this paper should be accepted.  However, my review is only medium confidence, given that this is not an area of research I have worked directly in.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a kernel-based calibration KCal where class proabilities are determined by per-class kernel density estimates in an embedding space. The embedding space is obtained by a multi-layer perceptron applied on top of the penultimate layer of the neural network. Extensive experiments demonstrate that KCal outperforms the methods on most relevant measures and on many datasets. KCal has asymptotic guarantees and finite sample bounds.

strength_and_weaknesses: Strengths:
* The paper is well-written and clear;
* The method is novel, theoretically justified and improves the state-of-the-art;
* The experiments have been carried out with great care in optimizing the hyperparameters for the existing methods;

Weaknesses:
* I didn't find any major weaknesses.

Minor weaknesses:
* The list of measures includes ECE, Accuracy, CECE, and Brier score. However, cross-entropy (NLL) could have been considered as well, because all proper losses decompose into refinement and calibration losses, not just the Brier score. NLL has different properties than the Brier score, and pays more attention to the probabilities near the extremes.
* The number of instances per bin could have been shown for Figure 2 (and for similar figures in the appendix), for example as a separate figure. This would help one to visually assess roughly how high ECE would be.

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written, original, and of high quality.

summary_of_the_review: The paper is well-written and has very few minor weaknesses. The proposed method is performing better than existing methods for most evaluation measures and datasets. Training of the projection MLP is an extra step that has to be done on the validation data.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new Kernel-based calibration method called KCal which  learns a metric space on the penultimate-layer
latent embedding and generates predictions using kernel density estimates on a calibration set. Theoretical and empirical results shows the effeciveness of KCal. 
strength_and_weaknesses: Strengths: 
1. A well-established method that builds a kernel-based calibration method. 
2. Theoretical proof of KCal.
3. Experimental result analysis.

Weakness:
1. Somewhat bold claims of first theoretical guaratee/kde calibration method: The conformal predictions is well-studied with theoretical guarantee[Shafer 2008, Angelopoulos 2020].  The kernel mean embedding for uncertainty calibration [Kumar 2008, Cui 2020, Bhatt 2022]. 
2. Calibration performance not better in all cases 
3. Computation time not fully explained/demostrated. As said in the paper, computation overhead of KCal is generally larger than ordinary methods and this needs to be empirically analyzed in different scales and acceleration settings. 


References
1. Shafer G, Vovk V. A Tutorial on Conformal Prediction[J]. Journal of Machine Learning Research, 2008, 9(3).
2. Angelopoulos A, Bates S, Malik J, et al. Uncertainty sets for image classifiers using conformal prediction[J]. arXiv preprint arXiv:2009.14193, 2020.
3. Kumar A, Sarawagi S, Jain U. Trainable calibration measures for neural networks from kernel mean embeddings[C]//International Conference on Machine Learning. PMLR, 2018: 2805-2814.
4. Cui P, Hu W, Zhu J. Calibrated reliable regression using maximum mean discrepancy[J]. Advances in Neural Information Processing Systems, 2020, 33: 17164-17175.
5. Bhatt D, Mani K, Bansal D, et al. f-Cal: Aleatoric uncertainty quantification for robot perception via calibrated neural regression[C]//2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022: 6533-6539.

clarity,_quality,_novelty_and_reproducibility: Clarity&quality: This paper is clearly written.
Novelty: kernel is not new to the probability calibration but accroading to the authors, kde is firstly introduced.
Reproducibility: The code to replicate all our experimental results is submitted along with supplementary materials.
summary_of_the_review: This paper proposes KCal which is claimed as the first kde-based calibration method that brings theoretical analysis. The method itself is interesting and promising but some claims and empirical results needs improving.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method called Kcal for the (full) calibration of deep neural networks.
Kcal consists in replacing the last layer (usually a softmax) of a neural network with a learned projection layer for dimensionality reduction, followed by a Kernel Density (KDE)-based estimator to produce output probabilities. The learnable parameters introduced by Kcal are the parameters of the dimension-reducing projection layer $\Pi$, as well the kernel bandwidth $b$. The parameters of $\Pi$ are learned by (stochastic) gradient descent with an appropriate training strategy: contrary to classical neural networks, the KDE “layer” involves all training points and not only the point which is flowed through the network. The kernel bandwidth $b$ is chosen on a calibration set to minimise the log-loss.

A theoretical analysis is provided in the form of high probability error bound, which shows that as the per-class sample size increases, the estimated probabilities converge to the true probability of the class $Y$ given the input score, i.e, the estimated probabilities are calibrated.

Experiments are provided for 4 vision datasets (CIFAR10, CIFAR100, SVHN, ImageNet) each time with a choice of pre-trained architecture, and 3 health monitoring datasets. Kcal is then compared to 8 calibration baselines in terms of ECE (top-label and class-wise), accuracy and Brier score of the top class, and shown to provide favourable performance across all measures (or on par for ECE).
strength_and_weaknesses: **Strength**.  
Kcal targets **full calibration**, i.e., a calibration conditioned on the full probability vector rather than just conditioned on the probability of predicted class (confidence calibration). This is an important problem since as stated by the authors, small probabilities should also be reliable in high stakes applications such as healthcare.

Kcal provides improvements in terms of Accuracy, Class-wise ECE and Brier score across most datasets.

**Weaknessess**.   
Using a KDE-based estimator requires an adapted training scheme, and comes with a presumably high computational cost. What is the training time of Kcal across the different datasets?  
It also introduces multiple hyper parameters: the kernel bandwidth, the projection architecture and output size, the batch size used to learn the projection, the number of samples used to build the KDE.  
Therefore, it comes with both a certain computational and implementation complexity.

No insights are provided to explain the performance of Kcal compared to the other baselines. I am wondering wether it could be due to the fact that it re-uses the whole train set to fit the projection map, while other methods such as Temperature Scaling only calibrate based on the smaller calibration set.

**Question**.  
This is a question regarding the choice of kernel bandwidth $b$. Since it can be folded in the projection map, it is implicitly learned while training the projection map. We see in appendix that across all datasets, the best kernel bandwidth equals 1. it would make sense if it has already been learned with the projection map. Hence the question: it is useful to have a bandwidth selection step on the calibration set?
clarity,_quality,_novelty_and_reproducibility: I found the paper well written, easy to follow, clear and pedagogical.

Kcal can be summarised as fine-tuning a DNN with a KDE as last layer instead of a softmax. I have not seen such as attempt earlier and this idea is novel to me.

Note 1: To improve clarity, it could be helpful to have a diagram showing an example of train set with a batch and a background set.

Note 2: There is a mistake in Definition 2 (Confidence Calibration). The argmax should be taken over the whole vector of probabilities, and the conditioning occurs on the maximum entry of $\hat p$. 

Note 3: In the sentence “Calibration is the process of closing the gap between the prediction and the ground truth distribution“, it would be better to use the word “reducing” instead of “closing” as calibration alone, even of a perfectly accurate classifier, is not enough to remove the ground truth probabilities.

summary_of_the_review: I think that the clarity of the paper as well as the favourable empirical results outweight the weaknesses.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The submission proposes a density estimate approach to generating calibrated outputs.  It essentially combines a learning procedure for a latent space with a classic density estimate regression approach - the main predictive equation of the paper appears e.g. after (12.3) in http://faculty.washington.edu/yenchic/18W_425/Lec12_class.pdf
The main algorithmic contributions is to do this on a space learned by a neural network (Alg 1).  The reviewers were generally pleased with this contribution, with a unanimous opinion that the submission passes the threshold for acceptance.  Sometimes a simple approach based on a classic result is a valuable contribution, especially when it has nice calibration guarantees and competitive empirical results with other more recently introduced methods.

I have serious concerns about the claims of novelty in the paper, and would like that the authors remove one of their claims, namely " To the best of our knowledge, this is the first method with a full calibration guarantee."  The method essentially performs a known classical density estimation approach albeit on a learned space.  The theoretical analysis in Section 3.3 is nice, but it essentially proves that a classical method also has this guarantee.  Thus, it is highly misleading to say that this is the first method to have a calibration guarantee.  Rather, it provides an analysis showing that regression based on KDE is a good method when calibration is required, and that it is particularly effective when combined with a learned subspace and careful analysis of the setting of the kernel bandwidth.  With this improvement, the paper can be a nice contribution to ICLR.

Furthermore, there are interesting concurrent works on KDE and calibration, with the following papers using KDE for calibration appearing in NeurIPS 2022:
https://openreview.net/pdf?id=PikKk2lF6P
https://openreview.net/pdf?id=HMs5pxZq1If - it appears an earlier version of this paper is on openreview as a submission to ICLR last year
https://openreview.net/pdf?id=q85GV4aSpt

justification_for_why_not_higher_score: The submission is an interesting combination of a learned latent space and a classical KDE regression method.  There is an appropriate theoretical analysis of the method showing calibration, but the algorithm does not have a high degree of novelty.  It is not well presented in the context of existing methods, with the authors strongly overstating their claims to being first.
justification_for_why_not_lower_score: The paper received unanimous accept reviews.  The method uses an interesting classical method in a new context to good effect.
Decision: Accept: poster