Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, a new training strategy which integrates conformal prediction in training stage is proposed. The proposed training algorithm uses an additional regularization term that encourages the conformity scores to follow a uniform distribution to improve the performance of final conformal prediction.
strengths_and_weaknesses: Strengths:

1. Previous training stage doesn't consider the performance of conformal prediction, and hence results in sub-optimal problem. The proposed method addresses this issue by encouraging the conformity scores to follow a uniform distribution.

2. The proposed differentiable approximation for the empirical CDF of the conformity score could be efficiently implemented. The validation dataset could be used in both training (for calculating $l_u$) and post-hoc calibration.

3. The experiments with synthetic data show some positive points of the proposed method, and also give some intuitive analyses.

Weaknesses:

1. The authors mentioned that "The idea is to mitigate overconfidence" in Abstract. Does the proposed method (i.e., adding conformal loss term $l_u$ in loss function) directly mitigate the over-confidence issue? 

2. The proposed method could be a regularization term on the loss function. What about the relationship between it and other regularization methods (which have been used to address the calibration problem) like label smoothing?

3. Other real-world datasets should be used in experiment part. This paper only uses CIFAR-10 for evaluating the effectiveness of the proposed method. As it is not too hard to classify, image augmentation processes are used to generate harder samples. Why not use datasets like CIFAR-100 or (Tiny-)imagenet for better evaluation?

questions: See weaknesses.
limitations: The authors have adequately addressed the limitations and potential negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a differentiable loss function for training conformal predictors. In conformal prediction, a popular scoring function for classification sets is the APS method of Romano et. al. [1], as if the learned conditional distribution $\hat \pi_y(x) \approx  \pi_y(x) := \mathbb{P}(Y = y \mid X = x)$ is exact, then the constructed prediction sets are the smallest randomized prediction sets with the desired conditional coverage, i.e., satisfying $\mathbb{P}(Y_{n+1} \in C(X_{n+1}) \mid X_{n+1} = x) \geq 1 - \alpha$. Typically, the way to go about solving this problem is to first learn $\hat \pi_y(x)$, and then plug it into the APS method (which is then calibrated using normal conformal techniques). Unfortunately, as the authors point out, predictions based on poorly pre-trained $\hat \pi_y(x)$ are hard to correct once $\hat \pi_y(x)$ is fixed. This paper links the two steps of conformal prediction into one joint loss, with a focus on improving conditional coverage via better recovering the oracle behavior of the true $\pi_y(x)$ combined with APS. This joint objective is composed of the normal cross-entropy classification loss, together with a regularizer that encourages conformal scores on held-out data to be uniformly distributed (as they would be if $\hat \pi_y(x) = \pi_y(x)$). The method is empirically validated on both synthetic and real datasets, though the gains are fairly minor depending on the setting.

[1] Classification with Valid and Adaptive Coverage. https://arxiv.org/abs/2006.02544.
strengths_and_weaknesses: === Strengths ===

- The paper is well-written and well-evaluated. The method is also well-motivated. In particular, I appreciate the focus on conditional coverage rather than set size, as set size can be gamed when only subject to marginal coverage constraints.

- The proposed algorithm is simple (at least when relying on pre-existing differentiable sorting algorithms). Proposition 1 yields good intuition (and also interestingly appears somewhat related to the fact that the distribution of $Z = F_{Y|X}(Y)$ is uniform for continuous r.v.s $Y$ with CDF $F_{Y|X}$, but here by construction for discrete $y$).

- Though I have some questions about the empirical effectiveness on real data (see below), at least under certain settings the method can lead to smaller prediction sets with better conditional coverage---two important and impactful qualities for real-world deployment of conformal algorithms.

=== Weaknesses ===

- When factoring in early stopping, the empirical gains on real datasets vs. baselines (i.e., Hybrid) appear minor at best (e.g., the best seems to be a +4% gain in corrupted coverage on CIFAR-10 when comparing fully-trained conformal with early-stopped hybrid?). The raw (top-1) accuracy also seems to be negatively on the credit card default task (whereas standard conformal methods that don't modify the base conformal score don't affect the top-1 accuracy).

- This is addressed in the paper, but the proposed method is quite expensive to train (2x compared to cross-entropy loss, as reported in Section 5). As a result, I'm not sure how well this would scale to larger models than a ResNet18. Given the somewhat small gains in efficiency/conditional coverage, the impact of this approach seems likely to be somewhat limited. (That said, I imagine that the "Hybrid method" of [1] is similarly slow, as it also involves a differentiable sort.)

- One thing which bothers me is the data inefficiency of the proposed approach. It seems rather wasteful to only use labeled data in $\mathcal I_2$ for regularization. It seems if the main goal is to simply get a better estimate of $\pi_y(x)$ by reproducing its behaviour when plugged into conformal APS (e.g., uniform score distr.), what if we just use that data to train a bigger, modern model (e.g., see models in [2]), optionally with better regularization (e.g., calibration objective in [3])? Or, since the method also requires validation of $\lambda$, early stopping on the same validation set (as done in the experiments, which improves baselines substantially) also seems to be a fair data-wise comparison. 

- Likewise, from the paper experiments, benefits become more pronounced with more data, however, one could argue that uncertainty estimation is particularly important for tasks in which we don't have much data (and hence base models are poorly trained). Note that this data inefficiency is also shared by [1], but the objective in [1] can dependent on properties of the set $C$ that may be orthogonal to conditional coverage (and recovering $\pi_y(x)$ may not yield the oracle).

=== Minor ===

- Notation: In several places $|\mathcal I_*|$ should be used in place of $\mathcal{I}_*$ (e.g., L165). 

[1] Learning Optimal Conformal Classifiers. https://arxiv.org/abs/2110.09192.

[2] Revisiting the Calibration of Modern Neural Networks. https://arxiv.org/abs/2106.07998.

[3] Trainable Calibration Measures For Neural Networks From Kernel Mean Embeddings. https://proceedings.mlr.press/v80/kumar18a.html.
questions: - Low 80% top-1 accuracy seems pretty low for a ResNet-18 on CIFAR-10 (see, e.g., https://github.com/kuangliu/pytorch-cifar). This makes me question if the cross-entropy baseline is indeed well trained? Can you explain this difference?
limitations: The authors have adequately addressed the limitations and potential societal impact of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a method of combining the learning and calibration phases
of conformal inference, reducing overconfidence in the ML model, and giving
smaller prediction sets. They minimize a new loss with minimizes the
discrepancy in conformity scores between the current model and an unknown
oracle.

strengths_and_weaknesses: # Strengths

- The method id well justified and comes equipped with in depth analysis
- The experiments verify the claims made within the text and also provide ablations on the proposed components.
- The experiments are thorough.

# Weaknesses

- Equation three seems to give incomplete infromation. $u$ and $\tau$ are
defined but there is no mention of how they contribute to the function
$\mathcal{S}$. It is then defined in the appendix, but A2 in the appendix takes
$t$ (not $\tau$ but I guess they are the same?) as an argument and equation 3 takes $1 - \alpha$ leaving $\tau$ unused.
This needs to be cleared up.

- The covariate shift experiments occur on the synthetic data. It would be nice
to see if these results hold for larger datasets as well. For example, what
would happen if a model were trained with regular CIFAR10 data with no
corruptions in either the training or calibration set, and then tested with the
random erase corruptions or CIFAR10-C [1] test set.

- L237: states that figure A6 shows typically higher conditional coverage as
compared to all benchmarks. It appears there is actually a clear trend which
shows that the proposed model performs worse on larger numbers of classes, yet
there is no mention of this. Can the authors provide any explanation for this?
 
# References

[1] https://github.com/hendrycks/robustness

questions: - I am not sure there is enough information in the current version of the paper in order to implement the algorithm. Section 3.3 is easy to intuitively understand, but it leaves me confused as to where to go in order to actually implement what is written in the paper. Could a more detailed explanation of the exact equation needed to reproduce $\ell_u$ be included?

# Summary

Overall, I think the contribution is solid, but there can be some improvements made as reflected in my comments and questions. 
limitations: The authors have highlighted the computational complexity as a limitation. I am not aware of any adverse societal impacts of their work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper combines ideas from conformal prediction and differentiable ranking/sorting to develop a loss function that improves the predictive uncertainty of a classifier by training it to make predictions with a desired conformity score. The paper reports competitive coverage and conformal prediction set sizes on synthetic data, CIFAR10 and a credit card default dataset.
strengths_and_weaknesses: Strengths:
* The method is novel as far as I can tell (although I am not familiar at all with the related literature on frequentist approaches for uncertainty estimation) and appears to be technically sound.
* The technical setup is well-structured and clear for the most part (although see exceptions).
* The proposed method seems to perform the best on the synthetic data and is competitive on real data.

Weaknesses:
* I found section 3.3 quite vague and think the paper would benefit from recapping the specific techniques it uses from the referenced papers. As it stands, the text is not self-contained and I don't think I'd be able to implement the method without working through other works. The algorithm box does not help in this regard as it is quite wordy rather than technical and concise.
* I was fairly surprised that the proposed method appears to require a post-hoc calibration step (lines 199-201, 259, 305), even though the introduction criticizes this two-step approach as a limitation of conformal learning.
* The empirical results on the real data seem to be somewhat of a mixed bag with many close scores. Reporting results without early stopping seems unnecessary to me since the classification performance is significantly better with early stopping (although I am surprised by the need for it; see questions).
* There is no ablation study on the hyperparameter for the weight of the conformal loss. I'd imagine that this parameter allows for trading off more accurate vs better calibrated predictions, but it would be helpful to see this confirmed experimentally.
questions: I'm overall not too confident in my review due to being unfamiliar with the related literature and commonly used experimental setups and results, but I'd appreciate the following questions being clarified:

* Are you indeed using a two-level procedure even with your conformal prediction loss? If yes, is it possible to use the loss as a one-level approach and if so, how does it perform? I had a look at the appendix and did not find anything there, but might have missed it.
* Do you have an explanation for where the need for early stopping arises? I can imagine that with an added regularizer this can happen, but the bad test accuracy for the cross-entropy baseline on CIFAR10 seems extremely strange to me, I don't think I have ever seen the test accuracy of a properly trained Resnet go down over time on that dataset. Is it due to the data corruption? That being said, it would be nice to have results trained on clean data as a baseline to see that the implementation is reasonable. 
* How do different choices of $\lambda$ affect performance?

Other (minor) notes:
* I'm a bit uncomfortable with some statements in the introduction. In particular, neural nets trained with proper scoring rules should be calibrated in the infinite data limit. Intrinsically noisy data is not problematic, as this would be reflected in the training set and lead to less confident predictions. The problem in benchmarking on datasets such as CIFAR10 is the opposite, namely that they are highly curated and there is hardly any noise in the data, but once the underlying data distribution shifts the predictions become overconfident. I'm probably making some statements here myself that are not perfectly accurate either, but I'd encourage have another pass over the first paragraph of the introduction and ensure that all statements are technically precise and correct.
* Exchangeability of the data seems to be a core assumption, it would be nice to have an evaluation of the method under some shift of the test distribution on the real data, e.g. using different degrees of corruption as in (Ovadia et al., 2019. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In NeurIPS).
* Table captions should go above the tables (unless the style instructions have been changed).
limitations: The computational overhead (roughly doubled training time in the experiments) is discussed explicitly. I did not see discussion around the reliance of the proposed loss on a post-hoc conformalization step.
review_title: null

Meta-review: metareview: Decision: Accept

This paper extends conformal prediction techniques to multi-class classification using deep neural networks and make the training of the neural network to be aware of the conformal inference processing. The main technical contribution is a differentiable objective to approximate a CDF-based test on the conformity score. The paper provides both theoretical analysis as well as empirical evaluation results of the proposed approach.

Reviewers found the paper to be well written and the approach to be well motivated and supported. There were a few technical concerns but many of them were addressed in author feedback. 

Still the main technical downside is the expensiveness of the approach, and the experiments being relatively small scale regarding network size & dataset size. Also a lot of practical issues are not discussed, e.g., data augmentation, distribution shift, etc.

However, this is indeed an early work in the conformal prediction area that tries to make neural network training adaptive to conformal inference "post-processing", and I believe the work is going to the right direction and will have good impact in "conformal prediction + DL" area.

As a side note, I'd encourage the authors to add discussions on related work that proposes regularisers for better neural network calibration, e.g., MMCE https://proceedings.mlr.press/v80/kumar18a/kumar18a.pdf.
confidence: Certain
Decision: Accept