Adapting Prediction Sets to Distribution Shifts Without Labels

Authors: Kevin Kasa, Zhiyu Zhang, Heng Yang, Graham W. Taylor
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords: Conformal prediction, Set-valued classification

Abstract:
Recently there has been a surge of interest to deploy confidence set predictions rather than point predictions. Unfortunately, the effectiveness of such prediction sets is frequently impaired by distribution shifts in practice, and the challenge is often compounded by the lack of ground truth labels at test time. In this paper, we present a method for improving the quality of outputted prediction sets using only unlabeled data from the test domain. This is achieved by two new methods called $\texttt{ECP}$ and $\texttt{E{\small A}CP}$, that sit on top of existing set-valued classification methods and adjust their intervals according to the base model's own uncertainty evaluation on the unlabeled test data. Through extensive experiments on a number of large-scale datasets and neural network architectures, we show that our methods provide consistent improvement over existing conformal prediction based baselines and nearly match the performance of fully supervised methods.

Decision: reject
Reviewer scores: 3, 3, 5

Reviews and discussion:
<thread>
creation date:
1729235466343

thread_type:
official_review

<thread_object>
creator: reviewer_cr3g

<content>
summary:
Conformal Prediction intends to predict confidence set predictions rather than point predictions. This paper studies to improve the performance when there is domain shift for testing. First, this paper proposes ECP, which addaptively controls the perdiction set size by utilizing the uncertainty evaluation of the base model itself. Second, being motivated by the concept of TTA, this paper uses entropy minimization for optimizing the uncertainty. Without label annotation, this paper shows good performances.

strengths:
The preliminaries are well formulated.

weaknesses:
Please refer to the questions below

questions:
- What is resp. in the first paragraph of introduction section? It is not desirable to use undefined abberviation term without explaining it.
- Why there are cases when entropy optimization shows worse performance than the original ECP?
- The problem setting suggested by the authors cannot be solved by the foundation model, e.g. CLIP or LLMs?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730778299981

thread_type:
official_review

<thread_object>
creator: reviewer_rzhg

<content>
summary:
This paper proposes a novel conformal prediction method under distribution shift without using any labels from a shifted distribution. The key idea is rescaling a scoring function based on entropy, which measures a degree of shifts. Also, this paper updates the scoring function using test-time adaptation to enhance the efficiency of conformal sets. The proposed methods are evaluated over variations of ImageNet for distribution shifts.

strengths:
This paper attacks an important problem of learning conformal sets under distribution shifts. The observation is simple and intuitive. It demonstrated the efficacy of the proposed method by using at least 6 shifted datasets.

weaknesses:
I think the main concern of this paper is that the proposed method does not provide any theoretical guarantee, which is the crux of conformal prediction. Including this, see the following for additional concerns. 

* The proposed method is heuristic, which does not align with paper trends of conformal prediction. Can you provide a theoretical coverage guarantee (i.e., the coverage probability bound) of the proposed method under the covariate shift assumption?
* For achieving a desired coverage, the method introduced two tuning knobs: \beta, f. \beta set to 1-\alpha, but choosing them is anyway very tricky in distribution shifts without labels. This suggests the proposed method is less practical. Could you suggest an adaptation method to learn f by only using unlabeled examples?
* In demonstrating the efficacy of the proposed method, it would be great if oracle results and no-shift results are included. In particular, you can consider SplitCP with access to labeled examples from a shifted distribution – this is an oracle result, which forms a empirical desired coverage / set size for the proposed method. Also, under no shift, the proposed method should work – this is also demonstrated to show the efficacy of the method as we don’t know whether there is a shift or not. 
* An important baseline is missing. The main competitor of this paper is conformal prediction under covariate shift (i.e., Tibshirani et al., 2019). The comparison should be conducted. Could you apply your method to visualize the second plot with oracle weights in Figure 2 of Tibshirani et al. (2019) and compare an empirical coverage along with average prediction set size?

questions:
* Can you provide a coverage guarantee of the proposed method?
* Can you add oracle results and no-shift results to justify the efficacy of the proposed method?
* Can you compare the proposed method with Tibshirani et al., 2019?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730796043132

thread_type:
official_review

<thread_object>
creator: reviewer_2xjb

<content>
summary:
The paper presents a novel approach to enhance the robustness of set-valued predictions, specifically Conformal Prediction (CP), in the face of distribution shifts without relying on test-time labels. The authors introduce two primary methods, ECP (Entropy scaled Conformal Prediction) and EACP (Entropy base-adapted Conformal Prediction), which leverage the uncertainty revealed by the base model to adjust prediction sets accordingly. Through extensive experiments on large-scale datasets and various neural network architectures, the paper demonstrates that these methods can significantly improve upon existing baselines and nearly match the performance of fully supervised methods.

strengths:
The investigation presented in this paper on Conformal Prediction (CP) under distribution shifts is of considerable value, addressing an area that has received relatively little attention in existing studies.

weaknesses:
The motivation behind this paper is that, under distribution shifts the overall output confidence of models tends to decrease, thereby affecting the performance of CP. However, there are several aspects in the method design that currently fail to convince me, such as:

1. Figure 1 observes the relationship between the model's output confidence and entropy, using this relationship as a key observation to guide the method design. But isn't this correlation trivial? Entropy and confidence are both indicators of model output uncertainty, especially when considering average entropy and confidence, as the method proposed in [1] treats both as indicators of model confidence.

2. The authors, through preliminary experimental observations, designed a method to enlarge the model's output under Out-Of-Distribution (OOD) conditions. I believe caution is needed for this design, as **existing research has widely pointed out that models face a more severe over-confidence phenomenon under OOD settings compared to In-Distribution (ID) settings**, and continuing to enlarge model outputs will inevitably exacerbate this issue. From this perspective, is the method in this paper reliable? Would a more over-confident model (which is certainly also more miscalibrated) be beneficial for CP?

3. Why do the authors use the quantile mechanism shown in Equation (3) to enlarge the model's output? This seems to lack insight. Assuming that the strategy to enlarge model confidence is correct (however, as mentioned in point 2, this assumption may not be correct), shouldn't we directly use a method like temperature scaling to align the model's average output on OOD and ID datasets?

I believe there are still many aspects of this paper that fail to fully convince the readers and require further clarification from the authors.

[1] Leveraging unlabeled data to predict out-of-distribution performance, ICLR 2022.

questions:
Please refer to weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733248247688

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
Dear reviewers,

Thank you for your comments and reviews of our work.

We are glad that Reviewer 2xjb acknowledges the significant improvement upon baselines of our method, and that the setting we investigate is both important and currently under-studied in the literature. The importance of our setting and efficacy of our method are also echoed by Reviewer rzHG and Reviewer cR3g.

We hope that our response has clarified the Reviewer 2xjb's concerns. Importantly, the core concern of over-confidence under OOD is addressed by the fact that our method **enlarges prediction sets under OOD data thus conveying *greater* uncertainty**. We also point to Figure 4 showing that the varying set sizes of our methods starkly contrast with the static sets of other baselines; this is indeed the desired behavior the reviewer mentioned. 

We have further updated our manuscript to contain the additional experiments suggested by Reviewer rzHG, demonstrating our method outperforms Tibshirani et al. (2019), and that it is comparable (and often better than) the Oracle in terms of prediction set sizes. We also appreciate their openness to "novel viewpoints on building prediction sets under shifts." While statistical guarantees may be sought after in some settings, we strongly emphasize the **immediate practical improvements** of our methods and significant improvements upon existing baselines that practitioners and researchers can already begin using. We certainly hope that future work can further expand the theoretical motivations of our approach, as has often been the case in ML research.

Finally, we believe our response has adequately addressed all of Reviewer cR3g's stated weaknesses, including their questions regarding foundation models and edge cases where entropy adaptation has room for improvement.

Thank you!
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper investigates an interesting problem where the goal is to improve the practical performance of conformal prediction using only unlabeled data from the shifted test domain. To solve this problem, this paper proposes two new methods, whose main idea is to adjust the score function in conformal prediction according to its base model's own uncertainty evaluation. Experiments show that the proposed methods outperform existing baselines.

Pros: 
- The investigated problem is interesting.
- The effectiveness of the proposed methods is demonstrated.

Cons:
- The motivation of the method design is not so convincing.
- No theoretical guarantee is provided for this conformal prediction study.

The above two defects of this paper are quite signficant and cannot be ignored for the research on conformal prediction. Therefore, I would like to recommend rejecting this paper.