Exploring the Link Between Out-of-Distribution Detection and Conformal Prediction with Illustrations of Its Benefits

Authors: Paul Novello, Joseba Dalmau, Léo Andéol
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Out-of-distribution detection, Conformal Prediction, benchmark, nonconformity scores

Abstract:
Research on Out-Of-Distribution (OOD) detection focuses mainly on building scores that efficiently distinguish OOD data from In Distribution (ID) data. 
    On the other hand, Conformal Prediction (CP) uses non-conformity scores to construct prediction sets with probabilistic coverage guarantees. In other words, the former designs scores, while the latter designs probabilistic guarantees based on scores. Therefore, we claim that these two fields might be naturally intertwined. 
    This work advocates for cross-fertilization between OOD and CP by formalizing their link and emphasizing two benefits of using them jointly.
    First, we show that in standard OOD benchmark settings, evaluation metrics can be overly optimistic due to the test dataset's finite sample size.
    Based on the work of (Bates et al, 2022), we define new *conformal AUROC* and *conformal FRP@TPR$\beta$* metrics, 
    which are corrections that provide probabilistic conservativeness guarantees on the variability of these metrics.
    We show the effect of these corrections on two reference OOD and anomaly detection benchmarks, OpenOOD (Yang et al, 2022) and ADBench (Han et al. 2022). 
    Second, we explore using OOD scores as non-conformity scores and show that they can improve the efficiency of the prediction sets obtained with CP.

Decision: reject
Reviewer scores: 5, 3, 6, 6

Reviews and discussion:
<thread>
creation date:
1729097262579

thread_type:
official_review

<thread_object>
creator: reviewer_u4qk

<content>
summary:
The authors ask themselves, and answer positively, to the question whether CP can be beneficial to OOD, and whether the vice versa holds as well. They also validate empirically their findings.

strengths:
The paper makes an extremely interesting parallel between OOD and CP. In addition, I found very interesting the marginal guarantees that the authors were able to find.

weaknesses:
CP is a method for *uncertainty representation*, not uncertainty *quantification*. Indeed, CP *represents* uncertainty via the conformal prediction region. It does not quantify it: there is no real value attached to any kind of predictive uncertainty (e.g. aleatoric or epistemic, AU and EU, respectively). Some claim that the diameter of the conformal prediction region quantifies the uncertainty, but even in that case, it is unable to distinguish between AU and EU. Indeed, the diameter is a positive function of both: it increases as both increase, and hence it cannot be used to distinguish between the two [1]. Please add this clarification in the camera-ready version of the manuscript.

The simplest CP technique is (arguably) transductive CP, not split CP.

Shouldn't $n_\text{val}$ in (2) and in (10) be substituted by $P$? In particular, CP guarantees hold for all exchangeable countably additive probabilities $P$ on the space $\mathcal{Y}$ of outputs.

How does the proposed method relate to the subsequent work by Kaur [2]?

---

[1] https://arxiv.org/abs/2302.09656

[2] https://arxiv.org/abs/2302.11019

questions:
See Weaknesses. In addition, Typo at line 69: eas (2023). Shouldn't it be Easa?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730268321934

thread_type:
official_review

<thread_object>
creator: reviewer_16uk

<content>
summary:
This paper explores the connection between Out-of-Distribution (OOD) detection scores and the non-conformity scores in Conformal Prediction (CP), showing the potential for cross-fertilization between these methods. Non-conformity scores offer probabilistic interpretation and correction for OOD scores, while OOD scores can enhance the efficiency of prediction sets obtained through CP.

strengths:
•	OOD scores can be unreliable, and employing new statistical developments to improve their reliability is a promising direction. Using the uncertainty estimates from CP appears to be a robust approach to address OOD score unreliability.

•	The paper targets a practical problem, effectively demonstrating key concepts through comprehensive case studies.

•	By linking OOD detection with CP, this work facilitates cross-fertilization that benefits both machine learning and statistics. The authors provide a good introduction to using CP for interpreting and correcting OOD scores.

weaknesses:
•	Non-conformity scores in CP require an evaluation dataset, and it reliability depends on the choice of this dataset. This introduces a practical challenge in implementing this statistically sound solution.

•	The paper presents limited innovation, as much of the work is based on existing research.

questions:
I think that there are two types of uncertainties for a model predicting a new test instance:

1.	Prediction Confidence: Assuming the instance is within the In-distribution, this type of uncertainty relates to the strength of evidence supporting the prediction.

2.	In-Distribution Uncertainty: As discussed in this paper, this refers to uncertainty regarding whether the instance truly belongs in the In-distribution.

I would appreciate further discussion from the authors on the distinctions and connections between these two types of uncertainties. Can they be quantified into a single metric?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730861427439

thread_type:
official_review

<thread_object>
creator: reviewer_nlkm

<content>
summary:
This paper investigates the relationship between Out-of-Distribution (OOD) detection and Conformal Prediction (CP). They apply conformal prediction methods from prior work by Bates et al. (2022) to the task of Out-of-Distribution (OOD) detection, which is to distinguish in-distribution (ID) data from OOD data. They introduce metrics conformal AUROC and conformal FPR@TPR95 which provide conservative probabilistic guarantees in OOD detection tasks, particularly relevant for safety-critical applications. The authors then empirically validate the performance of these metrics on common OOD benchmarks.

strengths:
The paper presents an application by adapting the conformal prediction methods from Bates et al. (2022) to OOD detection.

weaknesses:
(1) The methodology largely follows the framework established by Bates et al. (2022) in the conformal prediction literature, with limited novel contributions specific to OOD detection. The primary innovation appears to be an application of existing CP methods to OOD tasks rather than a new approach.

(2) While the idea of combining OOD detection with CP is interesting, the approach itself and the resulting metrics are relatively straightforward extensions and may not represent a significant advancement in either OOD or CP methodologies.

minors:
(1) Line 135: In Equation (2), n_val is not defined and weird, should be probability. Similarly, n_val is used as probability in Line 296, Equation (10). Other places, it is used as the number of data points. 
 
(2) A lot of places, for example, section 4.3.1, tau and t are used in mix to refer to the same thing.

questions:
Given the significant overlap with Bates et al. (2022) 's methods, what specific contributions does this paper make beyond adapting CP to OOD tasks?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730873158252

thread_type:
official_review

<thread_object>
creator: reviewer_41qn

<content>
summary:
The paper examines the relationship between conformal prediction (CP) and out-of-distribution (OOD) detection. It approaches the OOD detection problem from a conformal prediction perspective and, inspired by CP, proposes several modified metrics for OOD evaluation, testing them on various OOD benchmarks. Additionally, the paper constructs prediction sets for classification tasks, drawing on techniques from OOD detection.

strengths:
* The paper is well-structured, well-written, and easy to read.

* The paper introduces modified metrics for OOD evaluation that are more robust when considered from a hypothesis testing perspective.

* The OOD detection problem is significant for ML research, making this topic relevant to the ICLR community.

* The exploration of the link between OOD and CP is informative.

weaknesses:
* The main insight of the paper is that FPR@β can be interpreted as a p-value within a specific statistical hypothesis testing framework. By building on the work of [1], the paper proposes a corrected estimator for this metric to improve robustness. While this approach is interesting, I believe it is insufficient to warrant publication. Firstly, much of the technical foundation relies on [1], and the corrected metric for OOD detection performs similarly to the classical version. Therefore, I do not find the current version sufficiently novel for publication.

* Computing the proposed metric for OOD detection requires access to extra validation sets.

Minor Comments:
Line 22: FPR? (not FRP)

[1]Stephen Bates, Emmanuel Candès, Lihua Lei, Yaniv Romano, and Matteo Sesia. Testing for Outliers
with Conformal p-values, May 2022. URL http://arxiv.org/abs/2104.08279.

questions:
1. Is there a scenario where classical metrics like FPR@β or AUROC might fail for OOD evaluation, but your conformal metrics accurately capture the evaluation? How do the experimental results in Section 4 contribute to safer OOD evaluation?

2. When using OOD scores as nonconformity scores in CP, can you still obtain theoretical guarantees for the prediction sets?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732288838030

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response to reviewers

comment:
We would like to thank the reviewers for their constructive feedback. We took into account all the minor remarks and updated a corrected revision of the manuscript. Thank you for carefully reading the manuscript and quoting these typos!

Two concerns are shared by three of the four reviewers:

1. They deem that the contribution is limited due to the overlap with Bates et al. (2022) for corrected metrics
2. They are concerned with the need for extra-validation data to compute the proposed metrics. 

## 1. The contributions are not limited to the corrected metrics (reviewers 41Qn, NLkM and 16UK)

This criticism is supported by the observation that the technical foundations of our corrected metrics rely on Bates et al. (2022). This observation is true, and we acknowledge it several times in the manuscript.  However, deriving corrected metrics is far from being the only contribution of this work. In addition to this contribution:

- We point out that the common practice of OOD and Anomaly detection enables the use of this theoretical work. Drawing this link is a contribution.
- We apply it to extensive OOD and Anomaly Detection benchmarks. Some reviewers point out that the OOD baselines perform similarly to the vanilla metric. **While they consider it as a weakness, we argue that it is a strength**: it means that using corrected metrics, that provide **safety guarantees, comes with almost no cost** and could hence be used seamlessly in OOD. The conclusion is not the same for Anomaly Detection, which shows that current benchmarks are brittle when it comes to safety assessment, which is another strong insight.
- We also use OOD scores to build new prediction intervals, which is **not related to the work of Bates et al. (2022)**. It opens a whole avenue for nonconformity score crafting, which is central to CP performances.
- To us, perhaps the most important contribution is the **message** carried by the paper, supported by our several contributions : OOD and CP are closely related and should be used jointly. We believe that this message could have a significant impact on both fields, which are very dynamic and active research fields in AI.

We agree that each contribution taken independently would not be a sufficient contribution for a whole paper. But they form a coherent whole that advocates for further exploration of the link between CP and OOD, which is, in our opinion, a very important insight and could have a **broad impact on future research in both fields**.

## 2. We do not need extra validation (or calibration) data (reviewers 41Qn and NLkM)

We should have insisted more on that point because it is another advantage of our method: **it does not require extra validation data**.

Computing the conformal FPR only requires a correction on the estimated FPR that does not rely on a third calibration dataset. It is not like in CP where we need a calibration dataset to find a threshold based on nonconformity scores obtained on calibration data, that is subsequently used to provide CP confidence intervals. Here, there are no confidence prediction intervals; we only use CP theory to obtain probabilistic guarantees on the FPR. The steps are as follows:

- We compute the estimated FPR with the full validation data, as usual in OOD
- We apply the correction
- We compute the TPR using OOD validation data (as usual in OOD) but with a threshold that corresponds to the corrected FPR instead of the non-corrected FPR.

We added a remark in the main manuscript in Section 4.4 (Remark 4.1) to clarify this point.

When relevant, we copy these discussions in dedicated answers to facilitate the reading of our response, where we also address the reviewer's specific concerns.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper studies the relationship between conformal prediction (CP) and out-of-distribution (OOD) detection. It proposes to modify some metrics for OOD detection by taking a CP perspective. These were evaluated  It also investigates how to the OOD prediction scores can be used to improve prediction sets constructed by CP for classification tasks.  

Strengths: 
- paper well written, 
- it proposes more robust metrics for OOD evaluation, 
- it makes links with OOD and conformal prediction (CP).  

Weaknesses: 
- innovation limited due to the strong connection with Bates et al, 2022,   
- need of extra-validation sets,  
- the paper straightforward extensions when linking OOD and CP.  

The authors have provided additional precisions during the rebuttal, in particular they raised 2 main points: (i) theirs contributions are not limited to the corrected metrics (in particular they mention that they go beyond Bates et al. and provide improvements for understandings the links between OOD and CP), (ii) they do not need extra-validations datasets.   

During discussion, reviewers have appreciated the idea of studying the links between OOD detection and CP. However, a majority analysis for saying that the paper presents in its current form a limited contribution in the sense that the connections between CP and OOD are not sufficiently studied and more technical contributions, deeper analysis and extended evaluations are suitable for acceptance to ICLR. The last experiments did not convince totally. 

The remaining negative weaknesses focus on the core contributions of the paper which appear to have limitations requiring further improvements, I propose then rejection.   
However, the idea of studying the connections between OOD detection and CP seems very promising, I encourage the authors to improve their work for other venues.