Paper: Conservative Prediction via Data-Driven Confidence Minimization (ICLR-2024)

Reviewer comment (Writing):
The authors need to well-articulate the notations before these are used. For example, what is \(\mathcal{P}_{ID}\)? It is not friendly to readers not familiar with this topic. Please add the appendix reference for the definition of \(\delta\)-neighborhoods in the comment after Proposition 4.1. What does \(i\) in (4) stand for?

Author response:
> The authors need to well-articulate the notations before these are used. For examples,
What is $P_{ID}$? It is not friendly for readers not familiar with this topic.

$P_{ID}$ represents the distribution of in-distribution, or “known”, examples. It is the distribution of the training data. We’ve revised Section 2.2 to introduce this notation.


> Please add the appendix reference for the definition of $\delta$-neighborhoods in the comment after Proposition 4.1.

Thank you for the suggestion; we have added a pointer to the appendix when $\delta$-neighborhoods are first introduced in the main text.

>What does $i$ in (4) stand for?

$i$ was not needed in (4) and the proof; this was something we wrote in an earlier version of the proposition but forgot to delete. Thank you for catching this!

Reviewer comment (Theory):
The uniform label distribution used for "unknown" observations is ad-hoc. It sounds like you are assuming all unknown examples overlap together and hence you cannot clearly distinguish them. However, what if there are only several overlapped classes? For example, some unknown observations from class 1 overlap with class 2, but they are disjoint with other classes. In this case, it is not appropriate to give non-zero probability to generate other “pseudo” labels for these unknown examples from class 1. Moreover, if this unknown example is an OOD point that is unlike any of the "known" classes, does it still make sense to give a non-zero probability to be labeled as "known" classes?

Author response:
> The uniform label distribution used for "unknown" observations is ad-hoc. It sounds like you are assuming all unknown examples overlap together and hence you cannot clearly distinguish them. However, what if there are only several overlapped classes? For example, some unknown observations from class 1 overlap with class 2, but they are disjoint with other classes. In this case, it is not appropriate to give non-zero probability to generate other “pseudo” labels for these unknown examples from class 1. Moreover, if this unknown example is an OOD point that is unlike any of the "known" classes, does it still make sense to give a non-zero probability to be labeled as "known" classes?

We respectfully disagree with the claim that the uniform label distribution is ad-hoc. While we agree that it is possible for classes to have fine-grained relationships as the reviewer mentions, we are not assuming such additional knowledge. The uniform label distribution is the maximum entropy distribution among categorical distributions, and thus, regularizing towards it is a natural choice for increasing uncertainty. We note that regularizing towards the uniform distribution is a standard choice that has been successful in several prior works [1,2,3].

Reviewer comment (Theory):
The authors need to discuss the practicality of the assumption "\(D^\delta_{k}\) and \(D^\delta_{unk}\) are disjoint" in Proposition 4.2 since the proposed method follows the corresponding theoretical guidance. In other words, is this assumption strong? The unknown/misclassified points (I follow your definition of "unknown") in selective classification could be those hard points from some classes that intrinsically overlap with each other, then "known" and "unknown" are not disjoint. In OOD detection, now that we have the assumption of "disjoint", why do we still bother with the unlabeled data? Why not generate auxiliary data around but separable from ID?

Author response:
> Practicality of the assumption that known and unknown datasets are disjoint. The unknown/misclassified points (I follow your definition of "unknown") in selective classification could be those hard points from some classes that intrinsically overlap with each other, then "known" and "unknown" are not disjoint. In OOD detection, now that we have the assumption of "disjoint", why do we still bother with the unlabeled data? Why do not generate auxiliary data around but separable from ID?

For selective classification: even though the known and unknown examples come from the same classes, they can be disjoint partitions of the overall distribution.
For OOD detection: it’s not that any disjoint distribution works equally well; confidence minimization performs best when the uncertainty set is, in addition to being disjoint from the known data, close to the hard examples that the model may see at test time. Being separate from ID by itself does not make a dataset a useful uncertainty dataset.

Moreover, there are prior work that use synthetic outlier data: VOS [4], NPOS [5] and Dream-OOD [6]. We have provided comparisons to these methods on CIFAR-10 and CIFAR-100 as OOD datasets. Our method outperforms them in most cases, showing the efficacy of using the unlabeled auxiliary set.

| ID Dataset / Network | Method | SVHN | | LSUN (Crop) | | iSUN | | Texture | | Places365 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| | | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) |
| CIFAR-10 /ResNet-18 | | | | | | | | | | |
|| VOS | 96.37 | 15.69 | 93.82 | 27.64 | 94.87 | 30.42 | 93.68 | 32.68 | 91.78 | 37.95 |
|| NPOS | 97.64 | 5.61 | 97.52 | 4.08 | 94.92 | 14.13 | 94.67 | 8.39 | 91.35 | 18.57 |
|| DCM-Softmax |  99.7 (0.1) | 0.4 (0.3) | 98.6 (0.8) | 6.6 (3.0) | 99.7 (0.1) | 0.6 (0.2) | 97.1 (0.2) | 14.8 (0.3) | 92.4 (0.3) | 32.6 (2.1) |
|| DCM-MaxLogit |  99.8 (0.1) | 0.3 (0.1) | 98.7 (0.7) | 6.0 (3.0) | 99.8 (0.1) | 0.5 (0.2) | 97.1 (0.2) | 14.9 (0.4) | 92.5 (0.3) | 34.4 (2.1) |
|| DCM-Energy | 99.8 (0.1) | 0.1 (0.1) | 98.8 (0.7) | 5.3 (3.6) | 99.8 (0.1) | 0.1 (0.1) | 97.1 (0.2) | 16.1 (1.1) | 92.5 (0.3) | 35.6 (2.2) |
| CIFAR-100 /ResNet-34 | | | | | | | | | | |
|| VOS |  73.11 | 78.50 | 85.72 | 59.05 | 82.66 | 72.45 | 80.08 | 75.35 | 75.85 | 84.55 |
|| NPOS |  97.84 | 11.14 | 82.43 | 56.27 | 85.48 | 51.72 | 92.44 | 35.20 | 71.30 | 79.08 |
|| Dream-OOD | 87.01 | 58.75 | 95.23 | 24.25 | 99.73 | 1.10 | 88.82 | 46.60 | 79.94 | 70.85 |
|| DCM-Softmax |  99.3 (0.2) | 1.8 (0.9) | 98.6 (0.3) | 8.7 (1.9) | 99.3 (0.2) | 2.3 (1.4) | 88.5 (0.5) | 46.6 (2.7) | 78.6 (0.4) | 67.7 (2.3) |
|| DCM-MaxLogit |  99.3 (0.2) | 2.0 (1.1) | 98.7 (0.2) | 7.3 (1.9) | 99.3 (0.2) | 2.3 (1.4) | 88.5 (0.5) | 46.9 (3.0) | 78.6 (0.4) | 68.8 (1.8) |
|| DCM-Energy |  99.2 (0.2) | 2.2 (1.2) | 98.9 (0.2) | 5.2 (2.0) | 99.4 (0.2) | 1.9 (0.8) | 89.3 (0.6) | 48.6 (3.3) | 78.3 (0.6) | 68.9 (1.9)|

Reviewer comment (Theory):
The theoretical guidance is not that clear: Are the two theorems practically useful and how do the authors capitalize on these theorems in the experiments? In particular, as per Line 2, Page 5, what kind of "appropriate threshold" is used? Did the authors use the value of the left-hand side of the inequality in (4)?

Author response:
> The theoretical guidance is not that clear: Are the two theorems practically useful and how do the authors capitalize on these theorems in the experiments? 

Yes, the two theorems are practically useful. We explain their practical implications in the last part of Section 4. They show that it is important to use the right uncertainty dataset, one that includes examples close to the unknown examples that the model may encounter at test time.

In our experiments (Section 6), we capitalize on these findings by using the following uncertainty datasets: (1) unlabeled ID and OOD examples for OOD detection, and (2) misclassified validation examples for selective classification. These choices are validated by our results. We hope this clarification helps.

> In particular, as per Line 2, Page 5, what kind of "appropriate threshold" is used? Did the authors use the value of the left-hand side of the inequality in (4)?

Our empirical evaluation considers different thresholds of confidence, as is standard in prior selective classification works [7,8,9,10,11]: Acc@90 measures with a threshold such that the model predicts on 90% of the data, AUC is an aggregate metric that considers all thresholds, etc. 

Yes, we use a version of the LHS of (4), since all of our evaluation is based on the interplay between confidence (=MSP) and accuracy.

Reviewer comment (Theory):
\(\epsilon\) in Proposition 4.2 and the involved proof: The exact value of \(\epsilon\) depends on the model performance or \(\mathcal{L}(\theta)\), how can we allow \(\epsilon \leq \frac{1}{2N} \left( \frac{M-1}{(1+\lambda)M} \right)^2\) to conclude the inequality (15)? What is \(M\) in (15)?

Author response:
>  $\epsilon$ in Proposition 4.2 and the involved proof: The exact value of $\epsilon$ depends on the model performance or $L(\theta)$, how can we allow $\epsilon \leq \frac{1}{2N} \left(\frac{M - 1}{(1 + \lambda)M}\right)^2$ to conclude the inequality (15)?

The reviewer’s statement, “the exact value of $\epsilon$ depends on the model performance…”, is incorrect. $\epsilon$ is a free variable, and the lemma claims that there exists _some_ $\epsilon$ such that (8) is true.

This is a standard trick in proving inequalities. We know that (14) holds for _all_ $\epsilon > 0$. We can set $\epsilon$ to any positive value; by setting it to a value lower than $\frac{1}{2N} \left(\frac{M - 1}{(1 + \lambda)M}\right)^2$, we get equation 15.

Reviewer comment (Writing):
Please consistently add a comma after "i.e."

Author response:
> Please consistently add a comma after "i.e."

Thanks for the suggestion: we’ve added commas after all instances of “i.e.” in the revised version (in red text).

Reviewer comment (Writing):
What is (5) used for?

Author response:
> What is (5) used for?

(5) in the appendix states the DCM objective before we move on to the propositions and proofs. It is implicitly used in all later equations that involve the loss (6, 9, 10…)

Reviewer comment (Theory):
For the theretical part, what can we say when the following does not hold "(1) if the auxiliary set contains unknown examples similar to those seen at test time, confidence minimization leads to provable detection of unknown test examples". Specifically, what if the auxiliary set DOES NOT contain unknown examples similar to those seen at test time. It is important to know the theretical property in this case.

Author response:
> For the thoretical part, what can we say when the following does not hold "(1) if the auxiliary set contains unknown examples similar to those seen at test time, confidence minimization leads to provable detection of unknown test examples". Specifically, what if the auxiliary set DOES NOT contain unknown examples similar to those seen at test time. It is important to know the thoretical property in this case.

In the absence of any assumptions about how the auxiliary dataset relates to the test distribution, we cannot make any theoretical claims about the effect of confidence minimization on test inputs. In this scenario, Proposition A.1 still informs us about the resulting model’s behavior on the auxiliary data distribution: DCM will make the model more conservative in the sense that the predictive probability will be upper-bounded by the true probability distribution. How this affects the test distribution is a matter of generalization from auxiliary->test, which we would need further assumptions to study. We note that empirically, unrelated auxiliary distributions still help in making the model more overall conservative [1,2], though not as much as more relevant auxiliary data as in ours.

Reviewer comment (Experiments):
The experiment results are shown in CIFAR. I would be more interested in experiments on larger scale datasets with foundation models. For example, CLIP on ImageNet. Nowadays, the interest of the community has shifted to foundation models. I believe the paper can benefit from this aspect.

Author response:
> ‘The experiment results are shown in CIFAR…’

We thank the reviewer for their suggestions. We have run DCM on a ViT-B/16 model pre-trained on ImageNet-1K, and tested on 4 large OOD datasets, including iNaturalist and Places365. Our method outperforms all baselines on all 4 OOD datasets, including recent baselines such as VOS [3] and NPOS [4], by a large margin.

| Methods | iNaturalist |  |  SUN | | Places | | Textures | |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) |
| Fort et al.,MSP | 54.05 | 87.43 | 73.37 | 78.03 | 72.98 | 78.03 | 68.85 | 79.06 |
| VOS | 31.65 | 94.53 | 43.03 | 91.92 | 41.62 | 90.23 | 56.67 | 86.74 |
| VOS+ | 28.99 | 94.62 | 36.88 | 92.57 | 38.39 | 91.23 | 61.02 | 86.33 |
| NPOS | 16.58 | 96.19 | 43.77 | 90.44 | 45.27 | 89.44 | 46.12 | 88.80 |
| DCM-Softmax | 2.6 (0.5) | 99.2 (0.1) | 32.9 (1.5) | 94.2 (0.2) | 35.9 (1.8) | 93.8 (0.3) | 11.2 (1.0) | 97.9 (0.1) |
| DCM-MaxLogit | 1.8 (0.4) | 99.4 (0.1) | 27.5 (1.4) | 94.9 (0.2) | **32.5 (2.8)** | 94.5 (0.3) | 8.2 (0.8) | 98.3 (0.1) |
| DCM-Energy | **0.5 (0.2)** | **99.6 (0.1)** | **24.5 (1.7)** | **95.8 (0.2)** | **30.8 (3.0)** | **95.4 (0.3)** | **4.3 (0.6)** | **98.8 (0.1)** |

[1] Deep Anomaly Detection with Outlier Exposure, ICLR 2019

[2] Energy-based Out-of-distribution Detection, NeurIPS 2020

[3] VOS: Learning What You Don't Know by Virtual Outlier Synthesis, ICLR 2022

[4] Non-parametric Outlier Synthesis, ICLR 2023

Reviewer comment (Novelty):
There isn't much technical novelty on top of OE. The method is mainly about selecting a new uncertainty dataset, which is a fine direction to explore, but the approach is technically simple and possibly not substantial enough for ICLR.

Author response:
While our method is similar to OE in that both methods minimize confidence, we note the following key differences between our work and OE:
We analyze the role of the uncertainty set both theoretically and empirically.
We extend this confidence minimization framework to selective classification, which prior works such as OE do not explore.
Our method outperforms prior works that share our data assumptions (WOODS, ICML 2022 [1]; ERD, UAI 2022 [2]) while being simpler and computationally cheaper to run. Our method requires fewer algorithm-specific hyperparameters than WOODS [1] (1 for DCM, 8 for WOODS) and 5x reduction in compute compared to the ensemble-based ERD [2].

Reviewer comment (Experiments):
There are numerous more recent baselines, e.g., Virtual Outlier Synthesis. It would be good to include some of these.

Author response:
Thank you for the suggestions. 

We added comparisons to VOS [3], NPOS [4] and Dream-OOD [5] on CIFAR-10 and CIFAR-100 to Appendix H of our revised paper. DCM outperforms these methods by 2.5% on CIFAR-10 and 2.9% on CIFAR-100, in terms of OOD detection AUROC, averaged over 5 OOD datasets each.

| ID Dataset / Network | Method | SVHN | | LSUN (Crop) | | iSUN | | Texture | | Places365 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| | | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) | AUROC (↑) | FPR@95 (↓) |
| CIFAR-10 /ResNet-18 | | | | | | | | | | |
|| VOS | 96.37 | 15.69 | 93.82 | 27.64 | 94.87 | 30.42 | 93.68 | 32.68 | 91.78 | 37.95 |
|| NPOS | 97.64 | 5.61 | 97.52 | 4.08 | 94.92 | 14.13 | 94.67 | 8.39 | 91.35 | 18.57 |
|| DCM-Softmax |  99.7 (0.1) | 0.4 (0.3) | 98.6 (0.8) | 6.6 (3.0) | 99.7 (0.1) | 0.6 (0.2) | 97.1 (0.2) | 14.8 (0.3) | 92.4 (0.3) | 32.6 (2.1) |
|| DCM-MaxLogit |  99.8 (0.1) | 0.3 (0.1) | 98.7 (0.7) | 6.0 (3.0) | 99.8 (0.1) | 0.5 (0.2) | 97.1 (0.2) | 14.9 (0.4) | 92.5 (0.3) | 34.4 (2.1) |
|| DCM-Energy | 99.8 (0.1) | 0.1 (0.1) | 98.8 (0.7) | 5.3 (3.6) | 99.8 (0.1) | 0.1 (0.1) | 97.1 (0.2) | 16.1 (1.1) | 92.5 (0.3) | 35.6 (2.2) |
| CIFAR-100 /ResNet-34 | | | | | | | | | | |
|| VOS |  73.11 | 78.50 | 85.72 | 59.05 | 82.66 | 72.45 | 80.08 | 75.35 | 75.85 | 84.55 |
|| NPOS |  97.84 | 11.14 | 82.43 | 56.27 | 85.48 | 51.72 | 92.44 | 35.20 | 71.30 | 79.08 |
|| Dream-OOD | 87.01 | 58.75 | 95.23 | 24.25 | 99.73 | 1.10 | 88.82 | 46.60 | 79.94 | 70.85 |
|| DCM-Softmax |  99.3 (0.2) | 1.8 (0.9) | 98.6 (0.3) | 8.7 (1.9) | 99.3 (0.2) | 2.3 (1.4) | 88.5 (0.5) | 46.6 (2.7) | 78.6 (0.4) | 67.7 (2.3) |
|| DCM-MaxLogit |  99.3 (0.2) | 2.0 (1.1) | 98.7 (0.2) | 7.3 (1.9) | 99.3 (0.2) | 2.3 (1.4) | 88.5 (0.5) | 46.9 (3.0) | 78.6 (0.4) | 68.8 (1.8) |
|| DCM-Energy |  99.2 (0.2) | 2.2 (1.2) | 98.9 (0.2) | 5.2 (2.0) | 99.4 (0.2) | 1.9 (0.8) | 89.3 (0.6) | 48.6 (3.3) | 78.3 (0.6) | 68.9 (1.9)|