Paper: Guaranteed Out-Of-Distribution Detection with Diverse Auxiliary Set (ICLR-2024)

Reviewer comment (Novelty):
Mixup-based augmentation techniques are already considered in [a]. The manuscript should clearly outline the differences.

Author response:
PixMix stands out as an elegant and  effective data augmentation technique, introducing new complexity information into the training procedure through the utilization of fractals and feature visualizations, which comprehensively improves safety measures. However, there are several key differences between this approach and ours: 
(1) **Different Motivations:** Pixmix approach is introduced for multiple important purposes, e.g., enhancing OOD  robustness, ensuring prediction consistency, enhancing resilience  against adversaries, obtaining well-calibrated uncertainty estimates, and facilitating the detection of anomalous inputs. Our work focuses on theoretical analysis to show why and how the diversity of outliers impacts the OOD detection ability. As a result, we  apply mixup to increase the diversity of outliers and thus improve OOD detection capabilities. 
(2) **Different Mixup Goal:** The main role of mixup in Promix is to fuse the original images with complex fractals and feature visualizations, as fractals and feature visualizations do not belong to any particular class, this method does not change the semantics of the original data. However, mixup in our method is to combine outliers with different semantic information to generate new outliers, and the new outliers have different semantic information with the original data. Moreover, unlike pixmix, we do not perform any operations on the ID data, only on outliers.

Reviewer comment (Novelty):
Missing related works which replace auxiliary negatives with properly trained generative models [b,c,d].

Author response:
We thank the reviewer for pointing out these exciting works that use generative models to synthesize outliers for training OOD detectors. [b] is the first to propose using a generative model to generate outliers on  the classification boundary of ID data for training. [c] obtains the the synthetic outliers by sampling a generative model based on normalized flow that is trained alongside a dense discriminative model in order to produce samples at the border of the training distribution, which produces   better performance. [d] Building a discriminator on features computed by a closed-world  K-way network and carefully selected The GAN discriminators on some real outlier data, augmenting the outlier set with adversarially synthesized "fake"  data, which achieves good performance in open-set recognition. We will review these works in the manuscript.

[a] Dan Hendrycks, Andy Zou, Mantas Mazeika, Leonard Tang, Bo Li, Dawn Song, Jacob Steinhardt: PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures. CVPR 2022

[b] Kimin Lee, Honglak Lee, Kibok Lee, Jinwoo Shin: Training  Confidence-calibrated Classifiers for Detecting Out-of-Distribution  Samples. ICLR 2018

[c] Matej Grcic, Petra Bevandic, Sinisa Segvic: Dense Open-set  Recognition with Synthetic Outliers Generated by Real NVP. VISIGRAPP  2021

[d] Shu Kong, Deva Ramanan: OpenGAN: Open-Set Recognition via Open Data Generation. ICCV 2021

Reviewer comment (Theory):
Should the first term of Eq. 13 have D_inlier instead of D_aux?

Author response:
Regarding the error in Eq. 13, thank you for pointing it out. We have updated D_aux to D_id (Q2).

Reviewer comment (Theory):
Proof of Theorem 2, could you elaborate on the equality in the line above the Eq. 16?

Author response:
For the clarification on the equality above Eq. 16 in the proof of Theorem 2 (Q3), there was a notation error, which has now been corrected. The accurate representation is as follows:

$\text{max} ( \underset{h\in \mathcal H^*_{div}}{sup}\epsilon_{x\sim\mathcal P_{\mathcal X}}(h,h^*_{ood}),\underset{h\in \mathcal H^*_{aux}-\mathcal H^*_{div}}{sup}\epsilon_{x\sim\mathcal P_{\mathcal X}}(h,h^*_{ood}) )
=\underset{h\in\mathcal H^*_{aux}}{sup}\epsilon_{x\sim\mathcal P_{\mathcal X}}(h,h^*_{ood})$

Reviewer comment (Theory):
The authors are suggested to be more careful and rigorous with the theoretical terms used in the paper, such as the "generalization risk" is not rigorous in theory.

Author response:
**R1.** Thank you for the important suggestion. We have changed "generalization risk" to "generalization error."

Reviewer comment (Theory):
For theorem 2, did the authors consider the sample complexity of both the vanilla auxiliary outlier set and the diverse outlier set? The sample complexity and model complexity should play an important role in the bound during analysis for these two cases.

Author response:
**R3.** Thank you for raising questions regarding the complexity of samples and models. In our research, our primary focus lies in addressing the errors induced by distribution shift caused by auxiliary outliers that fail to fully cover the complexity of the real-world out-of-distribution (OOD) scenarios. This limitation stands as a critical factor constraining the performance of OOD detection. Issues related to generalization errors arising from model and sample complexities are more prevalent in traditional machine learning. However, in our specific task, such errors are often overshadowed by the disparities introduced by auxiliary outliers' inability to adequately cover the complexity of real-world OOD data. Therefore, in the context of our paper, we assume the model to be over-parameterized and the training data to be abundant, implying that the model can effectively fit the distribution of the training data [1]. We acknowledge your perspective, we will give a more comprehensive and precise analysis about how sample complexity and model complexity jointly influence the ultimate decision boundaries in future work.

Reviewer comment (Experiments):
For theorem 3, I am curious how well the h_mix compared with the predictor h_div.

Author response:
**R4.** We conducted experiments to demonstrate the performance comparison between $h_{mix}$, trained with mixed outliers, and $h_{div}$, trained with more diverse outliers. Specifically, we chose CIFAR-100 as the in-domain (ID) dataset. For $h_{mix}$, we extracted data from 250 classes in ImageNet-RC to serve as less diverse outliers, and applied mixup outliers during training. As for $h_{div}$, we utilized the entire set of classes (1000 classes) from ImageNet-RC as more diverse outliers for direct training. Additionally, we trained model $h$ by directly training on data from 250 classes as outliers from ImageNet-RC. The experimental results are as follows:

| model | outlier dataset | FPR $(\downarrow)$ | AUROC $(\uparrow)$ | AUPR_out $(\uparrow)$ | ID-acc $(\uparrow)$ |
| ---- | ---- | ---- | ---- | ----  |---- |
| $h$ | imagenet-RC (250 classes) | 23.01 ± 3.23 | 95.69 ± 0.53 | 96.26 ± 0.79 | 74.49 ± 0.28 |
| $h_{div} $ | imagenet-RC (1000 classes)| 18.82 ± 1.40 | 96.42 ± 0.28 | 96.45 ± 0.49 | 74.21 ± 0.25 |
| $h_{mix}$ | imagenet-RC (250 classes) | 18.25 ± 1.81 | 96.11 ± 0.35 | 98.24 ± 0.38 | 74.24 ± 0.33 |

From the experiments, it is evident that the model $h_{div}$, trained with more diverse outliers, outperforms the model $h$ trained without such diversity. However, after enhancing the diversity of outliers through mixup, the performance of $h_{mix}$ surpasses that of $h_{div}$. This validates the effectiveness of our approach and the underlying theoretical framework. The more intuitive comparison is illustrated in Figure 2(a) of our paper.

Reviewer comment (Theory):
It is still not intuitively making sense to me why does mixup can create diverse outlier examples. It will not change the data distribution much since it is only doing the interpolation work. What if the auxiliary outlier set is very constrained in the convex high-dimensional input space, would mixup increase the diversity of the outlier set significantly?

Author response:
**R5.** Thanks for the questions. 
(1) To understand why mixup can create diverse outlier examples, please note that: (1.1) The interpolation process of mixup brings about a significant alteration in the data distribution. Unlike traditional data augmentation methods, such as adding noise or rotation, which maintain semantic consistency with the original data, mixup operates as an out-of-manifold data augmentation method [2]. The soft labels used in mixup introduce probabilistic descriptions of similarity between the newly generated samples and the original class, resulting in samples that exhibit semantic inconsistency with the original data. Consequently, mixup-generated samples deviate from the original data manifold, leading to a notable change in data distribution. Considering our definition of diversity, these out-of-manifold samples generated by mixup extend diversity. (1.2) Mixup outliers can expand the high dimensional spatial coverage of auxiliary outlier data and enhance its diversity. Auxiliary outliers comprise data from diverse classes, indicating significant semantic distinctions among different categories, which implies that, in high-dimensional space, auxiliary outliers are clustered by category, with sparse distribution between outliers from distinct categories. Mixup interpolates between outliers with different semantic information belonging to distinct clusters, bridging the sparsity between these clusters. This process enables auxiliary outliers to cover a broader region in high-dimensional space, which means that mixup enhances the diversity of auxiliary outliers. (1.3) To provide a clearer understanding of this issue, we have included visualizations in Appendix, illustrating in the feature space why mixup can create diverse outlier examples.
(2) Actually, auxiliary datasets often exhibit sparsity in high-dimensional space and not constrained to a specific region. Assuming that in the worst case, auxiliary outliers are highly constrained within a convex high-dimensional input space, with all outliers being highly similar and belonging to the same class, mixup may not significantly increase data diversity. However, it is important to note that mixup does not adversely impact the model's performance under such conditions.

Reviewer comment (Experiments):
If possible, the authors are encouraged to provide empirical evidence on large-scale benchmarks. The current setting does not seem to suggest the diversity will create a performance improvement significantly.

Author response:
**R6.** We appreciate the suggestion to evaluate our approach on larger-scale benchmarks and provide the corresponding experiments under the setting in [3]. Our experimental setup involved utilizing 200 classes from ImageNet-1k (224x224 resolution) as the in-distribution (ID) class and an additional 800 classes as outliers. We conducted tests across five different out-of-distribution (OOD) test sets. Specifically, 'SSB-hard' and 'NINCO' were included as part of the near-OOD group for ImageNet-1k, while 'iNaturalist,' 'Textures,' and 'OpenImage-O' were classified under the far-OOD group. The experimental results are as follows:

| FPR95 ($\downarrow$) / AUROC ($\uparrow$) | NINCO | SSB_hard | Textures | iNaturalist | OpenImage-O | average |
|--- | ----- | -------- | ---   | ----------- | --------- | ------- |
|MSP | 78.90 / 80.24 | 76.25 / 81.11 | 67.64 / 81.97 | 76.12 / 82.77 | 78.19 / 80.90 | 75.42 / 81.40 |
| energy (w.o. outliers) | 73.56 / 84.12 | 68.86 / 84.91 | 52.55 / 86.78 | 67.68 / 87.66 | 72.76 / 84.18 | 67.08 / 85.53 |
| k+1 | 77.48 / 82.31 | 73.14 / 84.61 | 71.95 / 84.47 | 91.49 / 78.87 | 75.64 / 83.59 | 77.94 / 82.77 |
| OE | 67.78 / 84.85 | 65.07 / 85.81 | 51.31 / 88.92 | 65.72 / 86.64 | 62.75 / 86.71 | 62.53 / 86.59 |
| promix (k+1) | 50.37 / 90.42 | 43.99 / 91.76 | 55.62 / 88.91 | 45.30 / 92.33 | 40.31 / 92.80 | 47.12 / 91.24 |
| promix (OE) | **42.80 / 90.82** | **36.75 / 92.19** | **35.41 / 92.12** | **25.86 / 95.12** | **33.48 / 93.43** | **34.86 / 92.74** |

Among them, the MSP and energy methods were trained without using outliers, while the k+1 and OE methods were trained with outliers. To validate the versatility of our method, we adapted it to the OOD scores of k+1 and OE, denoted as promix(k+1) and promix(OE), respectively. The experimental results show that our method is naturally effective on large-scale datasets both on near OOD detection and far OOD detection. The large-scale setting provides a powerful validation that the diversity will create a performance improvement significantly.

Reviewer comment (Novelty):
Novelty Concerns: Applying mixup for OOD detection is not new. Many other works have explored this concept previously [1][2][3].

Author response:
Our contribution goes beyond employing mixup to improve OOD detection from the following perspectives.
(1) We first present a theoretical and empirical understanding demonstrating a more diverse set of auxiliary outliers is critical in improving OOD detection by lowering the upper bound of OOD detection error. Our findings could offer fresh theoretical insights for future OOD detection advancements.
(2) Building on the proposed theory, we develop a simple yet effective method by leveraging mixup to enrich outlier diversity with corresponding theoretical evidence. Note that unlike the previous mixup-based methods, our core motivation for introducing mixup is to improve the diversity of the auxiliary outlier dataset instead of designing novel mixup strategy. 
(3) Unlike the insightful methods proposed in [1, 2, 3] by applying mixup to ood detection, our main contribution is to explore a more diverse auxiliary outlier space. This focus sets our research different from existing ones.

Reviewer comment (Experiments):
The minor modification the authors propose to the original Mixup (explicitly using the existing model and selecting Mixup outliers with lower OOD scores) lacks clarity in its effectiveness. There doesn't seem to be theoretical justification or ablation studies to validate the efficacy of the modification.

Author response:
**Theoretical Perspective:** As the potential space for auxiliary outliers data can be excessively large, the majority of outliers uninformative for model regularization, which may lead to inefficient and insufficient exploration and exploitation of these outliers [4][5]. To mitigate this, we strategically select the mixed outlier samples that are closest to the in-distribution boundary. These samples are more likely to be misclassified as in-distribution (ID) data and need to receive more attention. In our theory, mixup can generate new outliers with semantics different from the original data, and this selection strategy can select the newly generated outliers that are easy to be misclassified by the model for training.
**Empirical Validation:** Ablation experiments were performed on this module in Table 2. We present the results here:
For cifar10 dataset:

|method |FPR $(\downarrow)$|AUROC$(\uparrow)$|AUPR$(\uparrow)$|ID-ACC|
|-|-|-|-|-|
|only mixup outliers| 3.28|99.23 |**99.45** |94.30 |
|mixup outliers + greedy sampling |**2.18** | **99.43** | 99.01|94.32 |

For cifar100 dataset:
|method |FPR $(\downarrow)$|AUROC$(\uparrow)$|AUPR$(\uparrow)$|ID-ACC|
|-|-|-|-|-|
|only mixup outliers| 13.33|97.35 |**98.66** | 74.28|
|mixup outliers + greedy sampling | **10.37** |**98.03** |98.63 | 74.26|

These results clearly illustrate that selecting Mixed outliers based on their out-of-distribution (OOD) score consistently enhances OOD detection performance compared to using only Mixup. This underlines the significant benefits of our proposed modification in improving the effectiveness of OOD detection.

Reviewer comment (Experiments):
The auxiliary dataset used for training is a downsampled version of ImageNet. Compared to the utilized ID datasets CIFAR-10 and CIFAR-100, its diversity seems significantly higher. Therefore, the OOD detection performance gains achieved with such an auxiliary dataset might not be wholly convincing, even though the methods authors compared against also adopt this approach.

Author response:
Thank you for the detailed and interesting comments. We would address this concern in the following ways:
(1) In practical applications, access to a more diverse auxiliary dataset is often possible, such as crawling data from the Internet. This approach allows to train the model with a more diverse auxiliary outlier dataset. Consequently, this setting has been adopted by many related studies within the field.
(2) While ImageNet-RC offers greater diversity compared to datasets like CIFAR-10/100, it still does not fully represent the vast and complex nature of true open-world out-of-distribution (OOD) data. This limitation highlights the need for even more diverse datasets to truly model the real-world data.
(3) We also experimentally confirm that our method is still effective even when the diversity of auxiliary outliers is not significantly higher than ID data. In the experiment "Q2 Reliability", we tested the effectiveness of our method under the outliers dataset with different diversity. As can be seen from Figure 2, on the cifar-100 dataset, when we restrict the diversity of the outliers dataset (using only 100 classes of the ImagenetRC dataset as outliers, the diversity of the outliers dataset is comparable to that of the ID dataset), the OOD detectors are still able to benefit from outliers. When our method is adopted to improve the diversity of outliers, the performance of the model is significantly improved (The FPR95 metric significantly decreased from 26% to 20%.), which also confirms the effectiveness of our method and theory.