Paper: SEE-OoD: Supervised Exploration for Enhanced Out-of-Distribution Detection (ICLR-2024)

Reviewer comment (Experiments):
I understand that your approach is suitable for low-data regime (especially when the number of OOD samples is low). It would be interesting to visualize the curves in figures 2 and 3 also as a number of several ratios (IND/OOD). How many number of OOD samples do you need to generate in each case?

Author response:
Thanks for your valuable suggestions about presenting the results. The reason why we choose to use numbers of OoD samples, instead of InD/OoD ratios, as X-axis is because, in practice, the OoD samples at hand are usually very scarce (ex, unseen manufacturing defects, rarely seen obstacles in autonomous driving), whereas the size of InD training data can grow easily. This makes the ratio metric not as informative as numbers. As for the number of OoD samples that we generate, we generate OoD samples in each iteration of the iterative training process and use those to regularize the model. For detailed information, we kindly refer the reviewer to Appendix D.2.2, where $B_G$ denotes the number of OoD samples generated in each iteration.

Reviewer comment (Experiments):
Why do you distinguish between two experimental scenarios (balanced vs imbalanced OOD classes)? It is assumed that OOD data is unlabeled. So, it does not matter how many data is in each class. Please clarify this aspect.

Author response:
We are glad that you point out that OoD data are unlabeled anyway. We totally agree with you. Although the OoD labels are not used in the model training, the inherent differences between OoD samples from different classes can affect the model performance. And there might be a distribution shift between observed and test OoD samples. The idea here is to control what has been observed by our method in the training time. Under the balanced sampling strategy in regime I, the testing OoD samples resemble part of training OoD samples. However, in practice, people only possess very limited knowledge about OoD samples so it is reasonable that some types of OoD samples are not observed. Under the imbalanced sampling strategy in regime II, the test OoD samples may be different from the observed OoD samples.

Reviewer comment (Experiments):
Another aspect which is not clear to me is how do you define the imbalanced regime for OOD, i.e. you mention 'only a few classes are observed'). I thought imbalance refers to different ratios between the samples of OOD classes or with respect to the samples in the IND classes. What is in each case the ratio between majority classes and minority classes?

Author response:
In the imbalanced regime, we deliberately leave one class (i.e., type) of OoD samples unobserved. For example, in the SVHN Within-Dataset experiment, classes 8 and 9 are considered OoD, but during training, only samples from class 8 are observed. We employ this setting to assess the generalizability of our method. For specific details regarding experimental setups and the number of samples included during training, we kindly refer the reviewer to Table 2 and Appendix E.

Reviewer comment (Theory):
If the generative adversarial approach is unconditional, why the case of imbalanced scenario is relevant? OOD data is unlabeled anyway.

Author response:
We are glad that you point out that OoD data are unlabeled anyway. We totally agree with you. Although the OoD labels are not used in the model training, the inherent differences between OoD samples from different classes can affect the model performance. And there might be a distribution shift between observed and test OoD samples. The idea here is to control what has been observed by our method in the training time. Under the balanced sampling strategy in regime I, the testing OoD samples resemble part of training OoD samples. However, in practice, people only possess very limited knowledge about OoD samples so it is reasonable that some types of OoD samples are not observed. Under the imbalanced sampling strategy in regime II, the test OoD samples may be different from the observed OoD samples.

Reviewer comment (Evaluation):
How do you evaluate the quality of the synthetic OOD samples? Some quantitative and qualitative analysis is indicated.

Author response:
Thank you for highlighting this crucial aspect. In our paper, Theorem 1 and Theorem 2 enable us to offer theoretical guarantees on the quality of generated outliers. We have demonstrated that, theoretically, the generated outliers will not be part of InD clusters. Regarding the experimental setup, we deliberately refrain from controlling the generation of outliers. This choice aligns with the primary goal and contribution of our method, which is to explore OoD spaces. As long as the generated outliers do not become part of InD, our objective is considered accomplished.

Reviewer comment (Experiments):
The proposition of the Within-Dataset OoD detection is undoubtedly a fresh perspective. However, its universal applicability remains a concern. This setting, which treats different classes within the same dataset as OoD with respect to one another, might not capture the diverse and multifaceted nature of real-world OoD scenarios. To many, this approach might appear as an analysis of variations within a single domain rather than a genuine OoD situation.

Author response:
We would like to thank Reviewer BP77 for pointing this out. As you alluded, the primary objective of introducing this new setting is to assess OoD detection accuracy when InD and OoD samples closely resemble each other. The resemblance between InD and OoD samples introduces additional challenges compared to the conventional between-dataset detection setting.

We agree with your comment on the universal applicability of Within-Dataset detection tasks.

However, in various real-world situations, InD and OoD samples may resemble each other. For instance, in anomaly detection, anomalies can manifest as small deviations from the InD samples (Sun et al., 2023). This insight motivated the Within-Dataset experiments, where OoD samples originate from classes within the same dataset. The goal was to create scenarios where InD and OoD samples come from the same dataset and emphasize our model's ability to effectively detect such cases.

That said, it is crucial to clarify that the within-dataset detection setting serves as an additional metric when compared to other methods. We have demonstrated that our method significantly outperforms baselines in the traditional Between-Dataset setting, particularly in terms of generalizability to test OoD samples not encountered during the training process. This also indicates that our method effectively leverages existing limited OoD samples and explores potential OoD spaces more efficiently compared to the baselines.

Reviewer comment (Experiments):
The scalability of the proposed method, especially when confronted with vast and diverse datasets, remains ambiguous.

Author response:
We kindly refer the reviewer to our general response. Following the settings in Liu et al. (2020) we added one more Between-Dataset experiment, CIFAR10-Texture with more InD classes, and one more baseline OE (Hendrycks et al., 2018), to illustrate the scalability and effectiveness of our method.

In addition, we would like to communicate the following perspective with the reviewer.

### Are the difficulties of OoD detection tasks determined by the complexities of datasets?

It is critical to point out that the complexity of the InD and OoD datasets is not the only factor that influences the difficulties of OoD detection tasks. According to Liang et al. (2020), Lee et al. (2018), and Liu et al. (2020), one of the central challenges for OoD detection tasks in classification models arises from the fact that DNN models are usually over-confident towards InD samples.

For example, OoD detection on a simple InD dataset like the FashionMNIST Within-Dataset experiment, where the classification accuracy can easily reach 97% (i.e. high predictive confidence for InD data), is not necessarily easier than CIFAR100-Texture Between-Dataset experiments done in Liu et al., 2020, where the classification accuracy is only about 79% (i.e. low predictive confidence). Indeed, this explains why the baseline methods, including OE (Hendrycks et al., 2018), Energy Finetuning (Liu et al. 2020), and VOS (Du et al., 2022) fail to achieve desirable performance on the simpler datasets but can achieve decent performance on those seemingly complex datasets like CIFAR100-Texture according to Liu et al. 2020.

That said, we have included another set of experiments on CIFAR10-Texture experiments to further highlight the benefits of our approach.

Furthermore, from our experiments, we notice that another critical factor influencing the difficulties of OoD detection tasks is the proximity between InD and OoD samples. This observation motivated the Within-Dataset experiments, where OoD samples come from classes within the same dataset. The idea here was to create situations where InD and OoD samples highly resemble each other and highlight the capability of our model to effectively detect such cases.

While in practice, this situation may not be universally applicable, it does represent various real-world situations, such as anomaly detection, where anomalies can manifest as small deviations from the InD samples or their features (Sun et al. 2023).

Reviewer comment (Presentation):
The paper does not provide visualizations of the generated outliers, which could offer more intuitive insights into the model's behavior and decisions.

Author response:
Thank you for your question on the interpretability of the generated outliers.

Given that our approach aims at the exploration of OOD spaces, the generated images need not look like observed OOD images. That said, one can add a distance regularization term within our objective to penalize the distance between OoD samples and the generated outliers. With such penalty, the generated images will resemble observed OoD samples. For your interest, we provided two generated images (for the MNIST experiment, where we treat classes 1 and 7 as OoD) with the distance regularization term. We can see meaningful images are generated in these cases, where interpretability is maintained.

- [Example 1](https://github-production-user-asset-6210df.s3.amazonaws.com/82349855/283628983-99d1d7bc-c707-4f95-87bd-c758fb38f200.png)
- [Example 2](https://github-production-user-asset-6210df.s3.amazonaws.com/82349855/283629133-beaa0b9f-d336-460c-bcef-10b8095d124f.png)

However, by imposing the penalty term, the exploration of OoD spaces will be constrained. Indeed, we found that such a constraint will decrease generalization power. This observation also agrees with existing literature showing that such a regularization term led to reduced generalizability (Wang et al. 2021, Liu et al. 2020). Therefore, we decided to remove the penalty term in our final formulation.

Reviewer comment (Experiments):
While the paper presents results on certain datasets, it would benefit from testing on larger and more diverse datasets to ensure the method's generalizability and robustness.

Author response:
We kindly refer the reviewer to our general response. In addition, we would like to communicate the following perspectives with the reviewer.

### Are the difficulties of OoD detection tasks determined by the complexities of datasets?

It is critical to point out that the complexity of the InD and OoD datasets is not the only factor that influences the difficulties of OoD detection tasks. According to Liang et al. (2020), Lee et al. (2018), and Liu et al. (2020), one of the central challenges for OoD detection tasks in classification models arises from the fact that DNN models are usually over-confident towards InD samples.

For example, OoD detection on a simple InD dataset like the FashionMNIST Within-Dataset experiment, where the classification accuracy can easily reach 97% (i.e. high predictive confidence for InD data), is not necessarily easier than CIFAR100-Texture Between-Dataset experiments done in Liu et al., 2020, where the classification accuracy is only about 79% (i.e. low predictive confidence). Indeed, this explains why the baseline methods, including OE (Hendrycks et al., 2018), Energy Finetuning (Liu et al. 2020), and VOS (Du et al., 2022) fail to achieve desirable performance on the simpler datasets but can achieve decent performance on those seemingly complex datasets like CIFAR100-Texture according to Liu et al. 2020.

That said, we have included another set of experiments on CIFAR10-Texture experiments to further highlight the benefits of our approach.

Furthermore, from our experiments, we notice that another critical factor influencing the difficulties of OoD detection tasks is the proximity between InD and OoD samples. This observation motivated the Within-Dataset experiments, where OoD samples come from classes within the same dataset. The idea here was to create situations where InD and OoD samples highly resemble each other and highlight the capability of our model to effectively detect such cases. While in practice, this situation may not be universally applicable, it does represent various real-world situations, such as anomaly detection, where anomalies can manifest as small deviations from the InD samples or their features (Sun et al. 2023).

Reviewer comment (Experiments):
The experiments only cover the simple datasets and settings as shown in Table 2. The “within-dateset” setting and the used datasets are simple. The dataset used in the “between-dataset” setting is also not complex enough to validate the methods. This is also why the proposed method can easily achieve very high performance after seeing real OOD samples. More “between-dataset” settings should be considered as in more recent OOD detection papers, such as (Liu et al., 2020).

Author response:
We address this question in our "general response". Specifically, we followed your suggestion and included one more Between-Dataset experiment on the CIFAR10 and Texture dataset introduced in Liu et al. 2020. In addition, we would like to communicate the following perspective with the reviewer.

### Are the difficulties of OoD detection tasks determined by the complexities of datasets?

It is critical to point out that the complexity of the InD and OoD datasets is not the only factor that influences the difficulties of OoD detection tasks. According to Liang et al. (2020), Lee et al. (2018), and Liu et al. (2020), one of the central challenges for OoD detection tasks in classification models arises from the fact that DNN models are usually over-confident towards InD samples.

For example, OoD detection on a simple InD dataset like the FashionMNIST Within-Dataset experiment, where the classification accuracy can easily reach 97% (i.e. high predictive confidence for InD data), is not necessarily easier than CIFAR100-Texture Between-Dataset experiments done in Liu et al., 2020, where the classification accuracy is only about 79% (i.e. low predictive confidence). Indeed, this explains why the baseline methods, including OE (Hendrycks et al., 2018), Energy Finetuning (Liu et al. 2020), and VOS (Du et al., 2022) fail to achieve desirable performance on the simpler datasets but can achieve decent performance on those seemingly complex datasets like CIFAR100-Texture according to Liu et al. 2020.

That said, we have included another set of experiments on CIFAR10-Texture experiments to further highlight the benefits of our approach.

Furthermore, from our experiments, we notice that another critical factor influencing the difficulties of OoD detection tasks is the proximity between InD and OoD samples. This observation motivated the Within-Dataset experiments, where OoD samples come from classes within the same dataset. The idea here was to create situations where InD and OoD samples highly resemble each other and highlight the capability of our model to effectively detect such cases.

Reviewer comment (Experiments):
Many strong OOD detection methods do not use OOD samples in training (in the more general and real setting) but have strong modeling. It is straightforward to introduce the known OOD samples in the training process easily. The authors should consider this case and conduct comparisons, especially considering that the experimented settings and datasets are very simple.

Author response:
Please refer to our responses in the "general response", where we reiterate the main contributions of this paper. Secondly, we respectfully disagree with Reviewer Qajt's comment suggesting that incorporating OoD samples into the training process is straightforward. It is, instead a very challenging task that needs to be done carefully. There are three primary reasons for this:

- Methods that solely rely on existing OoD samples without exploring OoD spaces, such as Energy Finetuning (Liu et al., 2020), WOOD (Wang et al., 2021), and OE (Hendrycks et al., 2018), fail to detect OoD samples not encountered in the training process, leading to poor generalizability. The main pitch of our method is to effectively use observed OoD samples to perform iterative exploration of OoD spaces. Our experimental results in the imbalanced regime substantiate this claim.
- Existing methods that exploit OoD samples are susceptible to overfitting when OoD samples are scarce in the training dataset, as demonstrated in our experiments. In such cases, augmenting the limited OoD samples is desirable, particularly in the original image space rather than the feature space. This is because feature spaces may exhibit high overlap, even for images that appear vastly different.
- We agree with Reviewer Qajt that many methods that do not use OoD samples have strong modeling. However, their modeling approach is mainly based on operations (including exploration) within InD spaces (Du et al. 2022; Ming et al. 2022; Lee et al. 2017). In these cases, there is no straightforward method to incorporate OoD samples during the training stage. Introducing OoD samples naively using score regularization is likely to give rise to the aforementioned two problems.