Paper: ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms (ICLR-2024)

Reviewer comment (Novelty):
An important reference, OpenOOD v1.5 [1] is missing (released on arXiv in June 2023). Their evaluation results in the full-spectrum OOD detection setting (considering semantic-shifted and covariate-shifted samples together [2]) are also concrete evidences that current OOD detection methods, not restricting to post-hoc methods, are very sensitive to covariate shifts. This is actually presented as one of their major observations, and thus I believe it is necessary to discuss this work at least in the Related Work section.

Author response:
Thank you for pointing this out. We updated the related works section (section A of appendix) to include a pointer to OpenOOD v1.5 (blue text on pg. 14). The key distinction with our analysis is that we studied the effect of covariate shift independently and found that the baseline MSP is more robust towards covariate shifts than modern OOD detection algorithms. OpenOOD v1.5 performed the analysis in the full spectrum (ID combined with covariate shift) setting, which hid this fact, and only concluded that such setting poses a challenge for OOD detection. We updated section 4.3 and introduction to make this point more clear. Additionally, we would like to note that as you point out, this work has only appeared on arxiv and has not yet been published in peer-reviewed venues, thus we hope it does not affect assessment of novelty of our work as per ICLR guidelines.

Reviewer comment (Writing):
This is not really a weakness. I like the example in Figure 1 where "Animal, Vehicle" are ID, and training images of "Animal" is dog while test images could be cat. This actually points to a type of covariate shift called Subpopulation Shift [3]. I think the discussion on this example could be made more clear by explicitly discussing the relationship between OOD detection and Subpopulation Shift (although I agree that this could often times be application-dependent).

Author response:
This is a great suggestion and we updated the paper to reflect this in the “semantically-grounded covariate shifts” paragraph of Section 3.2 (bottom of pg. 4, highlighted in blue).

Reviewer comment (Experiments):
As demonstrated by NINCO work, human inspection is necessary for constructing clean OOD datasets (which is also recognized in this work). However, human inspection could be extremely costly, which I believe is the reason why NINCO itself is limited in size (a few hundreds or at most thousands of images). IN-OOD in comparison has a total of 31,807 images. I was wondering how thorough and rigorous the human inspection was for the "final review" of IN-OOD at this size. How many human inspectors were involved and how long did it take for the final review?

Author response:
Two rounds of manual human inspection are performed by the authors of the paper, first at the class level (20 hours), then at the image level (6 hours). We first manually inspected sample images from direct neighbors of ImageNet-1K classes and selected 1000 classes that are visually unambiguous to their ImageNet-1K neighbors. Then, we inspect the top 1000 images that are closest to ImageNet-1K in terms of ResNet-50 feature distance to filter out classes that are visually ambiguous with non-neighboring ImageNet-1K classes to arrive at the final 637 classes. Since the ImageNet-21K images are all validated by crowd workers from Amazon Mechanical Turk, we dedicated most of our time to class-level inspection, while still performing image-level analysis as an additional layer of reassurance. Please refer to Appendix Section H for more details on the construction of ImageNet-OOD.

NINCO was a smaller dataset designed to avoid data contamination of in-distribution classes. In comparison, in addition to avoiding data contamination, our dataset is designed to mitigate unwanted covariate shifts, which is crucial for our analysis.

Reviewer comment (Experiments):
While covariate shift robustness is desirable, the goal of semantic shift detection is also useful in many applications like open set recognition. The heavy focus on covariate shifts undervalues semantic shift detection.

Author response:
**W1: focus on covariate shift undervalues semantic shift**

One of the primary goals of our work is to actually curate a clean dataset to better evaluate new-class detection.  We directly emphasize the motivation of constructing ImageNet-OOD at the start of Section 3: “...new-class detection is still relevant in practice with the growing interest in adaptive learning systems.” Additionally, we delineate the difference between new-class detection and failure detection at the end of section 2 to emphasize the importance of semantic shift. We made some minor changes at the beginning of section 3 (highlighted in blue) to make this point clearer.

Reviewer comment (Evaluation):
More analysis could be provided on the characteristics of examples that lead methods to confuse covariate and semantic shifts.

Author response:
**W2: analysis on characteristics of examples that lead methods to confuse covariate and semantic shift**

This is a great suggestion. We conducted additional qualitative analysis (included in Section G of the appendix) and found that the OOD detection algorithm ViM tends to score texture-like images as more OOD when compared to MSP and the OOD detection algorithm ASH-b tends to score text-like images more OOD when compared to MSP. Additionally, we observe that text-like/texture-like images from ImageNet-OOD are more closely aligned with the domain of ImageNet-1K. Since ASH-b and ViM are the two best performing algorithms on OpenImage-O, these visual examples explain the disappearing performance gains when evaluating on ImageNet-OOD.

Reviewer comment (Novelty):
The focus of this paper is primarily on the dataset and analysis, which are undoubtedly meaningful aspects. However, the author fails to provide their own methods to improve the effectiveness of OOD detection tasks, which results in a lack of depth and contribution in this paper.

Author response:
Absolutely, we do not provide a new method for OOD detection. As we note in the abstract, our contributions are as follows: “we present ImageNet-OOD, a clean semantic shift dataset that minimizes the interference of covariate shift. Through comprehensive experiments, we show that OOD detectors are more sensitive to covariate shift than to semantic shift, and the benefits of recent OOD detection algorithms on semantic shift detection are minimal. Our dataset and analyses provide important insights for guiding the design of future OOD detectors.”
Our work follows a long line of works (please see section A of appendix) which provide detailed analyses and insights into existing paradigms and approaches, rather than developing new models.

Reviewer comment (Experiments):
The author's comparison methods lack some of the latest approaches. In recent conferences such as CVPR 2023, new methods have been proposed. Including these methods in the comparison would make the article more comprehensive.

Author response:
Integrating recent CVPR methods is an excellent suggestion. We added a new method Max-Cosine (Zhang and Xiang) from CVPR 2023 and also another method ASH-B (Djurisic et al.) from ICLR 2023. We kindly ask that you please note that works published in CVPR 2023 are within the four months prior to ICLR submission and therefore, are considered contemporaneous by the ICLR policy.


We included Max-Cosine and ASH-B in Table 1 (pg 8), integrated them into the summary of the results in Section 4.3, and added additional qualitative analysis in Section G of the appendix (all highlighted in blue).  Briefly, we find the same conclusion: the best performing detector is now ASH-B on OpenImage-O with improvement of 5.5 points on AUROC while the detector perform worse than the MSP baseline on ImageNet-OOD; Max-Cosine is the best detector on ImageNet-OOD but the improvement from the MSP baseline is still less than 1 point on AUROC.

Reviewer comment (Experiments):
Many methods may be sensitive to hyperparameters, so it would be beneficial to discuss the adjustment of hyperparameters when changing to new datasets.

Author response:
Regarding the hyperparameters, the methods we used are either hyperparameter free (MSP, Energy, Max-Logit, Mahalanobis, and the newly added Max-Cosine) or the methods are tuned towards ImageNet, our particular setup,  in the original paper of the method (ViM, KNN, ReAct, and newly added ASH-B). As a result, for fair evaluation, we used the best suggested hyperparameter based on the ablation studies performed in the original paper. We added this clarification at the end of the second paragraph of section 4.

Reviewer comment (Experiments):
**W1** Introducing separate datasets for assessing semantic and covariate shifts is relevant for roughly understanding OOD detection capabilities. Yet, in a real-case scenario, both semantic and covariate shifts will be present, and whether we wish to ignore one of the two should be specified through the training set (e.g. using a multi-environment setup - see [1]). If both semantic and covariate shifts are to be considered, then it is expected that the OOD detection algorithm will first identify the one generating the highest shift. (see question **Q1**)

Author response:
**W1, Q1: real-world scenarios have both covariate and semantic shift present. Have we considered distance measures between datasets?**

Incorporating dataset distance measurements is a great suggestion and something that we have previously investigated. We tried using the OTDD work you suggested, but unfortunately it was too memory intensive to fit on ImageNet, ImageNet-OOD, ImageNet-R, and ImageNet-Sketch due to the high resolution images and high number of classes (they defined a dataset as feature-label pairs and they only tested on low resolution datasets such as CIFAR-10/Tiny-ImageNet). Instead, we did try a common distance metric from the field of generative modeling: Frechet Inception Distance (FID), which represents the Wasserstein-2 distance in feature space. We did not find a consistent relationship between dataset distance and OOD detection performance. On CLIP embedding, we receive the following FIDs against ImageNet-1K images
- 1.36 for ImageNet-Sketch
- 0.99 for ImageNet-R
- 1.07 for ImageNet-OOD 

We observe a similar pattern on FIDs using embeddings of a ResNet-50 model trained on ImageNet-1K against ImageNet-1K images
- 21.8 for ImageNet-Sketch
- 13.4 for ImageNet-R
- 13.9 for ImageNet-OOD

From FID, we observe that ImageNet-R and ImageNet-OOD have similar distances while ImageNet-Sketch is significantly further away. Despite the fact that ImageNet-Sketch is significantly further than ImageNet-R, our results in Table 1 between ImageNet-OOD vs. ImageNet-R/ImageNet-Sketch have similar performance. Thus, we did not find any consistent relationship between dataset distances and OOD detection performance, and therefore, the susceptibility towards covariate shift cannot be explained by dataset distance. 

Regarding real-world scenarios and what to do with covariate shift, we build off of existing literature for the proper procedure for handling covariate shift. As outlined in the introduction and described at the end of section 2 under the “modern approach” and “classic approach” subheadings (references provided in more detail in section A of appendix), OOD detection algorithms have two popular use cases: new-class detection and failure detection. The goal of our paper is to provide a dataset that can better assess an OOD detection algorithm’s performance for the task of new-class detection (i.e., in the context of semantic shift). Under the formulation of the task of new-class detection in previous works, covariate shift is not considered OOD, and therefore, should be not detected by an OOD detection algorithm. We added another sentence at the beginning of Section 3 (highlighted in blue) to further clarify.

Reviewer comment (Experiments):
**W2** The curation steps can benefit from an in-depth analysis. See questions **Q2**, **Q3**, **Q4** and **Q5**

Author response:
**W2, Q2-Q3: why repeat the hypernym and hyponym removal process for ImageNet-1K classes in addition to most general semantic classes? Why does the violin and viola example exist after removing semantically-grounded shifts?**

You are correct that this process does not need to be repeated for the original ImageNet-1K classes. We divided them in separate sections in the paper for the sake of clarity but we thank you for this suggestion and further clarified in the end of the paragraph describing “semantically-grounded covariate shifts” (Section 3.2, bottom of page 4, highlighted in blue). 

Regarding the violin and viola example:  ImageNet-1K contains the classes violin and cello, which share a common parent “bowed stringed instrument”. Therefore, violin class would still remain in the most general decision boundary. As a result, viola would still create issues with the violin class.

**W2, Q4-5: have we considered an automated process such as CLIP zero-shot embeddings to establish a threshold for removing visual similarities?**

This is a great suggestion but we were solving a different issue with our manual class selection. For our use case, we performed manual inspection of the classes and some of their respective images to eliminate visual ambiguity, not identify classes with high visual similarity. Using CLIP zero-shot classification would not be able to distinguish between the two. We illustrate two examples from experimenting with CLIP zero-shot classification on ImageNet-OOD images with label of the image and ImageNet-1k classes:
- The CLIP model confuses images of Capote (ImageNet-OOD class), which is a type of coat made of wool/fabric with a hood and typically worn by Native Americans, with images of Trench Coat (ImageNet-1K class), which is a long waterproof coat. This is an example of two objects that are visually similar but not visually ambiguous. If we manually inspect the images of Capote and Trench Coat, we can clearly see the subtle but undeniable difference between the objects.
- The CLIP model confuses dump cart (ImageNet-OOD class) vs. wheelbarrow (ImageNet-1K class). Both objects are visually similar but there is a clear semantic difference between the two objects: one wheel vs. two wheels.

Visual ambiguity, in contrast to visual similarity, arises due to flaws in the data annotation process of the original ImageNet-21K dataset rather than properties of the data itself. In the ImageNet-21K dataset, data labels are derived from queries to the internet search engine and verified using human annotators from Amazon Mechanical Turk. They ask human annotators the question “is X in the image?” based on the query used. This can create labeling mistakes from visually ambiguous objects such as the violin/viola example shown in Figure 1. Given a picture of viola without any context clues to its size, it is difficult for a human to distinguish the image from violin vs viola. Since the human annotators were asked “is this a violin” or “is this a viola” instead of contrasting between the two, the labeling can go either way depending which question was asked. We circumvent this issue with manual inspection of the classes using our prior knowledge and intuition about object properties to select 637 classes for ImageNet-OOD that would not have such an issue. This way we can be a lot more confident we did not inadvertently include images potentially corresponding to ImageNet-1K classes in the ImageNet-OOD set. 

Reference: Section 3 of Deng, Jia, et al. "ImageNet: A large-scale hierarchical image database." CVPR 2009

Reviewer comment (Evaluation):
**W3** Conclusion of Sec.4.2, where randomly initialized models are considered for testing OOD detectors. Here, the authors conclude that there is a bias towards detecting covariate shifts even for untrained models. This observed bias is most probably justified by the inductive bias of the considered ResNet-50 model. Randomly initialized models are more sensitive to specific covariate shifts, but this observation cannot be generalized to any covariate shift. (see question **Q6**)

Author response:
**W3, Q6: have we considered other types of covariate shifts on random models? How does semantic shift compare?**

This is a great question. We actually considered such covariate shifts in Figure 4 of the paper with ImageNet-R (https://github.com/hendrycks/imagenet-r). ImageNet-R is a dataset consisting of ImageNet-1K objects in different domains such as video games, paintings, etc. 
We did not include background shifts as such datasets do not currently exist for ImageNet-1K to the best of our knowledge and are difficult to curate. This is because ImageNet-1K, which is the most commonly-used in-distribution dataset in the field, is an object-centric dataset that contains no information on the background diversity. Due to the nature of ImageNet-1K, our paper focuses on object-level covariate shifts, which is an orthogonal issue to background-level covariate shifts.

Regarding semantic shift, this is an excellent suggestion and we updated the paper to include semantic shift results with ImageNet-OOD in Figure 4 (page 7). The results revealed that OOD detection algorithms on randomly-initialized models do perform very close to random chance when evaluated with the semantic shift of ImageNet-OOD. This is in contrast to covariate shift shown with ImageNet-R or ImageNet-C, which the OOD detector has a bias either towards detecting as ID or OOD.