Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary:
----------------
The goal of this paper is to evaluate methods for detecting out-of-distribution samples in a more comprehensive fashion than prior work. To this end, it distinguishes between three different cases of OOD samples:

(1) detecting irrelevant inputs: defined in this case as being "out of concern" for a task of interest, e.g. when classifying dogs is the task of interest, irrelevant inputs would be images of food, plants, birds, and cars

(2) detecting novel classes: e.g. if the task of interest is classifying 15 types of dog, a novel class would be an unseen breed of dog

(3) detecting domain shift: that is detecting samples that belong to a known class but stem from a different generating distribution due to image corruptions (a la Hendrycks &  Dietterich, 2019)  or a different imaging setup/setting (a la Saenko et al., 2010)

An impressive number of experiments are carried out in which the same set of OOD detection methods are compared across these different scenarios. From the results, it is concluded that using a pre-trained network improves OOD detection (in line with previous results), and that not all methods perform well across the different settings, but one particular method does: cosine similarity (Techapanurak et al., 2019; Hsu et al., 2020).

Strengths & Weaknesses:
----------------------------------
Given the dizzying amount of papers continually being released in any ML subfield, any attempt to step back and evaluate recent methods on even footing are always welcome in my book and can provide more value than adding yet another method to the literature. I appreciate the effort to distinguish between different cases of OOD detection as well as the amount of experiments with a good selection of methods including some very recent ones.

Despite the above, I have reservations about this paper:

While there are a lot of experiments, the novel conclusions are sparse and very little space is afforded for a discussion of these: a single paragraph on page 8. It's a bit difficult to see the forest from the trees. 

Looking beyond the individual results, the key insight here to me is that there are different kinds of OOD detection and that it's important to consider these different settings when evaluating a method. However, Hsu et al. (CVPR '20) also distinguish between two settings in their evaluation: semantic and non-semantic shift. These results are unfortunately not discussed in this submission. To be clear, here the distinction is a bit more nuanced. The first two settings (1) "irrelevant inputs" and (2) "novel classes" correspond to different kinds of semantic shift (the latter to fine-grained classes, e.g. dog breeds), and the last setting "domain shift" corresponds to non-semantic shift. While I agree that (1) and (2) are real and distinct practical scenarios, I don't think either implies a specific degree of "semantic closeness". What is "novel" and what is "irrelevant" is application-dependent. I would not tie "novel" and "irrelevant" to certain sets of classes, but instead focus on the effect of "semantic closeness" on OOD performance as a more nuanced way of looking at semantic shift. 

Now a detailed discussion of the experiments themselves:

The experiments on "detection of irrelevant inputs" (3.1) confirms results from Hendrycks et al. (2019) that using a pre-trained network is better for OOD detection than training from scratch. Given the choice of datasets, the OOD detection is almost perfect (above 95%) except when the in-distribution is the food dataset. This leads me to question the choice of datasets, because I'm certain that OOD detection remains a difficult problem these almost perfect results notwithstanding.

The experiments on "detection of novel classes" (3.2) find that within the food dataset, four methods perform almost identically well (Baseline, Calib, MC dropout, Cosine), and outperform ODIN* and Mahalanobis. The conclusion is similar from the Dog experiment, except that Cosine performs the best by some margin. This contradicts the results in Hsu et al. 2020, where it is shown that both ODIN* and a variant of Mahalanobis significantly outperform Baseline across different settings. Here it is chalked up to the similarity between in-distribution and out-distribution images.

There are also a couple of problems with the experimental setting: The dataset of dog images contains 25 classes, some of which overlap with the Imagenet classes. This violates the OOD aspect. The authors explain that they "intentionally leave this overlap to simulate a similar situation that could occur in practice", but this really isn't convincing. In the case of the food dataset on the other hand, "eight classes [out of 101] contained in the ImageNet dataset" are removed. Perhaps it is inconvenient to retrain an ImageNet classifier without the offending dog classes, but then maybe another dataset should have been selected. This dataset contamination was not discussed in section 3.1, despite the use of the same dataset. Was this not an issue?

The final set of experiments on "detection of domain shift" (3.3) were confusing to me. I found the description of the problem formulation as well as the experimental setting (3.3.1, first part of 3.3.2) hard to followm especially the second paragraph of 3.3.1. Here also the goal is now to predict the classification error from the OOD scores, which departs from the setting considered for the first two sets of experiments. I would recommend cutting out section 3.3.2 and spending more time on analysing the results as I don't see what these experiments add to the paper.

Also, a new method is suddenly introduced (PAD*) and I'm not sure why. The way it's trained also appears to contradict the following statement in the introduction "we follow the argument shared by many recent studies [...] that OOD detection methods should not assume the availability of explicit OOD samples at training time". PAD* involves "train[ing] a binary classifier using portions of D_s and D_t to distinguish the two".

Conclusion:
----------------

A lot of effort went into conducting a large number of experiments with the goal of comparing several OOD detection methods across different scenarios. While I appreciate this effort and agree that it makes sense to consider these different scenarios when comparing methods, I don't think there are sufficiently novel insights to warrant acceptance with very little discussion of the results.

Evaluating OOD detection on fine-grained vs. non-fine-grained classes is interesting, and I would have liked to see more exploration of this. Unfortunately, the choice of datasets for these experiments makes the task appear to be too easy in one case, and in the other case is not truly OOD due to overlap with ImageNet classes.

Given that this is an evaluation of existing methods with little analysis, I would have been more inclined to accept if this paper established a convincing benchmark that could be easily adopted by researchers in this area, but due to the aforementioned problems this is not really the case. Perhaps one could consider DomainNet (Peng et al. ICCV '19) as a dataset that would support such an analysis at different semantic levels together with the non-semantic component. Additionally, one could add the image corruptions proposed by Hendrycks & Dietterich.
review_title: Many experiments, not enough analysis/insights

final_score: rating: 3: Clear rejection
review_content: ##########################################################################

Summary:
 
The paper empirically analyzes the evaluation framework of the current OOD detection systems for the image recognition task, specifically the evaluation described in [1] using Max-softmax and calibrated confidence. They motivate the paper by the necessity of having better evaluation for OOD detection to be reflective of real world scenarios. The addressed problem is interesting and valuable for the field as many of the defined OOD datasets, and evaluation metrics may not cover many real-world scenarios. They specifically addressed three scenarios, inputs that i) are irrelevant to the task ii) are from novel classes and iii) are from another domain (domain shift), which for the first 2 scenarios, they only evaluate them as unseen classes and not distinguish between them. Based on my understanding of the paper, they compare 5 OOD detection methods from the literature, suggest a few test datasets/scenarios and conclude using cosine similarity is consistently favorable for evaluation, and the choice of using confidence-based methods in case of domain shift detection scenarios.					

##########################################################################	
[Q]: questions, and comments on the paper. [S] strength [W] weakness
		
Q1: Questions about the choices of the datasets:

1.1. What are the reasoning for the choices of datasets? 
1.2. Why are these datasets considered reflective of real world scenarios? For example, why didn’t you analyze the models on some OOD datasets like Imagenet-A, -O, -P.

Q2: Could you elaborate more on the specific contributions of the paper?

Q3: What does “genuine OOD samples” mean in section 2.2?

Q4: There is no related work part In the paper.

Q5: Many recent and relevant papers are not cited or compared against. As an example, it is mentioned in the Domain shift detection [2] study in NLP, that is not the focus of the paper. However, there are multiple recent works in Computer Vision also addressing the domain or distributional shift in vision scenarios. In particular, for the example surveillance system scenarios mentioned in page 2, there are datasets and papers addressing the domain shift that should be cited such as [3 (a review paper), 4] or [5]. 

Q6: The following statement from the paper is inaccurate as a general statement. Which kind of pre-training contributes to performance improvement? In addition, there are many scenarios where pre-training or fine-tuning lead to reduced performance on seen classes or on the original data distribution. 
“(i) Using a pre-trained network always contributes to performance improvements, confirming the study of Hendrycks et al [1]. (2019). ”

Q7: For the evaluation using fine-tuning, the impact of the results on the seen classes (the current classes in the dataset), are not re-evaluated. In many cases, fine-tuning in a new distribution, contributes to reduced performance in seen classes or former distribution. I recommend checking the literature of generalized zero-shot learning and specifically the use of harmonic mean as a measurement.

##########################################################################
Reason for the score: 

S1: The paper is well-motivated and addresses an interesting problem. 

S2: I believe there is room for improvement.

W1: The contributions of the paper are not clear. The ablation studies to compare the approaches are not 	sufficient. 

W2: I am not convinced the current set up can be sufficient for generalizable decisions on choice of evaluations.

W3: Choices of the datasets for comprehensive guidance is unclear.

W4: There is no related work part  in the paper and multiple recent and relevant papers are not being compared against or cited.	
					
##########################################################################		 	 	 		
References:					
[1] Hendrycks and Kevin Gimpel, A baseline for detecting misclassified and out-of-distribution examples in neural networks, ICLR 2017 

[2] Elsahar et al, To annotate or not? predicting performance drop under domain shift, IJCNLP 2019.

[3] Toldo et al, Unsupervised Domain Adaptation in Semantic Segmentation: a Review, CVPR 2020 

[4] Zhang et al, Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes, CVPR 2017.

[5] Ganin et al, Unsupervised Domain Adaptation by Backpropagation, ECCV 2016.

review_title: Important problem but the paper is not ready

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: #### Summary
This paper is about a comparison of methods for out of distribution detection on image classification. The authors compare along three different lines: Irrelevant inputs, novel classes, and domain shift. The results show that cosine similarity consistently outperforms other methods across all sub-tasks.

The comparison made in this work is novel, as most OOD detection methods are evaluated on standard academic datasets (CIFAR10/100, SVHN, etc), which could hide issues that are presented in more realistic scenarios.


#### Reasons for Score
I believe that this paper touches a subject of importance to practitioners and researchers, the performance of OOD detection methods in more realistic settings and with three different sub-tasks inside the OOD detection task.
The evaluations are made correctly, in a variety of datasets, ensuring the robustness of the conclusions that were made.
My only concerns with this paper are the lack of comparisons with ensembles, which could change which method works best.


#### Pros
- The evaluation is comprehensive, with a variety of datasets, properly selected, ensuring robust results and conclusions. There is also a variety of tasks inside out of distribution detection (new inputs, novel classes, and domain shift).
- The evaluation produces strong conclusions, with one method (cosine similarity) clearly outperforming the rest. This provides good evidence for practitioners to select OOD detection methods in the future.
- The evaluation is made with a practical point of view, motivated by real-world examples, that differ from purely academic benchmarks. This is very useful for practitioners.
- A nice appendix that provides detailed results that can be used for future comparison.


#### Cons
- I believe there is a missing reference that is very relevant for this work, namely Ovadia et al. "Can You Trust Your Model’s Uncertainty? Evaluating
Predictive Uncertainty Under Dataset Shift" NeurIPS 2019. Many methods evaluated in Ovadia et al. like ones relying on variational inference are not evaluated in this work, so I suggest to take a look and include additional baselines. Note that Ovadia et al. also defines corruption methods for evaluating dataset shift.
- I think there are missing methods to compare, for example ensembles. The paper mentions that MC-Dropout "is an approximation of BNNs that is computationally more efficient than en ensemble of networks". I do not think this is true, ensembles and MC-Dropout are equally slow in predicting a forward pass, but an ensemble usually performs better in OOD detection and produces higher quality uncertainty (see Lakshminarayanan et al. 2017, Figure 3 and Ovadia et al. 2019). If MC-Dropout is used for comparison, then Ensembles should also be used since generally ensembles outperform MC-Dropout in terms of uncertainty quality. Note that this might change which method works better, or even create new baselines, as ensembling can be combined with cosine similarity OOD detection.


#### Questions for Rebuttal Period
Why were ensembles not considered as one of the evaluated methods?


#### Minor Issues
- In Section 3.1, please mention on which dataset are models pre-trained, I assume it is on ImageNet, but this needs to be clearly specified.
- Please mention in the Appendix that number of forward passes used in MC-Dropout
- I think method should be separated into two groups, one where OOD detection capabilities are given by the quality of uncertainty (like Bayesian NNs, MC-Dropout, Ensembles, etc), and other methods where an score is produced specifically for OOD detection and are not direct measures of uncertainty (not probabilities). This could shed a light on why methods work so differently.
- Sections 2.2 and 3.3.1 could be rewritten to clearly describe the OOD score produced by each method, for example in some cases the ID score is describe, how is the OOD score derived in these cases? This is important since an OOD score is used in Section 3.3.
- For probability-based methods like MC-Dropout, entropy of the output probabilities can also be used as an OOD score.
- In Figure 3, the x labels for Maha (Sum) are too close to each other and might hinder reading, I recommend to rewrite as 25K/50K/75K or rotate them a bit.
- For more clarity and a self-contained work, the authors could add a summary description of each method in mathematical terms to the appendix.

review_title: good evaluation of practical OOD detection, only minor issues

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper provides an empirical evaluation of various OoD methods . The authors distinguish three different operating scenarios:

+ Irrelevant inputs: in this case, the inputs  simply need to be rejected.
+ Novel classes: detecting inputs that are of the same overall category as that used in training, but from an unseen class of that category.
+ Domain shift, where measurement artifacts and other disturbances cause a shift in the measured p(x) even though the underlying inputs belong to known classes

Multiple methods are compared, using different image datasets to compare OoD detection performance using the AUROC as the evaluation metric.

Pros 

+ Paper is reasonably well written, and easy to follow
+ Distinguishing different types of OoD scenarios is a clarifying aspect and useful in this line of work.
+ Shows which methods work under what scenarios.

Cons
+ The biggest concern with this type of work is lack of novelty.  The results are not surprising (as the authors themselves have noted). Fine-tuning/pre-training gives a strong performance boost for OoD detection (as shown in the work of Hendrycks et al 2019) and cosine similarity has been shown to be better than softmax for OoD detection (as shown in the work of Techapanurak et al. (2019); Hsu et al. (2020)).  Further, there is no real discussion or insights into why certain methods work in some scenarios and not others. This results in a rather underwhelming paper for  the reader, and while there are useful takeaways from the paper, as it stands, it does not meet the bar of ICLR acceptance. 

References
* Dan Hendrycks, Kimin Lee, and Mantas Mazeika Using pre-training can improve model robustness
and uncertainty. In Proceedings of the International Conference on Machine Learning, 2019.

* Engkarat Techapanurak, Suganuma Masanori, and Takayuki Okatani. Hyperparameter-free out-ofdistribution detection using softmax of scaled cosine similarity. arXiv:1905.10628, 2019.

* Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. In Proceedings of the Conference on Computer Vision and Pattern Recognition, 2020.
review_title: Experimental evaluation of OoD methods, but mostly a confirmation of previous work

Meta-review: This paper proposes an OOD evaluation framework under three categories: irrelevant input detection, novel class detection, and domain shift detection. As with several reviewers, the AC recognizes the importance and effort to distinguish between different cases of OOD detection, as well as the amount of experimental comparison across several prominent methods in literature (MSP, MC-dropout, cosine similarity, ODIN, Mahalanobis).  

Despite being well-motivated, three knowledgeable reviewers find the paper not ready yet for publication at ICLR. The AC recommends a rejection, given the standing major concerns from the reviewers. The AC is hopeful that the paper can be significantly improved by 

- sufficiently discussing and highlighting the novel insights of the results. 
- a more rigorous definition of  "novel" vs. "irrelevant" inputs. There seem to be overlapping definitions between what Hsu et al. considered vs. this paper. In particular,  Hsu et al distinguish i) samples of a novel class but in a known domain, called semantic shift (S), and ii) samples of known class in a novel domain, called non-semantic shift (NS), both of which are reconsidered in this paper. Therefore, the novelty of this submission is more precisely to distinguish within the category of semantic shift. The AC agrees that this might deem some more rigorous measurement and definition of "semantic closeness". 
- The AC also finds the evaluation of domain shift in Section 3.3.2 may be potentially misleading the community, as it falls out of the standard OOD scope. The notion of common corruption is closer to the robustness problem (which is how ML model predictions changes w.r.t some delta changes in the input space). The changes may not be substantial enough to be "out-of-distribution".  
Decision: Reject