Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: I read this paper with great interest. The authors propose an easy-to-understand, easy-to-implement baseline method for detecting when inputs to a ML model is out of distribution. The method involves augmenting the training dataset with  an out of distribution dataset and adding an additional class in the classification layer for out of distribution. The paper describes several experiments—some in computer vision, some in NLP—and then compares them to other OOD techniques. The results are comparable to other techniques, although the proposed technique is definitely simpler.

I believe that this is a direction worth exploring; however, I do feel that the authors could have provided some more insights into the following decisions:
1. What are the desired characteristics of the OOD dataset used to augment the training set? How and why was the tiny images dataset chosen to augment the training set? If we instead used the Gaussian dataset as the OOD training set, will the results have been similar?
2. How large should the OOD training set be? The paper used 100k images from the Tiny Images dataset for the vision tasks. Why 100k? What if we trained with 10k images instead? What should the balance between in-distribution and out-of-distribution images be in the training set?
3. Does the task performance remain the same? If not, then how does the OOD detector (task + OOD classifier) proposed in the paper compare against, say, a model that just learns to classify in-vs-out of distribution?

I look forward to hearing back from the authors and I’m open to changing my score.
review_title: Simple idea for an OOD baseline, but experiments need more analysis

final_score: rating: 4: Ok but not good enough - rejection
review_content: -- Summary: 
This paper presents experiments and results from using a reject-class in multi-class classification for the auxiliary task of out-of-distribution (ood) sample detection. Training requires unlabeled ood data disjoint from the training set. 


--  Review: 
1- The idea of using auxiliary task or reject-class for ood detection has been well explored in the past. Examples include Neal et al. [1] mentioned in the paper and recently by Mohseni et al [2] which is missing from the reference. This also has been used to improve representation learning in Zhang and LeCun [3]. 
 
2- Previous work accepted and recognized MSP score (with not additional data or hyperparameter) as the baseline for ood detection. What is your justification to propose your reject-class technique as the new baseline? 

3- A challenging test case for ood detection is training and testing on two similar datasets such as cifar10 and cifar100. The cifar10 as D-in for train and cifar100 as D-out for test (and vice versa) is a very common example. I wish we could see results on that. 

-- Strengths:
- The ood detection problem is an important and interesting topic
- Authors reviewed results and compared the proposed ood detection technique with multiple uncertainty-based techniques.


-- Weaknesses: 
- Lack of novel contributions.
- Experiments and results are limited. More recent papers studied the use of self-supervised learning (e.g. [4, 5] ) and contrastive learning (e.g., [6, 7,8]) to improve ood detection. 


[1] Neal, Lawrence, et al. "Open set learning with counterfactual images." Proceedings of the European Conference on Computer Vision (ECCV). 2018.'
[2] Mohseni, Sina, et al. "Self-Supervised Learning for Generalizable Out-of-Distribution Detection." AAAI. 2020.
[3] Zhang, Xiang, and Yann LeCun. "Universum prescription: Regularization using unlabeled data." Thirty-First AAAI Conference on Artificial Intelligence. 2017.
[4] Hendrycks, D., Mazeika, M., Kadavath, S., & Song, D. (2019). Using self-supervised learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems (pp. 15663-15674).
[5] Golan, Izhak, and Ran El-Yaniv. "Deep anomaly detection using geometric transformations." Advances in Neural Information Processing Systems. 2018.
[6] Tack, J., Mo, S., Jeong, J., & Shin, J. (2020). Csi: Novelty detection via contrastive learning on distributionally shifted instances. arXiv preprint arXiv:2007.08176.
[7] Winkens, J., Bunel, R., Roy, A. G., Stanforth, R., Natarajan, V., Ledsam, J. R., ... & Cemgil, T. (2020). Contrastive training for improved out-of-distribution detection. arXiv preprint arXiv:2007.05566.
[8] Liu, Hao, and Pieter Abbeel. "Hybrid discriminative-generative training via contrastive learning." arXiv preprint arXiv:2007.09070 (2020).
review_title: using a reject class for outlier detection 

final_score: rating: 5: Marginally below acceptance threshold
review_content: **Update after author response:** I have read the other reviewer's comments. My take is that at a high level the contribution of this paper is above the bar of ICLR, but the experiments aren't controlled enough so I vote for a weak reject.

Hendrycks et al propose encouraging high entropy predictions on the outlier exposure set, instead of classifying them into a reject class. Going further, Hendrycks et al claim "but we find that even with OE, classifiers with the reject option... are not as competitive". As I understand, Hendrycks et al are basically saying the (simpler) method in this paper does not work as well - but Hendrycks et al don't provide experimental evidence for this claim, it's merely stated. The original paper might in fact discourage practitioners from trying this approach, and instead using the high entropy approach. In fact, this was a question in my mind when I read Hendrycks et al last year.

Instead, this paper seems to show stronger results, with a more intuitive and simpler method (just classify the outlier exposure set into a separate class) that Hendrycks et al suggest doesn't work as well. Conceptually, the approach in Hendrycks et al also seems more brittle and there are distinctions between these two methods (e.g. see Vernekar et al 2019, “Analysis of Confident-Classifiers for Out-of-distribution Detection”).

So what's the potential practical impact? If this paper didn't exist, I suspect practitioners would use the method in Hendrycks et al, and not try the reject class method, because of that paper's claims. But with this paper, practitioners might use this method or try both, which seems like a good impact.

So barring problems with the experimental setup, I'd give the paper a 7 / accept, and so I'd encourage the authors to continue on with this work.

To me the decision hinges on the quality of the comparisons. I am inclined to agree with R1 on the quality of comparisons. Taking a closer look at their paper, they have no detailed discussion about the hyperparameters and experimental setup, which is key when the main contribution is a fine-grained comparison with Hendrycks et al. R1 raises a question about fine-tuning vs trained from scratch, and it does look like this paper trains from scratch whereas outlier exposure fine-tunes. The outlier exposure set is also different. While it is a smaller set in this paper, Hendrycks et al say "experiments in this paper often used around 1% of the images in the 80 Million Tiny Images dataset since we only briefly fine-tuned the models.".

Overall, I agree that the best thing would probably be for them to do a more careful  and controlled comparison, include all these details, and submit to the next conference. In my review I did mention that their comparisons were unclear, but they didn't take the chance to update their paper and misleadingly responded that "The OE method is closest to ours, so we are able to match their training regimen well", but as R1 points out there seem to be salient differences.

#########################################################################

Summary:

This paper tackles the problem of out of distribution detection. Concretely, they have an in-distribution set of inputs D_in and the goal is to reject inputs from an unknown set D_out as out of distribution, while accepting inputs from D_in as in-distribution. Like in the outlier exposure paper, they assume a known set of proxy out of domain distribution inputs \tilde{D}_out. The approach is to classify inputs in D_in into one of K classes, while classifying inputs from \tilde{D}_out into a “K+1”-th class. On a few vision and NLP datasets, they show better performance than outlier exposure and other methods.

#########################################################################

Reasons for score:

The main technical difference compared to outlier exposure is that they classify inputs from \tilde{D}_out into a K+1-th class, whereas outlier exposure enforces that the prediction over the K classes have high entropy if the inputs are from \tilde{D}_out. 1. One main concern is that they have other (hidden) differences from Hendrycks et al, for example I looked at their code and they use Mixup and a different \tilde{D}_out, so it is unclear if the gain is actually coming from the K+1-th class, or e.g. more modern data augmentation practices. They also don’t test on the most challenging image datasets from Hendrycks et al, in particular D_in = CIFAR-100, D_out = CIFAR-10 (they have distinct sets of classes). 2. This method was also proposed by Vernekar et al, “Analysis of Confident-Classifiers for Out-of-distribution Detection”.

I generally believe that this is the right thing to do, and seems simpler and more sound than enforcing high entropy, and Vernekar et al only have toy experiments. However, since the contribution is a small change to OE, to make this complete I believe they should have more complete experiments. In other words, I’m holding them to a higher bar experimentally, than if say their idea was novel or they had conceptual insights into the problem, because without more complete experiments the contribution to the community is limited. Note that unlike most works on OOD detection, they use more data, \tilde{D}_out, similar to outlier exposure. This is completely fine, but that's why I think simply having SOTA results is not quite enough to push this over the bar since there's only about 1 paper on this precise setup.

On the plus side, this work should at least convince practitioners to try both methods, K+1-th class and enforcing high entropy.

#########################################################################

Pros:

- Simple idea, and does better than competing methods on most baselines.

- Combination of vision and NLP datasets.

- Well written, clear and simple.

#########################################################################

Cons:

- I looked at this paper’s code, and Mixup is an additional difference compared to Hendrycks et al’s outlier exposure. There could be other differences I did not spot (e.g. training procedure? Optimization method). It’s great that the overall method does better, but I’d like to see at experiments on some datasets investigating what happens if you use the same procedure as OE except K+1-th class instead of entropy.

- Results on CIFAR-10 -> CIFAR-100, and CIFAR-100 -> CIFAR-10 would be good. Since the main contribution is experimental, it would be particularly compelling if a couple more datasets were added. E.g. the original OE paper is substantially more comprehensive. E.g. one combo could be Places365 -> ImageNet.

- No conceptual reason for why the K+1-th method does better (I am still happy to vote to accept without this, but this would make the paper more compelling). The earlier work by Vernekar et all doesn’t quite explain things for high dimensional data like images.


#########################################################################

Questions and things to improve:

If the comparisons to Hendrycks can be re-run with matching training procedure, and they show results on a few more cases (e.g. CIFAR-10 -> CIFAR-100, and CIFAR-100 -> CIFAR-10), I would seriously consider leaning towards an accept.

#########################################################################

review_title: Sensible idea, but needs more complete experiments

final_score: rating: 4: Ok but not good enough - rejection
review_content: - Summary:
This paper shows that introducing an abstention class for out-of-distribution (OOD) works well for detecting it when the in-distribution dataset is CIFAR and TinyImageNet is available during training as an OOD dataset.

- Reasons for score:
1. The proposed setting with a large OOD dataset has already been proposed by [Hendrycks et al.], and the proposed method has been experimented in [Lee et al. (a)] and [Dhamija et al.], so the technical novelty of this work is limited.
2. The experiments are not thoroughly conducted. The only value I can find in this paper is the empirical observation in a limited condition. More specifically, this paper found that adding an abstention class is better than prior methods when the in-distribution dataset is CIFAR and TinyImageNet is available during training as an OOD dataset. I am not sure this observation is consistent in other settings, so I recommend to conduct more thorough experiments, as done in [Hendrycks et al.]. Again, [Hendrycks et al.] considered a similar setting, but they proved the effectiveness of their method in image and natural language domains, with 7 in-distribution datasets and 3 large OOD datasets. However, even with more experiments, I am not sure this work is significant enough for publication in ICLR, because of the lack of novelty in both the experimental scenario and method.
3. The comparison is unfair. Performances of prior methods are borrowed from original works, but they are mostly experimented in settings different from this paper. In particular, some works like [Lee et al. (b)] and [Hsu et al.] considered training/validating without OOD data, because it is hard to assume to have a large OOD dataset covering all possible OOD in practice.

- Minor Comments:
4. Citation format issue: you can use \citet for noun and \citep for adverb.

[Lee et al. (a)] Training confidence-calibrated classifiers for detecting out-of-distribution samples. In ICLR, 2018.

[Lee et al. (b)] A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, 2018.

[Dhamija et al.] Reducing network agnostophobia. In NeurIPS, 2018.

[Hendrycks et al.] Deep anomaly detection with outlier exposure. In ICLR, 2019.

[Hsu et al.] Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data. In CVPR, 2020.

**After rebuttal**

I'd like to thank authors for their efforts to address my concerns. I didn't change my initial rating, due to the two main concerns below:

(1) To me, the main argument of this paper sounds "when a large (and maybe diverse) OOD is given, adding an OOD class to the classifier is better than baselines." Since the large OOD setting has already been proposed by [Hendrycks et al.], the only contribution of this work is on the empirical observation that the proposed method is better than baselines. While the observation is interesting, I think the contribution is not enough as a full ICLR paper at this point.

During the rebuttal period, R3 corrected it that "the main question investigated by the paper is how to best use the outlier exposure set," and this sounds better. However, authors didn't emphasize the setting but their method, such that their main argument is (if they intended to say as like what R3 understood) misleading. Training with a large OOD dataset like [Hendrycks et al.] is not common, and the observation in this paper is limited to this setting. However, the only statement about the setting I could find in the intro is that "as in Hendrycks et al. (2018), we uses additional samples of real images and text from non-overlapping categories to train the model to abstain, ..." i.e., rather than elaborating/emphasizing the setting (together with their method), they just cited a prior work.

In short, I recommend authors to rewrite abstract/intro as suggested by R3, to properly emphasize their contribution.

(2) The comparison is unfair, as authors didn't re-evaluate baselines in the same setting (they had to make it the same as much as possible) but just pasted numbers from original papers. Even the comparison with the closest prior work [Hendrycks et al.] is unfair, as the prior work fine-tuned the model while the proposed method trained the model from scratch.

Regarding the performance of similar methods evaluated in [Lee et al. (a)] and [Dhamija et al.], I think the main reason why they didn't get the same observation is on the size of OOD dataset, i.e., they didn't train their model with a large OOD dataset like [Hendrycks et al.] or this work. As this work claims, it might be true that when a large OOD dataset is available, adding an OOD class to the classifier is simply good enough.
review_title: The observation is interesting, but more experiments are required

Meta-review: This paper proposes a method for out-of-distribution (OOD) detection by introducing a K+1 abstention class for outliers, in addition to the in-distribution classes. While the method has shown promising performance compared to the Outlier Exposure (OE), the novelty is limited given the idea is almost identical to an AAAI'20 paper (Mohseni et al. 2020). In addition, several reviewers have raised concerns regarding the lack of fairness in the experimental setting. Authors are encouraged to address them in a future submission. 

The AC believes an interesting and valuable contribution to the community would be showing conceptual and theoretical reasoning for the pros and cons of using K+1 class vs. entropy regularization. Currently, the tradeoff between these two types of training objectives is not well understood. 

Overall, three knowledgeable reviewers in this area have indicated rejections. The AC discounted R2's rating due to the less familiarity in this area and lack of participation in the post-rebuttal discussion among reviewers. 

Lastly, the AC would like to greatly thank R1, R3, and R4 for the active engagement and participation in the paper discussion period. It was very helpful for the decision-making process. 





Decision: Reject