Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper provides theoretical analysis formalizing the intuition of mining hard outliers for improving the robustness of OOD detection. Motivated by the theoretical analysis, it proposes Adversarial Training with informative Outlier Mining (ATOM) and achieves SOTA results on both clean and perturbed OOD inputs with extensive experiments compared to a bunch of baselines. It also provides an ablation study on the effectiveness of adversarial training, which further proves its effectiveness. I think it's a good paper overall and should be accepted.
review_title: This paper provides a theoretically motivated method Adversarial Training with informative Outlier Mining (ATOM), which can improve the robustness of OOD detection. It conducts extensive experiments and achieves impressive results under a broad family of OOD evaluation tasks.

final_score: rating: 7: Good paper, accept
review_content: 1. The paper presents a lot of theory but insufficient evidence. It only employs limited image data (SVHN, CIFAR variants). The paper should be clear that the scope is limited to well-known image datasets only. This is because the approach is dependent on auxiliary data which is available for the image datasets. It is not clear if the approach might be more generally applicable to (say) network traffic, credit card transactions, natural language, etc. It would be better to include other types of data and along with auxiliary data generated through more generic means.

2. The presentation of the theory in Section 3 is not proper and makes it too simplistic. A better presentation should first present the general result first (rather than leave it to Appendix) and then show how that applies to a specific case such as Gaussian distribution.

3. Propositions 1, 2, 3: There seems to be a disconnect: what is the significance of n_0 in Proposition 1? What is its connection to the algorithm?

If we try to do a ballpark estimate with reasonable range of values: n_0 ~ 100, d ~ 50, \epsilon = 1/2, then the statement basically says that if n < 2 (optimistically), then we will have large errors. This type of limit on n is not very useful in practice as it is too loose. In fact, it seems that Proposition 1 would only be useful if d is very large. The paper should clarify that.

By and large, the Propositions 1, 2 and 3 are not helping much as the messages they are trying to convey (that auxiliary data helps and large number of samples are more accurate) are already well known. They do not add new knowledge. These might be moved to appendix.

4. Appendix B2, the general error bound (Proposition 5): The assumption is that d(Q_x, U_x) is small. The proposition basically states that if Q_x is similar to U_x, then a small FPR on U_x will also have a small FPR on Q_x. This is rather trivial. It is *much* harder in practice to get a U_x that is similar to Q_x. A more interesting case would be to show that even if U_x was not very similar to Q_x, then is the FPR within reasonable limit?
review_title: An out of distribution data detector that selects informative OOD data during training so that the performance is good and robust.

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper attempts to formulate the task of robust OOD detection by expanding the usual definition of OODs, and then presents a theoretical analysis in the simple setting of a Gaussian data model by porting results from adversarial defense. An experimental comparison is also presented. 

Pro:

+ The paper addresses a very important problem which is growing in importance. A number of approaches have been recently proposed to address this problem that attempt statistical, explanation-based, and relational/semantic techniques. The paper takes a simple approach of class augmentation and then picking hard OODs to solve this problem. 

Cons:

- Why is it surprising that the "majority of auxiliary OOD examples may not provide useful information to improve the decision boundary of OOD detector"? This seems to be rather obvious both for the toy case of Gaussian model or in general. It is well known that hard examples help train. The triplet loss function and other methods have been used for years based on this insight. The paper appears to present this obvious observation as the central finding of the paper.  For e.g. in the context of embedding, see how finding hard examples has been found to be useful in https://openaccess.thecvf.com/content_ICCV_2017/papers/Yuan_Hard-Aware_Deeply_Cascaded_ICCV_2017_paper.pdf 

- The paper appears to be mixing a lot of different problems to formulate a very generic notion of OOD. Typically, OODs have been used for out of distribution datasets (SVNH for CIFAR10), new classes on which model was not trained (leave some classes out of CIFAR10), or transforms such as rotations or other transformations of inputs. The paper mixes adversarial examples to it and then considers even further compositions. While there is significant effort to break OOD by building a useful taxonomy of it to refine the sources of uncertainty and address these in a principled way, the paper creates a monolithic problem of detecting when inputs don't belong to the training distribution for any reason. This creates a rather ill-defined mathematical problem because modeling the exact distribution of the training data is quite infeasible. The theoretical treatment of the experiments in the paper fail to convince the reviewer that there is an advantage in merging these separate problems into a single problem.

- The theoretical treatment are directly borrowed from http://papers.nips.cc/paper/9298-unlabeled-data-improves-adversarial-robustness.pdf where the treatment was for adversarial examples - since, the paper's definition of OOD includes these, the previous results are directly applicable. It would help to make this obvious in the main text instead of having this reference in the appendix. The connection is very strong. Also, this reviewer currently does not see the challenge in lifting the result from the reference to the theoretical analysis in this paper (please see question below). 

- The experimental evaluation is limited and does not meet the usual set of experiments reported for OOD detection which makes it difficult to understand the empirical benefit of the presented approach. 

- While there are other methods that also use "exposure to outlier" in OOD detection, it is debatable such an approach would be useful in practice, particularly, if one also included adversarial examples. But given past literature on this topic, the reviewer is less concerned about this aspect. 

In summary, the idea of using hard examples is not new. The definition of OOD to include adversarial examples is not well motivated. The theoretical results are borrowed without significant extension from literature on adversarial defenses. The experimental results do not meet the expectation of a paper on this topic to judge its empirical value against existing approaches. 

Questions for the author:

- Why is it helpful to merge different problems of adversarial example detection, detection of data away from training distribution due to new classes, detection of corrupted inputs into a single problem? Does a uniform treatment yield some new insight or enable some particular approach? Does it beat the state of the art in any of these sub-problems? 

- Can you elaborate on the nature of "natural OODs" in the experiments so that it is easier to understand comparison with ODIN, Mahalanobis, etc methods? They perform differently when OODs are because of new datasets or because of held-out classes. This "natural OOD" is better split into at least these two different types before evaluating the methods. 

- How is the theoretical analysis presented here not a trivial application (from adversarial setting to a superset definition of OOD) of results in http://papers.nips.cc/paper/9298-unlabeled-data-improves-adversarial-robustness.pdf  and https://arxiv.org/pdf/1804.11285.pdf ? Could you help identify the challenge and the novelty in this extension? 

The paper currently appears to be building on the premise that if we think of adversarial examples as a subclass of OODs, theoretical results from adversarial examples are applicable to OODs. A simple supervised approach using hard examples is then presented to solve this problem. Its empirical evaluation appears to behind the state-of-the-art in OOD detection as well as adversarial defense, but mixing both does not make this any stronger a paper. In its current form, the paper appears to need better motivation and identification of novelty. 

After discussion with authors
------------------------------------------

Some of the concerns of the reviewer have been addressed and the reviewer is raising the score to reflect it. The paper still has some major concerns preventing the reviewer from recommending acceptance of the paper: 

- Similarity of the theoretical analysis to https://proceedings.neurips.cc/paper/2019/file/32e0bd1497aa43e02a42f47d9d6515ad-Paper.pdf  and https://arxiv.org/pdf/1804.11285.pdf .  

- Hard negative mining is pretty standard in many learning domains. 

Both of these issues can be addressed by a better review of the related work and more accurate identification of the novelty in the paper. 

Given the limited theoretical novelty, a more robust empirical evaluation establishing the value of the extreme outliers against state of the art approaches would have been useful. Also, if the authors want to investigate theoretical results on OODs, one challenge is the lack of a formal definition of OODs. 

The paper is a good work in progress but not yet ready for publication. The reviewer is hopeful that the above suggestions will make the paper stronger.  
review_title: Needs better motivation and identification of novelty 

final_score: rating: 3: Clear rejection
review_content: In this paper the authors propose a method for training a classifier to be more effective at OOD (out of distribution) detection. Many OOD detection methods work by utilizing an auxiliary dataset as examples of OOD-ness. This is the approach taken in this paper and OOD is trained as being a k+1 classification class.  When training the OOD class the proposed method allows for adversarial perturbation of the OOD examples to help improve training. This is a pretty common technique in deep learning, see for example "Deep Robust One Class Classification."  Finally the main novelty of the method proposed by the authors is to sort a collection of OOD examples and sort according to the OOD score of the current model and use the "qNth" to be presented as OOD examples during the next epoch during training. The authors term this "Informative Outlier Mining." The authors demonstrate that this method works well experimentally.

The authors present some theoretical justification for their approach via a very toyish analysis of a OOD detection with a Gaussian data model.

While the proposed method seems to work reasonably well I find the novelty of this method to be too low to be interesting being more of a small trick rather than deserving of being the topic of an entire paper. I do think, however that this idea, in combination with some other OOD tricks could be an interesting paper.
review_title: Simple approach for improving OOD detection.

Meta-review: In this paper, the authors studied a robust method for detecting out-of-distribution (OOD) instances. OOD instance detection is an important practical problem, and multiple reviewers recognized the proposed approach is interesting. However, it was the common opinion of several reviewers that the main theoretical analysis was imported from existing studies, and the novelty is not sufficiently high. It was also observed that the relationship between the proposed method and closely related studies was not properly discussed. Although this point has been improved in the revision, a reviewer and area chair still concern that enough evidence is not provided for some of the points the authors claim as advantages over existing studies. Although the proposed method is interesting and could be an important contribution to the ICLR community, the current paper needs non-trivial revision before publication.
Decision: Reject