Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This paper presents an OOD detection method that combines robust feature extraction, OOD feature generation and adversarial training. This Adversarially Trained Discriminator aims to provide a robust OOD detection result that can adapt to larger perturbation sizes. The experiment is conducted on different detector-extractor pairs with respect to applying attack on different data, and the result shows the proposed method consistently outperforms the SoTA methods.

strengths_and_weaknesses: ++ This paper specifically pays attention to generating OOD samples and applying Adversarial Training on it to improve the OOD detection performance.

++The paper is well written and easy to follow.

-- Table 3 is not self-contained. It does not mention which dataset are used.

-- In the setup of the experiments, the In-Distribution datasets only contain CIFAR-10 and CIFAR-100 but do not set any combinations with the other data sets like these shown in Out-of-Distribution datasets. Experiments with the different combinations of the ID / OOD dataset could be more convincing.

questions: > Since It’s a key point for the performance improvement, there should be more detailed description about how to use HAT to extract features.

> I don't see any analysis for the different OOD types. Ad the authors stated in Sec. 5, “Moreover, ATD utilizes HAT to extract robust features from the real samples, and a generator to craft a broad range of OOD data features”. However, there is no experiment setup relating to different OOD types, which may not prove ATD being robust enough.
limitations: As the authors have mentioned, this method significantly outperforms other SOTA OOD detection methods when end-to-end PGD attack is applied. However, further justification should be applied to evaluate the effectiveness/contribution of each detector-extractor pair on different OoD types.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper uncovers the vulnerability of the existing OOD detection methods to adversarial inputs. It then goes on to introduce a robust OOD detection method called Adversarially Trained Discriminator (ADT). ADT combines adversarial training with generative models to improve the detection robustness. In particular, the GAN model in this approach allows generating OOD samples beyond the available open sets during training and therefore boosts the performance of the detection method as a whole. The idea is simple and makes use of recent related work including HAT. While none of the components of the proposed model are novel, the particular system is a novel combination of these components. Results show substantial improvement in robustness beyond other baselines considered. 


POST-REBUTTAL UPDATE: 
I thank the authors for responding to my comments and appreciate the new experiments in support of their arguments. I am increasing my score by 1 to recommend acceptance. 
strengths_and_weaknesses: **strengths** 
- The method is simple and intuitively makes sense. 
- The text is well written and a helpful toy example is included to help the reader better understand the effect of the different components of the model. 
- The results section involves comparison across many different baselines. 

**weaknesses** 
Adversarial robustness has been a constantly evolving field in the past years and the frequent breakage of SOTA defenses by new adversarial attacks has made everyone working in this field very cautious about new proposed defenses. Much of recent work in the AT field points to the insufficiency of PGD as a reliable attack method and many defense methods in recent years have successfully "fooled" this attack without providing a "robust" form of robustness. AutoAttack has become a more reliable substitute to PGD and have proven to be much more difficult to be fooled. In the light of this, I would like to see how the proposed method stands against this stronger attack to be more sure about the reliability of the present results. 

typo: line 45 vise-versa 

Fig2. OpenMax score is not defined which makes it hard to judge what this figure is telling. 
questions: The text was clear and I do not have any additional questions. 
limitations: While CIFAR10 and CIFAR100 are good starting points, many of the past methods have been shown to not scale well to larger, more sophisticated datasets. It would great to see some experiments on how this model would scale to other datasets like tinyimagenet.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: he authors of this paper claim to be showing that the current state-of-the art models for the adversarially robust detection of out-of-distribution samples are not effective when evaluated on sufficiently strong threat models. They then go on to propose a method that they claim remedies this issue by incorporating GAN training and manages to simultaneously achieve high clean OOD detection performance, adversarially robust OOD detection and high accuracy. 
strengths_and_weaknesses: The paper generally tackles an interesting problem of adversarially attacking out-of-distribution samples and they choose a fairly standard threat model. Unfortunately, the novelty of their approach is quite limited because they omit a lot of prior benchmarks. For example, they write "The first [approach] uses some OOD data, uniformly sampled from the open set classes and disjoint from the closed set, and employs adversarial training to encourage the model to assign a uniform label, as opposed to one-hot labels, to anomalous samples even under adversarial perturbations [19]." This describes the method ACET [Hein19], but instead they cite ATOM [19] (which uses an out-class) and which they nonetheless don't compare to for some reason. Both methods should be used as baselines. They furthermore use ALOE as a baseline which according to Eq. 7 is identical to the work RATIO [Augustin20] which should at the very least be cited in this context. On top of this, there even exist several works that tackle the issue of unreliable evaluations of adversarial OOD robustness by providing provable guarantees [Bitterwolf20, Meinke21]. Clearly, these are relevant baselines as well and should at the very least be cited. Additionally, even the idea of using GANs in order to produce adversarial OOD samples has previously been proposed in the well-known paper by [Lee18].

The authors point out that prior works supposedly only studied "small perturbation sizes (e.g. $\epsilon=\frac{1}{255}$ in the CIFAR-10 dataset)". It probably makes sense to explicitly cite exactly which papers they are referring to here. A similar comment applies to a statement such as "The existing OOD detection methods are far from being robust against strong attacks, contrary to what has been claimed previously."

The paper uses 8 out-distributions for each in-distribution which is good. Unfortunately, they do not list the individual results for each out-distribution which makes it very hard to check or compare their results. These need to be listed individually in the appendix. Also, some of the out-distributions clearly overlap with the classes of the in-distributions, for example birds and the bird class from CIFAR10. This needs to be discussed somewhere in the paper, or alternatively different out-distributions should be used.

The authors' method seems to work without relying on a difficult training out-distribution. While they do use SVHN for this, it is quite impressive that their training apparently generalizes to out-distributions that are very different from this. 

The authors should provide slightly more details on their evaluation attack, i.e. initialization, restarts and step size, potentially in the appendix. 

Overall, however, the paper's results seem encouraging. If their model does not get broken by a stronger attack, the fact that they attain high adversarially robust OOD detection performance and clean OOD detection performance at the same time is significant.
They attempt to account for potential gradient obfuscation in the paper via transfer attacks which is good.
They also provide both their code as well as pre-trained models which is also good.

The authors' claim that their method achieves "high accuracy" seems misleading. The backbone feature extractor that they use has a clean accuracy of 91.47% on CIFAR10, which of course is far from "high" compared to SOTA CIFAR10 models. As far as I can tell, this is not pointed out anywhere in the paper. 

I believe the paper cannot be accepted in its current form, because it a) does not individually list the performance with respect to different out-distributions and b) does not adequately discuss how its method relates to and compares to prior work that is substantially similar. If these points are adequately addressed I would be willing to raise my score, because the results are indeed promising.

[Hein19] Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem, Hein, Andriushchenko, Bitterwolf, @CVPR'19

[Augustin20] Adversarial robustness on in-and out-distribution improves explainability, Augustin, Meinke, Hein @ECCV'20

[Bitterwolf20] Certifiably adversarially robust detection of out-of-distribution data, Bitterwolf, Meinke, Hein @NeurIPS'20

[Lee18] Training confidence-calibrated classifiers for detecting out-of-distribution samples, Lee, Lee, Lee, Shin, @ICLR'18

[Meinke21] Provably Robust Detection of Out-of-distribution Data (almost) for free, Meinke, Bitterwolf, Hein @UDL workshop, ICML'21
questions: Minor comments:
- They refer to OOD detectors that use the softmax score as "likelihood"-based. Likelihood-based methods generally refer to models that fit the density of the data distribution $p(x)$.
- The authors forgot to delete the instructions part of the checklist.
limitations: I don't think the authors adequately discuss limitations of their approach. For example, they do not acknowledge that similar to AT, their model's accuracy is significantly reduced. They should also openly discuss how computationally expensive their method is.
I agree with the authors' self-assessment in the checklist that their work is unlikely to have negative ethical implications.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper analyzes the robustness of outlier detection, and proposes to adversarially train the discriminator in order to get better robustness. Experiments demonstrate the effectiveness of the method
strengths_and_weaknesses: I will start with the most important weakness I saw: The idea of adversarially training the discriminator has already been explored in:

Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Atom: Robustifying out-of-distribution
337 detection using outlier mining. In Joint European Conference on Machine Learning and Knowledge
338 Discovery in Databases, pages 430–445. Springer, 2021.

And also the recent ICML paper (pushing this even further):

https://arxiv.org/abs/2206.13687 (ICML 2022 (Long Talk))

This paper cited the first paper, but never did comparison in experiments (not to mention the second paper), which I find to be a main weakness.

On the strengths side, this paper does a good job in explaining the ideas and the results are good

questions: The Atom paper is cited as "Similar to other learning domains, OOD detection methods also suffer from the
44 existence of adversarial examples. A small perturbation can cause a sample from the closed set to be
45 classified as anomaly and vise-versa [17, 18]."; so it seems that we only talk about the attack part of that paper, is there a particular reason why we don't consider the defense there? (in particular, that paper consider adversarial training of the discriminator)
limitations: As I mentioned above, as far as I understand, the idea in this paper is not new and has already been explored in previous work (not one work)
review_title: null

Meta-review: metareview: The paper considers OOD detection facing adversarial attacks. It shows existing defenses are ineffective to end-to-end attacks on both in/out distribution data with larger perturbation than previous work. With the intuition of training with adversarial examples on both in/out distribution data, it then proposed a method (ATD) that uses a generator model to create OOD samples. It provided experiments showing the method outperforms previous methods. 

The paper considers an important topic and made significant contributions: new evaluation of existing methods under stronger attacks, a well-designed method, quite thorough experiments and strong performance of the proposed method. 

The reviewers have some concerns but are largely addressed by the authors' repsonses. The major ones are:

1. Missing citation of/comparison to some existing work. Though ATOM only considers out distribution robustness, it is suggested that the authors include the comparison in the revision for completeness. In the response, the authors have provided results for the comparison. There are also related certified defenses [Bitterwolf20, Meinke21]. The response also added discussion about these.

2. Details should be included in the appendix, e.g., detailed experimental results, evaluation details like hyperparameters, etc. The revision has added these details.

3. Evaluation on more combinations of the ID / OOD datasets. The authors have also provided more in the response; it is suggested to also include them in the revision.

Overall, the authors have made significant improvements addressing the reviewers' concerns/suggestions. The revised paper is of good quality for acceptance. 

confidence: Certain
Decision: Accept