Introduction
Advances in the deep neural networks have led to their widespread use in real-world applications, such as object detection and image classification sarker2021machine ; Zhong2019 .
These models have a high degree of generalizability to the extent that they assign an arbitrarily high probability even to the samples not belonging to the training set Xiuming2018 . This phenomenon causes problems in safety-critical applications like medical diagnosis or autonomous driving that should treat anomalous data differently.

The problem of identifying out-of-distribution data has been widely explored in different categories
such as Novelty Detection, Open-Set Recognition and Out-of-Distribution (OOD) detection salehi2021arae ; Weitang2020 ; zhou2021learning . These categories are almost similar since they all consider two disjoint sets called the closed (or normal) and open (or anomalous) sets, and the model should discriminate them salehi2021unified . So far, outstanding models have been proposed in this research field ming2022poem ; zhang2020hybrid ; chen2020learning ; vaze2022openset . Still, these models need to be examined from other aspects, specifically robustness against adversarial perturbations added to the input data.

Deep networks are vulnerable against adversarial examples. This phenomenon was first noticed in the image classification, but it also extends to other domains Szegedy2014 ; Goodfellow2015 ; akhtar2018threat . Adversarial examples are the inputs that are slightly perturbed with a bounded perturbation (e.g. through bounding the \ell_{p} -norm) to cause a high prediction error in the model. Several defenses have been proposed to make the image classification robust, none of which is as effective as adversarial training and its variants madry2018towards ; zhang2019theoretically ; Wong2020Fast ; azizmalayeri2021lagrangian . Similar to other learning domains, OOD detection methods also suffer from the existence of adversarial examples. A small perturbation can cause a sample from the closed set to be classified as anomaly and vice-versa chen2021atom ; fort2022adversarial .

Robust OOD detection is the connection between these two safety-critical issues. Since a sample does not change semantically under appropriately bounded perturbation in the adversarial attacks, the OOD detector model is expected to be able to label them as open/closed correctly. As a first thought, one might envision solving the issue by using the image classification defenses. Despite advances in these defenses, they have no intuition of the OOD data and not seen them in the training, making them less efficient against end-to-end attacks on the OOD detection method.

Two main approaches are introduced in previous works for the robust OOD detection. The first one uses some OOD data, uniformly sampled from the open set classes and disjoint from the closed set, and employs adversarial training to encourage the model to assign a uniform label, as opposed to one-hot labels, to anomalous samples even under adversarial perturbations augustin2020adversarial ; hein2019relu ; chen2022robust . The second approach conducts adversarial training only on the closed set, but encourages the model to learn more semantic features using representation learning techniques such as auto-encoders, self-supervised learning, and denoising methods shao2020open ; shao2022open . Despite the benefits of these methods, they still have fundamental issues such as:

They did not evaluate their method with an end-to-end attack on the detection method, e.g. they attack the classification model , and not the detector, to evaluate the detection method shao2020open ; shao2022open . We show that this is not an effective and strong attack on the detector.

They consider small perturbation sizes (e.g. \epsilon=\frac{1}{255} in the CIFAR-10 dataset chen2022robust , or not attacking the in-distribution chen2021atom ; hein2019relu ) to protect the standard classification accuracy, while we show that images do not change perceptually with larger perturbation sizes (e.g. \epsilon=\frac{8}{255} in the CIFAR-10 dataset). Thus, these methods should be trained and evaluated against larger perturbations.

The second approach is still unaware of OOD samples during training. Furthermore, the first approach also cannot cover various aspects of OOD data since they can only use a small number of OOD training samples.

In this work, we first evaluate previous defenses as well as standard OOD detection methods against an end-to-end PGD attack madry2018towards . As shown in Fig. 1 , almost all of the earlier methods perform worse than random in the binary OOD detection with \epsilon=\frac{8}{255} in CIFAR-10. As a result, the robust OOD detection method is still far from being solved at present. In spite of this, it is possible to design a robust model by addressing the drawbacks of previous methods. To this end, we propose Adversarialy Trained Discriminator (ATD) that uses a GAN-based method to generate OOD samples and discriminate them from the closed set. An ideal generator naturally deceives the discriminator, and therefore produces adversarial outputs from the open set schlegl2017unsupervised ; liu2019generative ; kong2021opengan . This leads to the implicit robustness of the discriminator against adversarial attacks on the open set. In addition, we also conduct adversarial training on the closed set and the tiny real open set (known as outlier exposure) to achieve robustness on them as well. Our results (e.g., Fig. 1 ) show that ATD outperforms previous methods by a large margin. Therefore, this article has taken a significant step in this area by revealing the vulnerability of all previous methods and providing a solution.