ATD: Adversarially Trained Discriminator
A robust OOD detection method must consider several factors to achieve robustness on both open and closed sets. There has been an extensive study of closed set robustness, which shows that variants of AT are the most effective defenses. Still, pure AT cannot provide robustness in OOD detection for two primary reasons.
First, an AT model achieves a lower accuracy due to the trade-off between accuracy and robustness zhang2019theoretically ; tsipras2018robustness . Note that accuracy plays an important role in the OOD detection vaze2022openset .
Second, the AT does not consider samples from the open set during training. ALOE tried to mitigate this issue using open datasets in training. However, this strategy works only when the open training data is a close proxy for the OOD data. Unfortunately, this is not true in most cases.

To resolve these issues, we consider OpenGAN as our baseline. Here, the discriminator performs the in/out binary classification. In this setting, a generator crafts OOD images to deceive the discriminator by images similar to the closed set. An ideal generator would cover a broader range of OOD distributions, which is a perfect setting for our problem. Furthermore, as the generator is trained to deceive the discriminator, the generated samples are naturally adversarial examples to the discriminator, which meets the needs in the AT. Hence no adversarial attack is need on such samples. Thus, we only need to perturb the closed set to adversarially train the discriminator. We call this method Adverarially Trained Discriminator (ATD), which can be summarized as:

\min_{G}\ \max_{\mathcal{D}}\ \mathbb{E}_{x\in D_{in}}\ \log\min_{\|x-x^{*}\|_{\infty}\leq\epsilon}\mathcal{D}(x^{*})\ +\ \mathbb{E}_{z\in\mathcal{N}}\ \log(1-\mathcal{D}(G(z))), (8)

where the \mathcal{D} and G are the discriminator and generator, respectively. Moreover, the inner minimization is approximated with the PGD attack and the outer minimization and maximization with SGD.
Using adversarial perturbations to train GANs makes them more robust to adversarial examples, just like other architectures liu2019rob ; wang2021improving . On the other hand, adding perturbations to the closed set makes it difficult to optimize the discriminator. As a result, the generator would not be trained to cover a broad range since it could easily fool the discriminator in the min-max game for training GANs.

To get closer to the ideal generator that can cover a broad range of OOD distributions, unlike previously proposed robust GANs liu2019rob , we make the generator craft features instead of images. It is evident that generating and discriminating the low-dimensional features is an easier problem than the high-dimensional images, which brings the generator closer to the ideal case. As a result, we also need a model that can extract robust features from the closed set. For this purpose, we use a pre-trained model with the HAT rade2022reducing method that achieves a satisfactory accuracy along with robustness on the closed set, considering the trade-off between accuracy and robustness tsipras2018robustness . The last layer, which could be thought of as a linear classifier, is excluded to obtain features instead of logits.

Finally, similar to the OpenGAN and ALOE, we utilize an open dataset in the training to stabilize the training. This data is also attacked during training since it is neither originally adversarial, nor is crafted by the generator. With these in mind, the objective function of the ATD can be reformulated as follows:

\begin{split}\min_{G}\ &\max_{\mathcal{D}}\ \mathbb{E}_{x\in D_{in}}\ \log\min_{\|x-x^{*}\|_{\infty}\leq\epsilon}\mathcal{D}(f_{\theta}(x^{*}))\ +\ \ &\alpha\ .\ \mathbb{E}_{x\in D_{out}}\ \log(1-\max_{\|x-x^{*}\|_{\infty}\leq\epsilon}\mathcal{D}(f_{\theta}(x^{*})))+\ (1-\alpha)\ .\ \mathbb{E}_{z\in\mathcal{N}}\ \log(1-\mathcal{D}(G(z))),\end{split} (9)

where f_{\theta} is the pre-trained robust feature extractor and \alpha controls the weight of real open dataset in the optimization. Fig. 3 shows a schematic representation of ATD.