Paper: RODEO: Robust Out-of-Distribution Detection Via Exposing Adaptive Outliers (ICLR-2024)

Reviewer comment (Experiments):
The comparisons with the baselines may not be fair. The proposed method uses a generative diffusion model and a CLIP model, which are pre-trained on millions of data. However, some of the baselines (e.g., ATOM and ALOE) considered in the experiments don't use such extra data for training. Thus, the comparisons may not be fair. It is known that using extra data can significantly improve the model's performance.

Author response:
We understand  the reviewer's concerns regarding the fairness of comparisons due to the use of large pre-trained model. To address this, we would like to provide some key points that highlight our approach's novelty and justify the fairness of our comparisons:

* **Motivation:** Developing robust OOD detection methods is a crucial task due to their application in safety-critical real-world problems such as autonomous driving. While the clean results on challenging benchmarks are near 100% AUROC, the best performance of previous works under attacks is less than random detection even on tiny datasets.

* **Robust Methods:**  Previous robust OOD detection methods mostly focused on a specific setup of outlier detection, such as multi-class outlier detection. For instance, ALOE, ATD, and ATOM are only applicable in the multi-class setup and cannot handle other outlier detection setups like one-class outlier detection. In contrast, our method can generalize across different detection setups and datasets, including medical imaging and industrial datasets, despite the limitations of previous works.
 
* **RODEO vs. SOTA:** Several studies have utilized pre-trained models to enhance outlier detection. These studies have employed models such as ViTs (trained on 21M samples), BERT (1B data), and CLIP (400M data)[1,2] in the outlier detection domain. It is worth noting that we utilized a smaller version of CLIP, trained on 65M data, which is less data-intensive than the previously mentioned models. Moreover, while previous works directly leveraged the rich features of CLIP to develop a detector, we have adopted a different approach by implicitly using CLIP's guidance.

Furthermore, we conducted an experiment to compare our method with EXOE[2], a recent  SOTA method based on CLIP-400M.  The results indicate that our method slightly outperforms EXOE in clean performance, albeit with a minor margin. On the other hand, the significant superiority of our robust detection method highlights our contribution.

Reviewer comment (Experiments):
The PGD attack used is weak. The authors should use a PGD attack with multiple random restarts (e.g., 10 restarts) and more attack iterations (e.g., 1000 steps). Although it uses the strong attack AutoAttack, it doesn't explain how it adapts the AutoAttack to its settings. Since AutoAttack is not designed to attack OOD detectors, it cannot be directly used to attack OOD detectors. Besides, in Tables 2 and 3, the results for AutoAttack are missing.

Author response:
OOD detection can be  formulated as:

$$
g_\lambda(\mathrm{x}) = 
\begin{cases} 
\text{ID} & \text{if } O(\mathrm{x}) \leq \lambda \\\\
\text{OOD} & \text{if } O(\mathrm{x}) > \lambda
\end{cases}
$$

where $O(x)$ is the detection score, and $\lambda$ is the threshold. In PGD attacks, as stated in the Appendix, instead of maximizing the loss value, we try to increase $O(x)$ if $x$ belongs to in-distribution samples and decrease it otherwise. The formulation of the attack would be:

$${x_{0}^*}=x,
x^{t+1} = x^t + \alpha\cdot \text{sgn}( \nabla_{ x}( y.O_{\boldsymbol{\theta}}(x^t)) )$$

Where $y=1$ for in-distribution samples and $y=-1$ for OOD samples. The same setting holds for AutoAttack. AutoAttack is an ensemble of six attack methods: APGD with CE loss, APGD with DLR loss, APGDT , FAB, multi-targeted FAB, and Square Attack. Due to the presumption in DLR loss-based attacks that the model's output size must be three or greater, we had to exclude these specific attacks in our adaptation of AutoAttack, resulting in a total of four attacks. Addressing your concerns regarding the strength of the PGD attack, we conducted additional experiments with a PGD attack with 1000 steps and 10 random restarts, with results indicating negligible performance reduction. To further validate our method's adversarial robustness, we employed the Adaptive Auto Attack(A3)[3], which has been shown to be a stronger attack than AutoAttack. These enhancements substantially strengthen our novelty detection method's robustness evaluation.


ND:
|Attack\Dataset|CIFAR10|CIFAR100|MNIST|FMNIST|SVHN|MVTecAD|Head-CT|BrainMRI|Tumor Detection|Covid19|
|-|-|-|-|-|-|-|-|-|-|-|
|Clean|87.4|79.6|99.4|95.6|75.2|61.5|87.3|76.3|89.0|79.6|
|PGD-100|71.1|62.8|95.7|88.1|35.4|15.9|70.0|71.1|67.5|59.4|
|PGD-1000|70.2|62.1|94.6|87.2|33.8|14.9|68.6|68.4|67.0|58.3|
|AutoAttack|69.3|61.0|95.2|87.6|33.2|14.2|68.4|70.5|66.9|58.8|
|A3|70.5|61.3|94.0|87.0|31.8|13.4|68.1|67.7|65.6|57.6|

OOD Detection:
|Dataset\Dataset|Clean|PGD-100|PGD-1000|AutoAttack|A3|
|-|-|-|-|-|-|
|CIFAR10|93.2|70.4|69.5|69.0|68.8|
|CIFAR100|88.1|66.4|64.7|65.3|63.2|

OSR:
|Attack\Dataset|MNIST|FMNIST|CIFAR10|CIFAR100|
|-|-|-|-|-|
|Clean|97.2|88.8|79.6|48.5|
|PGD-100|87.7|67.1|64.1|37.7|
|PGD-1000|85.0|65.3|62.7|35.3|
|AutoAttack|86.4|66.8|63.5|36.9|
|A3|84.1|62.9|63.0|35.4|

[1] S Esmaeilpour  Zero AAAI 2023

[2] P  Liznerski  Exposing  TMLR 2023

[3] Y Liu Practical CoRR  2022

Reviewer comment (Experiments):
Could the authors explain whether the comparisons with the baselines are fair or not?

Author response:
We understand  the reviewer's concerns regarding the fairness of comparisons due to the use of large pre-trained model. To address this, we would like to provide some key points that highlight our approach's novelty and justify the fairness of our comparisons:

* **Motivation:** Developing robust OOD detection methods is a crucial task due to their application in safety-critical real-world problems such as autonomous driving. While the clean results on challenging benchmarks are near 100% AUROC, the best performance of previous works under attacks is less than random detection even on tiny datasets.

* **Robust Methods:**  Previous robust OOD detection methods mostly focused on a specific setup of outlier detection, such as multi-class outlier detection. For instance, ALOE, ATD, and ATOM are only applicable in the multi-class setup and cannot handle other outlier detection setups like one-class outlier detection. In contrast, our method can generalize across different detection setups and datasets, including medical imaging and industrial datasets, despite the limitations of previous works.
 
* **RODEO vs. SOTA:** Several studies have utilized pre-trained models to enhance outlier detection. These studies have employed models such as ViTs (trained on 21M samples), BERT (1B data), and CLIP (400M data)[1,2] in the outlier detection domain. It is worth noting that we utilized a smaller version of CLIP, trained on 65M data, which is less data-intensive than the previously mentioned models. Moreover, while previous works directly leveraged the rich features of CLIP to develop a detector, we have adopted a different approach by implicitly using CLIP's guidance.

Furthermore, we conducted an experiment to compare our method with EXOE[2], a recent  SOTA method based on CLIP-400M.  The results indicate that our method slightly outperforms EXOE in clean performance, albeit with a minor margin. On the other hand, the significant superiority of our robust detection method highlights our contribution.

Reviewer comment (Evaluation):
Could the authors explain how they use AutoAttack for evaluation?

Author response:
OOD detection can be  formulated as:

$$
g_\lambda(\mathrm{x}) = 
\begin{cases} 
\text{ID} & \text{if } O(\mathrm{x}) \leq \lambda \\\\
\text{OOD} & \text{if } O(\mathrm{x}) > \lambda
\end{cases}
$$

where $O(x)$ is the detection score, and $\lambda$ is the threshold. In PGD attacks, as stated in the Appendix, instead of maximizing the loss value, we try to increase $O(x)$ if $x$ belongs to in-distribution samples and decrease it otherwise. The formulation of the attack would be:

$${x_{0}^*}=x,
x^{t+1} = x^t + \alpha\cdot \text{sgn}( \nabla_{ x}( y.O_{\boldsymbol{\theta}}(x^t)) )$$

Where $y=1$ for in-distribution samples and $y=-1$ for OOD samples. The same setting holds for AutoAttack. AutoAttack is an ensemble of six attack methods: APGD with CE loss, APGD with DLR loss, APGDT , FAB, multi-targeted FAB, and Square Attack. Due to the presumption in DLR loss-based attacks that the model's output size must be three or greater, we had to exclude these specific attacks in our adaptation of AutoAttack, resulting in a total of four attacks. Addressing your concerns regarding the strength of the PGD attack, we conducted additional experiments with a PGD attack with 1000 steps and 10 random restarts, with results indicating negligible performance reduction. To further validate our method's adversarial robustness, we employed the Adaptive Auto Attack(A3)[3], which has been shown to be a stronger attack than AutoAttack. These enhancements substantially strengthen our novelty detection method's robustness evaluation.


ND:
|Attack\Dataset|CIFAR10|CIFAR100|MNIST|FMNIST|SVHN|MVTecAD|Head-CT|BrainMRI|Tumor Detection|Covid19|
|-|-|-|-|-|-|-|-|-|-|-|
|Clean|87.4|79.6|99.4|95.6|75.2|61.5|87.3|76.3|89.0|79.6|
|PGD-100|71.1|62.8|95.7|88.1|35.4|15.9|70.0|71.1|67.5|59.4|
|PGD-1000|70.2|62.1|94.6|87.2|33.8|14.9|68.6|68.4|67.0|58.3|
|AutoAttack|69.3|61.0|95.2|87.6|33.2|14.2|68.4|70.5|66.9|58.8|
|A3|70.5|61.3|94.0|87.0|31.8|13.4|68.1|67.7|65.6|57.6|

OOD Detection:
|Dataset\Dataset|Clean|PGD-100|PGD-1000|AutoAttack|A3|
|-|-|-|-|-|-|
|CIFAR10|93.2|70.4|69.5|69.0|68.8|
|CIFAR100|88.1|66.4|64.7|65.3|63.2|

OSR:
|Attack\Dataset|MNIST|FMNIST|CIFAR10|CIFAR100|
|-|-|-|-|-|
|Clean|97.2|88.8|79.6|48.5|
|PGD-100|87.7|67.1|64.1|37.7|
|PGD-1000|85.0|65.3|62.7|35.3|
|AutoAttack|86.4|66.8|63.5|36.9|
|A3|84.1|62.9|63.0|35.4|

[1] S Esmaeilpour  Zero AAAI 2023

[2] P  Liznerski  Exposing  TMLR 2023

[3] Y Liu Practical CoRR  2022

Reviewer comment (Experiments):
The paper could benefit from a more thorough comparison with other methods of OOD detection, especially those that also use a data-centric approach or those that have been designed specifically for adversarial settings. This could help to place the proposed method in a broader context and demonstrate its advantages and disadvantages more clearly.

Author response:
It should be noted that many of the advanced methods developed for robust detection tasks, including ATD, ALOE, and ATOM, are founded on a data-centric approach. These methods typically employ an outlier exposure (OE) technique, which incorporates an auxiliary random outlier set, like ImageNet, during the process of adversarial training. Our method has been benchmarked against these in Tables 2 and 3 of the paper, and for your convenience, which are provided here as well for your review:

Results on OSR setup:

|Dataset|Mode|ATOM|ALOE|ATD|OURS|
|-|-|-|-|-|-|
|MNIST|Clean/PGD|74.8/6.3|79.5/38.2|68.7/56.7|**97.2**/**88.8**|
|FMNIST|Clean/PGD|64.3/4.7|72.6/29.0|59.6/43.0|**87.7**/**67.1**|
|CIFAR10|Clean/PGD|68.3/5.2|52.4/25.7|49.0/33.6|**79.6**/**48.5**|
|CIFAR100|Clean/PGD|51.4/3.2|49.8/18.6|50.5/36.6|**64.1**/**37.7**|

Results on OOD detection setup:

|In-Dataset|Attack|ATOM|ALOE|ATD|OURS|
|-|-|-|-|-|-|
|CIFAR10|Clean/PGD|82.7/25.1|**97.8**/6.0|94.3/69.3|93.2/ **70.4**|
|CIFAR100|Clean/PGD|**91.6**/5.4|79.3/26.4|87.7/55.3|88.1/**66.4**|

Reviewer comment (Evaluation):
The method involves several complex steps, including the generation of outliers and their incorporation into the training process, which might be computationally expensive. The paper has not yet provided a discussion on the trade-off between the performance gain and the added computational cost.

Author response:
We have provided details on computational cost in Appendix 10.3. To address your concern, we have added further details about the trade-off between performance gain and the additional computational cost here and have updated the appendix accordingly. 

Computational cost:

|Phase|ND|OSR|OOD|
|-|-|-|-|
|Adaptive Generation| 30 min|3h| 5h|
|Adversarial Training|100 min|9h|16h|

To mitigate the computational costs, one alternative is to employ the ImageNet dataset for OE in adversarial training. We have examined this approach in our ablation study. Below, we present the average performance of our detector across two scenarios: (1) using ImageNet as OE for adversarial training, and (2) generating OE with our proposed pipeline and then leveraging it for adversarial training:

|Technique|Clean|Adv|
|-|-|-|
|Vanilla OE|75.6| 34.6|
|Adaptive OE|84.0|66.8|

Reviewer comment (Experiments):
The paper could provide more details on the performance of the method in non-adversarial settings, and it could compare the method with a wider range of existing methods.

Author response:
In response to your comment, we have conducted further experiments and included the results here and in the appendix. This includes scenarios where our detector was trained in a non-adversarial setup, with evaluation under both clean and adversarial settings

ND-setup:

|Train Mode|Test Mode|CIFAR10|CIFAR100|MNIST|FMNIST|Head-CT|Covid19|
|-|-|-|-|-|-|-|-|
|Clean|Clean|93.1|86.6|98.4|94.8|96.1|89.2|
||Adv|0.0|0.0|0.0|0.0|0.0|0.0|
|||||||||
|Adv|Clean|87.4|79.6|99.4|95.6|87.3|79.6|
||Adv|71.1|62.8|95.7|88.1|70.0|59.4|

OSR-setup:

|Train Mode|Test Mode|CIFAR10|CIFAR100|MNIST|FMNIST|
|-|-|-|-|-|-|
|Clean|Clean|84.3|69.0|99.1|91.9|
||Adv|0.0|0.0|0.0|0.0|
|||||||
|Adv|Clean|79.6|64.1|97.2|87.7|
||Adv|48.5|37.7|88.8|67.1|

OOD-setup:

|Train Mode|Test Mode|CIFAR10 vs CIFAR100|CIFAR100 vs CIFAR10|
|-|-|-|-|
|Clean|Clean|83.0|71.2|
||Adv|0.0|0.0|
|||||
|Adv|Clean|75.6|61.5|
||Adv|38.7|30.7|

Reviewer comment (Theory):
The paper assumes that effective outliers should be 'near-distribution', 'diverse', and 'conceptually differentiable' from the inliers. While this makes sense in theory, it may not hold in all practical scenarios. There could be situations where effective outliers do not meet all these criteria, and it would be interesting to see how the proposed method would perform in such cases.

Author response:
We address these considerations in our ablation study, which examines scenarios where auxiliary outlier datasets are either distant from the inlier distribution or lack diversity. The results, extracted from Table 3, help clarify these situations.

|OE\Target|CIFAR10|MNIST|
|-|-|-|
 Gaussian Noise|54.4 / 11.3|56.1 / 12.4|
|Vanilla OE|87.3 / 70.0|90.0 / 43.0|
|Adaptive OE|**87.4** / **71.1**|**99.4** / **95.7**|

Regarding Near-Distribution, the results show that both Adaptive OE and Vanilla OE (ImageNet) enhance clean and robust detection for the CIFAR10 dataset. However, for MNIST, Vanilla OE (ImageNet) is much less effective compared to our Adaptive method. This difference is likely due to the closer distribution between ImageNet and CIFAR10, in contrast to the larger gap between ImageNet and MNIST distributions.
Regarding diversity, our results suggest that using Gaussian noise as OE is less effective due to its lower diversity compared to other techniques we evaluated.
With respect to FDC metric, which serves to assess the diversity and how close is the OE dataset to in-distribution, it is evident that Adaptive OE outperforms the other two methods in this respect. The Ablation Studies section contains tables that compare the AUROC and FDC metrics of our approach with all the OE methods evaluated in our research, and the respective values have been presented here as well:

|OE\Target|CIFAR10|MNIST|
|-|-|-|
 Gaussian Noise|0.822|0.704|
|Vanilla OE|7.647|0.795|
|Adaptive OE|**8.504**|**11.395**|

Reviewer comment (Experiments):
The approach is only evaluated on computer vision tasks and datasets. It would be interesting to see applicability to other modalities.

Author response:
Your suggestion indeed underscores an important avenue for future research in applying our approach to diverse modalities.

We deeply appreciate your insights, which have been instrumental in elevating our work; we humbly believe these enhancements might resonate in an uplifted assessment.

Reviewer comment (Experiments):
In fact, CLIP alone has been shown to be good at OOD detection [A] (not discussed in the submission).

Author response:
We understand concerns about using large pre-trained models akin to extra datasets. Yet, notable methods like ATD and ALOE also use external datasets like TinyImageNet as OE dataset. To highlight our work's novelty, here are some key points:

* **Motivation :** Developing robust outlier detection methods is essential for safety-sensitive real-world applications, like autonomous driving. Although clean results on challenging benchmarks are near 100% AUROC, the top results from previous works drop to below random levels of detection under attack scenarios, even with tiny datasets.

* **Data Complexity :** We agree with the reviewer that utilizing CLIP may provide supervision for our pipeline, however, the real-world application's demand for reliability necessitates robust methods, even at the expense of data complexity. As the reviewer noted, several studies have utilized pre-trained models to enhance outlier detection. These studies have employed models such as ViTs (trained on 21M samples)[1], BERT (trained on 1B data)[2], and CLIP (trained on more than 400M data)[3,4,5] in the outlier detection domain. Despite their notable performance in clean settings, they are vulnerable to adversarial. In contrast, our approach employs a smaller CLIP variant, trained on 65M samples, which not only withstands strong attacks more effectively but also delivers competitive performance in standard settings.

* **Robust Methods :** Prior robust OOD detection techniques were largely tailored to certain scenarios like multi-class outlier detection, with examples like ALOE, ATD, and ATOM being restricted to this setup. Our method, however, is versatile, effectively working across various detection scenarios and datasets, including those in medical imaging and industry, overcoming the constraints of earlier approaches.

* **Different usage of CLIP :** The utilization of CLIP in previous OOD detection methods differs from our approach. Previous works directly leveraged the rich features of CLIP to develop a detector. However, we implicitly utilize CLIP's guidance to generate near and diverse auxiliary outliers for utilizing adversarial training in subsequent steps.

In order to better address the reviewer's concern, we also compare our method with EXOE[5], a recent state-of-the-art novelty detection method based on a CLIP model trained on 400M data, and provide the results here for review:

|||CIFAR10|CIFAR100|MNIST|FMNIST|SVHN|MVTecAD|Mean|  
|-|-|-|-|-|-|-|-|-|
|RODEO|CLEAN/PGD-100|87.4/71.1|79.6/ 62.8|99.4/ 95.7|95.6/88.1|78.6/45.4|61.5/ 15.9|83.6/**64.1**|  
|EXOE|CLEAN/PGD-100|99.6/0.6|97.8/0.0|96.0/0.1|94.7/2.4|68.2/0.0|76.2/0.4|**88.7** /0.6|

Our method demonstrates superior effectiveness in outlier detection, outperforming other SOTA techniques in both clean and adversarial environments, with the latter being equally crucial.

Reviewer comment (Evaluation):
The authors do not describe the limitations of their work, e.g. what happens if the OOD class is also OOD for clip and the diffusion model?

Author response:
One limitation of our method is its adversarial performance in fine-grained, pixel-level outlier detection, specifically in datasets like MVTecAD. Although our robust detection outperforms other methods, it still requires further development to achieve promising results. A potential reason for our method's limitation with this dataset is that inliers and outliers have very subtle differences at the texture level, making them susceptible to conversion between inliers and outliers by perturbations from strong attacks like PGD.

Reviewer comment (Writing):
What is $c$ in e.g., $\mu(x_t|c)$?

Author response:
This was a typo; 'C' must be replaced with $y_{\text{NOOD}}$.

Reviewer comment (Writing):
Page 6: Adverserial

Author response:
We appreciate your insightful review and sincerely apologize for the typos. These issues have been addressed in the revised version, and we will enhance the clarity of our paper.

Reviewer comment (Presentation):
Page 15: an extra labels.

Author response:
We appreciate your insightful review and sincerely apologize for the typos. These issues have been addressed in the revised version, and we will enhance the clarity of our paper.

Reviewer comment (Writing):
Figure 5: purturb

Author response:
We appreciate your insightful review and sincerely apologize for the typos. These issues have been addressed in the revised version, and we will enhance the clarity of our paper.

Reviewer comment (Presentation):
The explanation of Figure 1 is confusing. I am confusing about the explanation about “This suggests that an OE dataset closer to the normal dataset distribution is significantly more beneficial than a distant one.” Does this mean that the samples obtained from previous adversarial training are not effective? The author needs a more detailed explanation here.

Author response:
We apologize for any inconvenience caused by the lack of clarity in the caption of Figure 1. We have since revised the caption to provide a clearer understanding and would like to offer additional insight into Figure 1 here.

Figure 1 represents the motivation for pursuing adaptive outlier exposure (OE) in our paper. OE refers to a technique used in outlier detection literature, which leverages a fixed auxiliary outlier dataset (e.g., ImageNet) during training to improve detection methods by providing more information about outlier data.

In the cases shown in Figure 1, CIFAR10 is treated as the inlier training set, while CIFAR100 serves as a source of Out-of-Distribution (OOD) data unavailable during training time. The ALOE and ATD aim to robustly discriminate them during test time. Both methods employ Tiny ImageNet as an auxiliary outlier set (i.e., OE) during adversarial training, as integral components of their proposed frameworks. In our experiment, while keeping all other aspects of the original methods constant, we replaced Tiny ImageNet with SVHN, MNIST, and Gaussian noise and repeated the experiments for both ALOE and ATD. This replacement led to a notable decline in OOD detection performance for ALOE and ATD on the CIFAR10 vs. CIFAR100 task, particularly under adversarial attack conditions. We attribute this performance drop to the fact that the SVHN, MNIST, and Gaussian Noise distributions are more distant from CIFAR10 (the inlier distribution in this task) compared to Tiny ImageNet. The results are also presented in detail here:

|Methods  \ Auxilary Outlier Dataset (OE) |Tiny Imagenet | SVHN |MNIST |Gaussian Noise |
|-|-|-|-|-|
| ALOE(Clean/Adv.) |  78.8/17.0 | 61.2/5.3 |58.7/2.3 |48.2/1.1 |
| ATD (Clean/Adv.) | 83.1/37.4 | 78.5/ 33.8 |47.2/5.0 |38.1/3.0 |
 
> **W2-2:**

Previous methods utilized random, fixed auxiliary outlier datasets as OE, which we believe make their methods ineffective when the inlier distribution and the auxiliary outlier set are distant. We support this with our Figure 1, theoretical insight, and extensive ablation studies. For instance, as our Table 4 results indicate, using ImageNet as OE for MNIST as the inlier set is less effective compared to using ImageNet as OE for CIFAR10 as the inlier set. This is because both ImageNet and CIFAR10 include natural images and have less distribution distance compared to ImageNet and MNIST.