Informative Outlier Matters: Robustifying Out-of-distribution Detection Using Outlier Mining
Authors: Jiefeng Chen
Yixuan Li
Xi Wu
Yingyu Liang
Somesh Jha
Venue:  (2021)
Keywords: OOD detection
informative outlier mining
robustness

Abstract: Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in an open-world setting. However, existing OOD detection solutions can be brittle in the open world, facing various types of adversarial OOD inputs. While methods leveraging auxiliary OOD data have emerged, our analysis reveals a key insight that the majority of auxiliary OOD examples may not meaningfully improve the decision boundary of the OOD detector. In this paper, we provide a theoretically motivated method, Adversarial Training with informative Outlier Mining (ATOM), which improves the robustness of OOD detection. We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks. ATOM achieves state-of-the-art performance under a broad family of classic and adversarial OOD evaluation tasks. For example, on the CIFAR-10 in-distribution dataset, ATOM reduces the FPR95 by up to 57.99\% under adversarial OOD inputs, surpassing the previous best baseline by a large margin. 

Under review as a conference paper at ICLR 2021
INFORMATIVE OUTLIER MATTERS: ROBUSTIFYING OUT-OF- DETECTION USING OUTLIER MINING
Anonymous authors Paper under double-bling review
ABSTRACT
Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep learning models in an open-world setting. However, existing OOD detection solutions be brittle in the open world, facing various types of adversarial OOD inputs. While methods leveraging auxiliary OOD data have emerged, our analysis reveals key insight that the majority of auxiliary OOD examples may not meaningfully improve the decision boundary of the OOD detector. In this paper, we provide theoretically motivated method, Adversarial Training with informative Outlier Mining (ATOM), which improves the robustness of OOD detection. We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks. ATOM achieves state-of- of-the-art performance under broad family of classic and adversarial OOD evaluation tasks. For example, on the CIFAR- 10 in-distribution dataset, ATOM reduces the FPR95 by up to 57.99% under adversarial OOD inputs, surpassing the previous best baseline by large margin.
INTRODUCTION
(OOD) detection has become an indispensable part of building reliable open- world machine learning models (Amodei et al., 2016). An OOD detector determines whether an input is from the same distribution as the training data, or a different distribution (i.e., out-of-distribution The performance of the OOD detector central for safety-critical applications such autonomous driving (Eykholt et al., 2018) or rare disease identification (Blauwkamp et al., 2019).
Despite exciting progress made in OOD detection, previous methods mostly focused clean OOD data (Hendrycks Gimpel, 2016; Liang et al., 2018; Lee et al., 2018; Lakshminarayanan et al., 2017: Hendrycks al., 2018; Mohseni al., 2020). Scant attention has been paid to the robustness aspect of OOD detection. Recent works (Hein et al., 2019; Sehwag et al., 2019; Bitterwolf et al., 2020) considered worst-case OOD detection under adversarial perturbations (Papernot al., 2016; Goodfellow et al., 2014; Biggio et al., 2013; Szegedy et al., 2013). For example, an OOD image (e.g.. mailbox) can be perturbed be misclassified by the OOD detector as in-distribution (traffic sign data). Such an adversarial OOD example is then passed to the image classifier and trigger undesirable prediction and action (e.g., speed limit 70). Therefore, it remains an important question to make out-of-distributio detection algorithms robust in the presence of small perturbations to OOD inputs.
In this paper, we begin with formally formulating the task of robust OOD detection and providing the- oretical analysis in a simple Gaussian data model. While recent OOD detection methods (Hendrycks et al., 2018; Hein et al., 2019; Meinke & Hein, 2019; Mohseni et al., 2020) have leveraged auxiliary OOD data, they often sample randomly uniformly from the auxiliary dataset. Contrary to the common practice our analysis reveals a key insight that the majority of auxiliary OOD examples may not provide useful information improve the decision boundary of OOD detector. Under Gaussian model of the data, we theoretically show that using outlier mining significantly improves the error bound of OOD detector in the presence of non-informative auxiliary OOD data.
Motivated by this insight, we propose Adversarial Training with informative Outlier Mining (ATOM), which justifies the theoretical intuitions above and achieves state-of-the-art performance broad

---

Under review as a conference paper at ICLR 2021
x
Traffic
CNN
Akp nan
Dad-ol-iitiiaation didtttution
Without Informative Outleer Mining
With Informative Outtier Mining
Train classifier (K)) build ous-af-distribution detector
Test example
CNN
100
CNN 1b0
"Reael
label
CNN
"Swwd 25"
label
CNN
TDU
"Jap Slye
label
GM
G(A)
G(d)
Reject
Reject
Reject
Reject
(5)
Natural OOD
(b) Lup 0O0 (White-box Attack)
(c) Corruption 00O
(d) Comp. OOD (White-box Attack)
(No Attack)
(Blcck-box Attack))
Figure I: When deploying image classification system (OOD detector G(z) + image classifier fxx) in an open world, there can be multiple types of out-of-listribution examples. We consider broad family of OOD inputs, including (a) Natural OOD, (b) Loc OOD, (c) corruption OOD, and (d) Compositional OOD. detailed description of these OOD inputs can be found in Section 5. In (b-d), a perturbed OOD input (e.g.. a perturbed mailbox image) can mislead the OOD detector to classify it as an in di-ditrribution sample. This can trigger the downstream image classifier f(z) to predict it adversarial training with informative outlier mining (ATOM), our method can robustify the decision boundary of OOD detector G(z), which leads improved performance across all types of OOD inputs. Solid lines are actual computation flow.
of the in in-distribution classes (e.g.. speed limit 70). Through
family of classic and adversarial OOD evaluation tasks for modern neural networks. We show that, by carefully choosing which OOD data to train on, of an OOD detector, and somewhat surprisingly, generalize to unseen adversarial attacks. We note that while hard negative mining has been extensively used in various learning tasks such as object recognition (Felzenszwalb et al., 2009; Gidaris & Komodakis, 2015; Shrivastava et al., 2016). to the best of our knowledge, we are the first to exploit the novel connection between hard example mining and OOD detection. We show both empirically and theoretically that hard example mining significantly improves the generalization and robustness of OOD detection.
significantly improve the robustness
To evaluate our method, we provide unified framework that allows examining the robustness of OOD detection algorithms under a broad family OOD inputs. as illustrated in Figure Our evaluation includes existing classic OOD evaluation task Natural OOD, and adversarial OOD evaluation task T.oo OOD. Besides, we also introduce new adversarial OOD evaluation tasks Corruption OOD and Compositional OOD. Under these evaluation tasks, ATOM achieves state-of- the-art performance compared to eight competitive OOD detection methods (refer to Appendix B.3 for detailed description of these methods). On the Natural OOD evaluation task, ATOM achieves comparable and often better performance than current state-of-the of-the-art methods. On Lx- OOD evaluation task, ATOM outperforms current state-of-the-an method ACET by a large margin (e.g. on CIFAR-10, outperforms it by 53.9%). Under the new Corruption OOD evaluation task, where the attack is unknown during training time, ATOM also achieves much better results than previous methods (c.g. on CIFAR-10, 10, outperform previous best method by 30.99% % ). While almost every method fails under the hardest Compositional OOD evaluation task, ATOM still achieves impressive results (e.g. on CIFAR-10, 10, reduce the FPR by 57.99%) ). The performance is noteworthy since ATOM not trained explicitly on corrupted OOD inputs.
In summary, our contributions are:
Firstly, we contribute theoretical analysis formalizing the intuition of mining hard outliers for improving the robustness of OOD detection.
Secondly, we contribute theoretically motivated method, ATOM, which leads to state-of- the-art performance on both classic and adversarial OOD evaluation tasks. We conduct extensive evaluations and ablation analysis to demonstrate the effectiveness of informative outlier mining-
Lastly, we provide a unified evaluation framework that allows future research examining the robustness of OOD detection algorithms under broad family of OOD inputs.
2

---

Under review as a conference paper at ICLR 2021
2 PRELIMINARIES
In this section, we formulate the problem of robust out-of-distribution detection, and provide back- ground knowledge on the use of auxiliary data for OOD detection.
Problem Statement, We consider training dataset Dirain drawn i..d. from data distribution Px,y y where X is the sample space and Y = 21. ... K} is the set of labels. A classifier f(x) is trained on the in-distribution, Px. the marginal distribution of Px.y. The OOD examples during test time, which are from a different distribution Qx, potentially with perturbations added. The task of robust out-of-distribution detection is learn a detector G x x 11, which outputs 1 for x from Px and output -1 for a clean or perturbed OOD example x from Qx.
=
revealed
Formally, let ((xx) be a set of small perturbations on an OOD example x. The detector is evaluated on x from Px and on the worst-case input inside 22(x) for an OOD example from Qx. The false negative rate (FNR) and false positive rate (FPR) are defined as:
FNR(G) = Ex~Px [x( -1],
FPR(G; Qx,9) = Ex~Qx max [[G(x + 5) 1]. bef(x)
(1)
Note that no data from the test OOD distribution Qx are available for training.
Use of Auxiliary Data for OOD Detection While it is impossible to anticipate the test OOD data distributions Qx for training, recent works (Hendrycks et al., 2018; Hein et al., 2019; Meinke & Hein, 2019; Mohseni et al., 2020; Liu et al., 2020) have shown the promise using auxiliary data as proxy for estimating the decision boundary between in- vs. OOD data. The idea is illustrated in Figure 1, where outlier data is randomly sampled to regularize the model outputs (e.g., low confidence for OOD data and high confidence for in-distribution data). Formally, we assume the auxiliary OOD dataset D is sampled from a different distribution Ux. The difference between the auxiliary data Ux and test OOD data Px raises the fundamental question of how to effectively leverage Domxillary for improving learning the decision boundary between in- vs. OOD data.
3 THEORETICAL ANALYSIS: INFORMATIVE OUTLIERS MATTER
In this section, we present a theoretical analysis' that motivates the use of informative outlier mining for OOD detection. To establish formal guarantees, we use Gaussian data model model data Px, Qx, and Ux. Different from previous work by Schmidt et al. (2018) and Carmon et al. (2019), our analysis gives rise to a separation result with or without informative outlier mining for OOD detection. To this end, we note that while hard negative mining has been explored in different domains of learning (e-g. object detection, deep metric learning, please refer to Section for details), the vast literature of out-of-distribution detection has not explored this idea. Moreover, most uses of hard negative mining are on a heuristic basis, but in our case, the simplicity of the definition of OOD (see Section 2) allows us to derive precise formal guarantees, which further differs us from previous studies of hard negative mining. As a remark, our analysis also establishes formal evidence of the importance of using auxiliary outlier data for OOD detection, which is lacking in the current OOD) detection studies. We refer readers to Section A for these results.
At high level, our analysis provides two important insights: (1) First, show that detection algorithm can work very well if all data is informative; yet it can fail completely in a natural setting where we mix informative auxiliary data with non-informative auxiliary data. (2) Second, we show that, tweaking the algorithm with simple thresholding to choose mildly hard auxiliary data (in our setting they are exactly the informative ones), can lead to good detection performance. Combining both thus provides direct evidence about the importance of hard negative mining for OOD.
Gaussian Data Model. We now describe the Gaussian data model, inspired by the model in (Schmidt et al., 2018; Carmon et al., 2019), but with important adjustment to the OOD detection setting. In particular, our setting has a family of possible test OOD distributions and have only in- distribution data for training, modeling that the test OOD distribution is unknown for training. Given j E Idd, o > 0, v > O, we consider the following model:
Px (in-distribution data): . (n, a² T); The in-distribution data {x,) is drawn from Px. Qx (out-oof-iittributtin data) can be any distribution from the family @ = (N(-u
0,0²1) : v E Rdd ||0||2 < v).
Due lack of space, proofs deferred Appendix A.
3

---

Under review as a conference paper at ICLR 2021
Hypothesis class of OOD detector: g {Go(x) sign(f* x) R.
A concrete instance of the model is defined by a set of parameter values for d. H. o, and v: see Appendix A.2 for the family of instances analyze. While the Gaussian model may be much simpler than the practical data, its simplicity is desirable for our analytical purpose for demonstrating the insights. Furthermore, the analysis in this simple model has implications for more complicated and practical methods, which we present in Section 4. Finally, the analysis can be generalized to mixtures of Gaussians which models practical data much better. Below, consider the FNR and the FPR under fo perturbations of magnitude Since Qx is not accessible at training time, our goal is to bound su (P; Qx, fxx.e
Failing a good detector by mixing non -on-informative auxiliary data. We start by considering the case where all auxiliary data are informative: That is, all auxiliary {x'))= come from uniform mixture of the possible test OOD distributions in Q. In this case, it is straightforward to show that a simple averaging- based detector,
On n'
x
performs very well (See Proposition 4 in the appendix).
Unfortunately, this detector can be easily failed by considering the following simple auxiliary data distribution Umix, which mixes the ideal auxiliary data with non-informative data
mix (Non-ideal mixture): Umix is uniform mixture of N(-H, and N(p,, I21) with 14o = 10p.
Importantly, the distribution Umix models the case where the auxiliary OOD data has some non- informative outliers, and also with a small probability mass of samples (e.g., tail of N(-H,o01) I)) in the support of in-distribution. In this case, the simple average method leads to E[0,,,, -7p/4 with large error, since Or is misled by auxiliary data from M(J,, o² I) or tail (-,,,, of in the support of in-distribution.
Fixing the detector with informative outlier mining. We now show an important modification to the detection algorithm by using informative outlier mining, which leads to good detection performance. Specifically, we first use in di-distribuuion data to get an intermediate solution: 6rm 1 -.= x, Then, we use a simple thresholding mechanism to only pick points with mild confidence scores, which removes non-informative outlies. Specifically, we only select outliers x whose
confidence scores f() = 1/(1 + -=6bna/d) fall in an interval [a, bj. The final solution dom is times the average of the selected outliers. We can prove the following:
Proposition 1. (Error bound with outlier mining.) For any e E (0,1/2) and any integer no > O, there exist a family of instances of the Gaussian data model such that the following is true. n' auxiliary OOD data from Unix specified above. There exist thresholds a and b for 0am and universal constant e > O such that if the number of in-distribution data n. 2 c(no log + vdno) and the number of auxiliary data n' 2 (d+ noy (c2) /d2 /d//n. then dmm has small errors:2
Eg... FN 5 E sup FPR(G Doa (x)) S 10 QXEQ
(3)
Intuitively, the mining method removes the misleading points (most points in ((Ho 0²I) and tail of ((-u,, o² () in the support of in-distribution). Outliers selected in this way are mostly informative and thus give accurate final detector, which justifies outlier mining in the presence of noniinformative data. If we compare this bound with that for the case without auxiliary data (Proposition 3), we can see that for sufficiently high dimension d, with the same amount of in-distribution data, any algorithm without outliers must fail but our outlier mining method can learn good detector. We also note that our analysis and the result also hold for many other auxiliary data distributions Umix and the particular Umix used here is for the simplicity of demonstration; see the appendix for more discussions. In the following section, we design a practical algorithm based on this insight and present empirical evidence of its effectiveness.
?The error bound in the proposition can be made arbitrarily small and with high probability. The current. bound presented for simplicity.
4

---

Under review as a conference paper at ICLR 2021
Hatagram of OOD Score EEpoch 30)
Number of Polnts
09800
40000
20800
60
0.2
0.4
8.6
0.5
1.0
0OD Score
(a) OOD score distribution
(b) highest OOD scores
(c) lowest OOD scores
Figure 2: On CIFAR-10, train DenseNet with objective (4) for 100 epochs without informative outlier mining. At epech 30, we randomly sample 400,000 data points from Damillar and plot the OOD score frequeney distribution (a). We observe that the model quickly converges to solution where OOD score distribution becomes dominated by easy examples with score closer to as shown in (b). Therefore, training on these easy OOD data points can no longer help improve the decision boundary of OOD detector. (c) shows the hardest examples mined from Tinylmages w.r. CIFAR-10.
ATOM: ADVERSARIAL TRAINING WITH INFORMATIVE OUTLIER MINING
In this section, we introduce Adversarial Training with informative Outlier Mining (ATOM), which justifies our theoretical analysis and effectiveness in the context of modern neural networks. We first present the adversarial training objective and then describe how we use informative outlier mining to robustify OOD detection.
Training Objective. We consider a (K + 1)) classifier network f. where the (K 1)-th class label indicates out-of-distribution class. Denote by (x) the softmax output of f on x. The robust training objective is given by
minimize
((..))) ( y: Po)] + A Bx max
[((x', K + 1; Po)).
x (x
where is the cross entropy loss, and out is the OOD training dataset. We use Projected Gradient Descent (PGD) (Madry et al., 2017) to solve the inner max of the objective, and apply it to half of minibatch while keeping the other half clean to ensure performance on both clean and perturbed data. Once trained, the OOD detector (xx) can be constructed by:
G(x) = [-1
if F(x) 2 Y if fxx) <7,
(5)
where y is the threshold, and in practice can be chosen on the in-distribution data so that a high fraction of the test examples are correctly classified by G. We call Fxx) the OOD score of x. For an input that labeled as in-distribution by G. one can obtain its semantic label using F(x):
F(x)
arg max F(x),
(6)
vE(122..... .K
Informative Outlier Mining. Motivated by theoretical analysis in Section 3 and our empirical observation shown in Figure 2, we propose to adaptively choose OOD training examples where the detector is uncertain about. We provide the complete training algorithm using informative outlier mining in Algorithm 1. Our method is different from random sampling as used in previous works (Hendrycks et al., 2018; Hein al., 2019; Meinke & Hein, 2019; Mohseni et al., 2020). Specifically, during each training epoch, we randomly sample N data points from the auxiliary OOD dataset auuiiliaryy and use the current model to infer the OOD scores". Next, we sort the data points according to the OOD scores and select a subset of n < N data points, starting with the / data in the sorted list. We then use the selected samples as OOD training data Dirain for the next epoch of training. Intuitively, g determines the informativeness of the sampled points w.r.t the OOD detector. The larger 9 is, the less informative those sampled examples become. Note that informative outlier
Since the inference stage he fully parallel, outlier mining can he applied with relatively low overhead.
5

---

Under review as a conference paper at ICLR 2021
Algorithm I ATOM: Adversarial Training with informative Outlier Mining
imput Dirain Douxillary, ,Fo. m, N, n. output F. G for t = 1,,,,-..,. do
Randomly sample N data points from Danxiliary to get a candidate set S. Compute OOD scores S using current model Fo get set V (F(x) x E). Sort scores in V from the lowest to the highest.
Ditni + VigN gNV + ni]
{q [0, 1 [0,1-n/M]] n/N]]
Train Fo for one epoch using the training objective of (4). end for
Build G and F using (5) and (6) respectively.
mining is performed on (non-adversarial) auxiliary OOD data. Selected examples are then used in the robust training objective (4).
To see how informative outlier mining alone improves the OOD detection, we also consider the following objective without adversarial training:
minimize ((xx)))==mm (e(x, y; Fo)) A Ex [e(x, K Fo),,
which we name Natural Training with informative Outlier Mining (NTOM).
EXPERIMENTS
In this section, we describe our experimental setup (Section 5.1) and show that ATOM can substan- tially improve OOD detection performance on both clean OOD data and adversarially perturbed OOD inputs. We also conducted extensive ablation analysis to explore different aspects of our algorithm. Our experiments are mainly on image data, which is common in previous work, but we believe that our insights and method can be applied to other types of data, which is left for future work.
5.1 SETUP
In-distribution Datasets. We use SVHN (Netzer et al., 2011), CIFAR-10, and CIFAR- 100 (Krizhevsky al., 2009) datasets as in distriibbution datasets.
Auxiliary OOD Datasets. By default, use 80 Million Tiny Images (Tinylmages) (Torralba et al., 2008) as Diuxiliary which is common setting prior works. We also use ImageNet-RC, variant of ImageNet (Chrabaszez et al., 2017) as an alternative auxiliary OOD dataset.
Out-of-distribution Datasets. For OOD test dataset, we follow the procedure in (Liang et al., 2018; Hendrycks et al., 2018) and use six different natural image datasets. For CIFAR-10 and CIFAR-100 we use SVHN, Text ures (Cimpoi et al., 2014), Places36 65 (Zhou et al., 2017), LSUN (crop), LSUN (resize) (Yu et al., 2015), and 1SUN (Xu al., 2015). For SVHN, Textures, Places365, 365, LSUN (crop), LSUN (resize) and SUN. Besides natural image datasets, we also consider Gaussian Noise and Uniform Noise OOD test data.
CIFAR- O,
The hyperparameter is chosen a separate validation set from Tinylmages, which does not depend on test-time OOD data (see Appendix B.7). Based on the validation results in Table 5, we comparison, in each epoch, ATOM uses the same amount of outlier data as OE, where n is twice larger than the in-distribution data size (i.e., 50,000). For all experiments, we À 1. For CIFAR-11 10 and CIFAR- 100, we set N = 400, 000, and n = 100, 000; For SVHN, we set N = 586, 056, and n 146, 514. More details about experimental set up are in Appendix B.I.
Hyperparameters.
0 for SVHN, q 0.125 for CIFAR- 10 and 0.5 for CIFAR-100 100. To ensure fair
Robust OOD Evaluation Tasks. We consider the following family of OOD inputs, for which provide visualizations in Appendix B.5:
Natural OOD: This is equivalent to the classic OOD evaluation with clean OOD input and 2
Loo attacked OOD (white-box): We consider small L -orm bounded perturbations OOD input x (Madry et al., 2017: Athalye et al., 2018). which induce the model to produce high confidence scores (or low OOD scores) for OOD inputs. We denote the adversarial

---

Under review as a conference paper at ICLR 2021
FPR (5% KNR)
AUROC
FPR
AUROC
FPR
AUROC
FPR
AUROC
(5% KNR)
HNR)
ENR)
Din
Method
Natural OOD
Corruption OOD
Loc OOD
Comp. OOD
MSP
38.84
93.57
99.68
68.48
99.89
100.00
ODIN
31.45
93.52
97.11
63.21
99.86
0.61
100.00
0.05
SVHN
Mahalanobis
22.80
95.57
93.14
60.78
97.33
8.89
99.89
0.23
SOFL
0.06
99.98
99.07
46.78
01
OE
0.60
99.88
23.44
96.23
69.36
52.19
99.65
1.27
ACET
0.49
99.91
17.03
97.23
29.33
86.75
99.85
5.13
CCU
0,50
99.90
24.17
96.11
52.17
62.24
99.42
ROWL
2.04
98.87
55.03
72.37
77.24
61.27
99.79
50,00
NTOM (ours)
0.04
99.98
2.87
99.16
60.28
64.06
99.78
1.50
ATOM (ours)
99.97
98.52
MSP
50.54
91.79
100.00
58.35
100.00
13.82
100.00
13.67
ODIN
21.65
94.66
99.37
$1.44
99. .9
0.18
100.00
0.01
CIFAR-
Mahalanobis
26.95
90.30
91.92
43.94
95.07
12.47
99.88
10
SOFL
2.78
99.04
62.07
88.65
99.98
1.01
100.00
0.76
OE
3.66
98.82
56.25
90.66
99.94
0.34
99.99
0.16
12.28
97.67
CCU
3.39
98.92
56.76
89.38
99.91
0.35
99.97
0.21
ROWL
25.03
86.96
94.34
52.31
99.98
49.49
100.00
49.48
NTOM (ours)
99.28
94.67
90
99
ATOM (ours)
I,69
99.20
25,26
95.29
20.55
88,94
38.89
86.71
MSP
78.05
76.11
100.00
30.04
100.00
2.25
100.00
2.06
CIFAR-
56.77
83,62
100.00
36,95
100.00
100.00
Mahalanohis
42.63
87,86
95.92
42.96
95.44
15.87
99.86
2.08
SOFL.
43.36
91.21
99.93
45.23
100.00
0.35
100.00
0.27
OK
49.21
ACET
50.93
89.29
99.53
54.19
76.27
59.45
99.71
38.63
CCU
43.04
90.95
99.90
48.34
100.00
0.75
100.00
0.48
ROWL
93.35
NTOM (ours)
36.94
92.61
98.17
65.70
99.97
0.76
100.00
0.16
ATOM (ours)
32.30
93.06
93.15
71.96
38.72
88.03
93.44
69.15
Table 1: Comparison with competitive OOD detection methods. We DenseNet as network architecture for all methods. We evaluate on four types OOD inputs: (1) natural OOD, (2) corruption attacked OOD, (3) Los attacked OOD, and (4) compositionally attacked OOD inputs. The description these OOD inputs can be found in Section 5.1. indicates larger value is better, and J indicates lower value is better. All values are percentages and are averaged over six different OOD test datasets described in Section 5.1. Bold numbers superior results. Additional results different architecture, WideResNet, provided in Appendix B.9.
perturbations by (x)),(x), where is the adversarial budget. We provide attack algorithms for all eight OOD detection methods in Appendix B.4.
realistic type of attack based on common corruptions (Hendrycks & Dietterich, 2019), which could appear naturally in the physical world. For each OOD) image, generate 75 corrupted images (15 corruption types x 5 severity levels), and then select the one with the lowest OOD score.
Corruption attacked OOD (black-box): We consider
Compositionally attacked OOD (white-box): Lastly, consider applying Loe -orm bounded attack and corruption attack jointly to an OOD input x, as considered in (Laidlaw & Feizi, 2019).
Evaluation Metrics. We measure the following metrics: the false positive rate (FPR) at 5% false negative rate (FNR) and the area under the receiver operating characteristic curve (AUROC).
5.2 RESULTS
How does ATOM compare to existing solutions? We show in Table that ATOM outperforms competitive OOD detection methods on both classic and adversarial OOD evaluation tasks. First, on classic OOD evaluation task (clean OOD data). ATOM achieves comparable or often even better performance than the current state-of-the-arr methods. Second, on the existing adversarial OOD evaluation task Toxe OOD, ATOM outperforms current state-of-the-an method ACET (Hein et al., 2019) by large margin (e.g. on 10.AR-10, our method outperforms ACET by 53.9% % measured by FPR). Third, while ACET somewhat brittle under the new Corruption OOD evaluation task, method can generalize surprisingly well the unknown corruption attacked OOD inputs, outperforming the best baseline by large margin (e.g. on CIFAR-10, by up to 30.99% measured by FPR). Finally, while almost every method fails under the hardest compositional OOD evaluation task, our method still achieves impressive results (e.g. on CIFAR- 10, reduces the FPR by 57.99 %). The performance is noteworthy since method is not trained explicitly corrupted OOD inputs.

---

Under review as a conference paper at ICLR 2021
FPR (5% FNR)
AUROC
FPR
AUROC
EPR
AUROC
FPR
AUROC
Dis
Model
(5% FNR
(5% FNR)
(5% FNR)
Natural OOD
Corruption COD
OOD
Comp.
OOD
ATOM (rand. sample)
0.35
99.91
13.09
97.30
11.72
96.6.
98, 66
40.48
ATOM (g=0.0)
0.07
99.97
5.47
98.52
7.02
98.00
96.33
49.52
SVHN
ATOM (g=0.125)
1.30
99.63
34,97
94,97
39.61
82.92
99.92
6.30
ATOM (g)).25)
1.36
99.60
41 1988
94.30
52.39
71.34
99.97
1.35
ATOM (g=0.5)
2.11
99.46
44.85
93.84
59.72
65.59
99.97
3.15
ATOM (g=0.75)
2.91
99.26
$1.33
93.07
66.20
57.16
99.96
2.04
ATOM (rand. samplo)
2.65
99.11
42.28
91.94
44.31
68.64
65.17
72.62
ATOM (g=0.0)
CIFAR-
ATOM (q=0.125)
1.69
99.20
25.26
95.29
20.55
88.94
38, 89
86.71
10
AIOM (g=).25)
ATOM (g=0.5)
4.03
98,97
33.93
93.51
22.39
95.16
45.11
90.56
ATOM (g=0.75)
91.98
ATOM (rand. sample)
51.50
89.62
99.70
58.61
70.33
58,84
99.80
34.98
CIFAR- 100
ATOM (q=0.125)
ATOM (q=0.0)
26.91
44.38
94.97
98.35
71.53
34,66
87.54
98.42
68.52
ATOM (g-0,25)
72.61
97.
ATOM (q=0.5)
32.30
93.06
93.15
71.96
38.72
88.03
93.44
69.15
ATOM (g-0.75)
38.56
91.20
58.53
62.66
78.70
97.97
54.89
Table 2: Ablation study on informative outlier mining. We DenseNet network architecture. indicates larger value better, and & indicates lower value better. All values are percentages and are averaged over six natural OOD test datasets mentioned in section 5.1. We do not use OOD test set for tuning Please refer to Table 5 for validation results.
Our training method leads to improved OOD detection while preserving classification performance on in-distribution data. Consistent performance improvement is observed other in-distribution datasets (SVHN and CIFAR-100 100), alternative network architecture (WideResNet), and with alternative auxiliary dataset (lmageNet-RC).
How does ATOM compare to NTOM? We perform ablation study that isolates the effect of adversarial training. In Table 1, we show that NTOM achieves comparable performance as ATOM on natural OOD and corruption OOD. However, NTOM is less robust under L. OOD (with 79.35% reduction in FPR on CIFAR-10) and compositional OOD inputs. This underlies the importance of having both adversarial training and outlier mining (ATOM) for overall good performance.
How does the sampling parameter affect performance? Table 2 shows the performance with different sampling parameter q. For all three datasets, training on OOD inputs primarily with large OOD scores (i.e., too easy examples with q 0.75) worsens the performance, which suggests the necessity to include examples on which the OOD detector uncertain about. We also show that using informative outlier mining overall works better than random sampling under properly chosen Interestingly, in setting where the in-distribution data and auxiliary OOD data are disjoint (e.g., SVHN/Tinylmagest q is optimal, which suggests that the hardest outliers are mostly useful for training. However, in a more realistic setting, the auxiliary OOD data can almost always contain data similar to in-distribution data (e.g., CIFAR/ (EEFAR/inyImages) Even without removing near-duplicates exhaustively, ATOM can adaptively avoid training on those -upr-dupl of in- an-distribution data (e.g. using g 0.125 for CIFAR-10 and q = 0.5 for CIFAR-100).
How does the choice of auxiliary OOD dataset affect the performance? To this, we addition- ally experiment with using ImageNet-RC as auxiliary OOD data. We observe consistent improvement of ATOM, and in many
with performance better than using TinyImages. For example, on CIFAR-100, the FPR under natural OOD inputs is reduced from 32.20% (w/ Tinylmages) to 15.49% (w/ ImageNet-RC), Interestingly, in all three datasets, using q 0 (hardest outliers) yields the optimal performance since there is substantially less duplicates between ImageNet-RC and in-distribution data. This ablation suggests that ATOM's success does not depend on a particular auxiliary dataset. Full results are provided in Table 6 (Appendix B.8).
RELATED WORK
Robustness of OOD Detection. Worst-case aspects of OOD detection have previously been studied in (Hein et al., 2019; Sehwag et al., 2019). However, these papers are primarily concerned with Loe norm bounded adversarial attacks, while
evaluation also includes common image corruption attacks. Besides, Meinke Hein; Hein et al. only evaluate adversarial robustness of OOD detection random noise images, while we also evaluate natural OOD images. Hein al. (2019) has
8

---

Under review as a conference paper at ICLR 2021
theoretically analyzed why ReLU networks can yield high-confidence but wrong predictions for OOD data. Meinke & Hein has shown the first provable guarantees for worst-case OOD detection on some balls around uniform noise, and Bitterwolf et al. recently studied the provable guarantees for worst-case OOD detection not only for noise but also for images from related but different image classification tasks. In this paper, we propose ATOM which achieves state-of-the-are performance on a broader family of clean and perturbed OOD inputs. The key difference of our method compared to prior work is introducing the informative outlier mining technique, which could significantly improve the generalization and robustness of OOD detection.
Discriminative Based Out-of- Diitii--iitiittiinn Detection. Hendrycks & Gimpel introduced a baseline approach for OOD detection using the maximum softmax probability from pre-trained network. Several works attempt to improve the OOD uncertainty estimation by using deep ensembles (Laksh- minarayanan al., 2017). the calibrated softmax score (Liang et al., 201 and the Mahalanobis distance-based confidence score (Lee et al., 2018). Some methods also modify the neural networks by re-training or fine-tuning on some auxiliary anomalous data that are either realistic (Hendrycks et al., 2018; Mohseni et al., 2020; Papadopoulos et al., 2019) or artificially generated by GANs (Lee et al., 2017). Many other works (Subramanya et al., 2017; Malinin & Gales, 2018; Bevandić et al., 2018) also regularize the model to have lower confidence for anomalous examples.
Generative Modeling Based Out-of-disribbutio Detection. Generative models (Dinh et al., 2016; Kingma & Welling, 2013; Rezende al., 2014; Van den Oord et al., 2016; Tabak & Tumner, 2013) can be alternative approaches for detecting OOD examples, as they directly estimate the in-distribution density and can declare a test sample be out-of-distribttion if it lies in the low-density regions. However, as shown by Nalisnick et al., deep generative models can assign a high likelihood to out- of-distribution data. Deep generative models can be more effective for out- cattof-distribution detection using alternative metries (Choi & Jang, 2018), likelihood ratio (Ren et al., 2019; Serrà et al., 2019), and modified training technique (Hendrycks et al., 2018). Recently, Pope et al. shows that flow- based generative models are sensitive under adversarial attacks. Note that we mainly considered discriminative-bate approaches, which can be more competitive due to the availability of label information (and, in some cases, auxiliary OOD data (Hein et al., 2019; Hendrycks et al., 2018; Meinke & Hein, 2019; Mohseni et al., 2020)).
Adversarial Robustness. Adversarial examples (Goodfellow et al., 2014; Papernot et al., 2016; Biggio et al., 2013; Szegedy et al., 2013) have received considerable attention in recent years. Many defense methods have been proposed to mitigate this problem. One of the most effective methods is adversarial training (Madry et al., 2017), which uses robust optimization techniques to render deep learning models resistant to adversarial attacks. Carmon et al.; Najafi et al.; Zhai et al.; Uesato et al. show that unlabeled data can improve adversarial robustness for classification. In particular, the analysis in Carmon et al. (2019) has inspired our analysis in the Gaussian data model. But these studies are for robust classification, while our work focuses on roubst OOD detection where the key challenge is that the auxiliary OOD and test OOD distributions can differ.
Hard Example Mining. Hard example mining was introduced in the work (Sung, 1996) for training face detection models, where they gradually grow the set of background examples by selecting those examples for which the detector triggers a false alarm. The idea has been used extensively for object detection literature (Felzenszwalb et al., 2009; Gidaris & Komodakis, 2015; Shrivastava et al., 2016). also has been used extensively in deep metric learning (Cui et al., 2016; Simo-Serra et al., 2015; Wang & Gupta, 2015; Suh et al., 2019) and deep embedding learning (Yuan et al., 2017; Smirnov et al., 2018; Wu et al., 2017; Duan et al., 2019). To the best of our knowledge, we are the first to explore hard example mining for out-of-distribution detection.
CONCLUSION
In this paper, we propose Adversarial Training with informative Outlier Mining (ATOM), a method that enhances the robustness of the OOD detector. We show the merit of adaptively selecting the OOD training examples which the OOD detector is uncertain about. Extensive experiments show ATOM can significantly improve the decision boundary of the OOD detector, achieving ot the-art performance under a broad family of clean and perturbed OOD evaluation tasks. We also provide theoretical analysis that justifies the benefits of outlier mining. Further, our unified evaluation framework allows future research to examine the robustness of the OOD detector. We hope our research can raise more attention to a broader view of robustness in out- distriibibtttinn detection.
9

---

Under review as a conference paper at ICLR 2021
REFERENCES
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint orXiv:/666.000 2016.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint
2018. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1-2)11511175, 2010.
Petra Bevandić, Ivan Krešo, Marin Oršić, and Siniša Šegvić. Discriminative out-of-distribution detection for semantic segmentation. arXiv preprint
2018.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šmdić, Pavel Laskov. Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Julian Bitterwolf, Alexander Meinke, and Matthias Hein. Provable worst case guarantees for the detection of out-of-distribution data. arXiv preprint XXXi:220070084733 2020.
Timothy A Blauwkamp, Simone Thair, Michael J Rosen, Lily Blair, Martin S Lindner, Igor D Vilfan, Trupti Kawli, Fred C Christians, Shivkumar Venkatasabrahhanyamm Gregory D Wall, et al. Analytical and clinical validation of microbial cell-free dna sequencing test for infectious disease. Nature microbiology, 4(4):663-674 6744 2019.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In Advances Neural Information Processing Systems, pp. 11190-1 1201, 2019.
Hyunsun Choi and Eric Jang. Generative ensembles for robust anomaly detection. 2018.
Patryk Chrabaszez, Ilya Loshchilov, and Frank Hutter. A downsampled variant of imagenet as an alternative to the cifar datasets. arXiv preprint xrXiv:1707.08819 2017.
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Puttern Recognition (CVPR), 2014.
Yin Cui, Feng Zhou, Yuanqing Lin, and Serge Belongie. Fine-grained categorization and dataset bootstrapping using deep metric learning with humans in the loop. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1 1153-1162, 2016.
Laurent Dinh, Jascha Sohk-Dic,,,ein, and Samy Bengio. Density estimation using real nvp. arXiv preprint /605 160500803.. 2016.
Yueqi Duan, Lei Chen, Jiwen Lu, and Jie Zhou. Deep embedding learning with discriminative sam- pling policy. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4964 4973, 2019.
John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(uul)2121-21559 2011.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-worke attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625-1634, 2018.
Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. Object detection with discriminatively trained past-based models. IEEE transactions on pattern analysis and machine intelligence, 3279): 2009.
Spyros Gidaris and Nikos Komodakis. Object detection via a multi-region and semantic segmentation- aware enn model. Proceedings of the IEEE international conference on computer vision, pp. 1134-1142, 2015.
10

---

Under review as a conference paper at ICLR 2021
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 41-50, 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. arXiv preprint arXiv:1903. 2019.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv: 1610.02/3, 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas G Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 18/2.0060000 2018.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4700-4708, 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6990, 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Cassidy Laidlaw and Soheil Feizi. Functional adversarial attacks. In Advances in Neural Information Processing Systems, pp. 10408-10418, 2019.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances neural information processing systems, pp. 6402-6413, 2017.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training comfidence-ccllibrateed classifiers for detecting out-of-distribution samples. arXiv preprint arXiv:1711.0932.. 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-disttibuttin samples and adversarial attacks. In Advances in Neural Information Processing Systems, pp. 71 7167-7177, 2018.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of- image detection in neural networks. In 6th International Conference on Learning Representations, ICLR 2018, 2018.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy- out-of-distriburt detection. Advances in Neural Information Processing Systems, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1706, 100..7006000000 06083, 2017.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In Advances in Neural Information Processing Systems, pp. 7047-7058. 2018.
Alexander Meinke and Matthias Hein. Towards neural networks that provably know when they don't 't know. arXiv preprint arXiv: 12180, 2019.
Sina Mohseni, Mandar Pitale, JBS Yadawa, and Zhangyang Wang. Self-supervised learning for generalizable out-of-distribution detection. 2020.

---

Under review as a conference paper at ICLR 2021
Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyato. Robustness to adversarial perturbations in learning from incomplete data. In Advances in Neural Information Processing Systems, pp. 5541-5551, 2019.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't 't know? arXiv preprini arXiv: 180.001// 2018. 2018.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Aristotelis-Angeloo Papadopoulos, Mohammad Reza Rajati, Nazim Shaikh, and Jiamian Wang. Outlier exposure with confidence control for out-of-distribution detection. arXiv preprint arXiv:1906.035099 2019.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P), pp. 372-387. IEEE, 2016.
Phillip Pope, Yogesh Balaji, and Soheil Feizi. Adversarial robustness of flow-based generative models. arXiv preprint arXiv:1911 0.6554, 2019.
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. In Advances in Neural Information Processing Systems, pp. 14680-14691, 2019.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference deep generative models. arXiv preprint 4rXiv:1401.4082 2014.
Ludwig Schmidt, Shihani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data. In Advances in Neural Information Processing Systems, pp. 5014-5026, 2018.
Vikash Schwag, Arjun Nitin Bhagoji, Liwei Song, Chawin Sitawarin, Daniel Cullina, Mung Chiang, and Prateek Mittal. Analyzing the robustness of open-world machine learning. In Proceedings of the 12th ACM Workshop Artificial Intelligence and Security, pp. 105-116, 2019.
Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F Núñezz and Jordi Luque. Input complexity and duitr--disttiuution detection with likelihood- generative models. arXiv preprint arXiv::909. / 1480, 2019.
Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 761-769, 2016.
Edgar Simo-Serra, Eduard Trulls, Luis Ferraz, lasonas Kokkinos, Pascal Fua, and Francesc Moreno- Noguer. Discriminative learning of deep convolutional feature point descriptors. In Proceedings of the IEEE International Conference on Computer Vision, pp. 118-126, 2015.
Evgeny Smirnov, Aleksandr Melnikov, Andrei Oleinik, Elizaveta Ivanova, Ilya Kalinovskiy, and Eugene Luckyanets. Hard example mining with auxiliary embeddings. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 37-46, 2018.
Akshayvarun Subramanya, Suraj Srinivas, and R Venkatesh Babu. Confidence estimation in deep neural networks via density modelling. arXiv preprint arXiv:1707.07011 2017.
Yumin Suh, Bohyung Han, Wonsik Kim, and Kyoung Mu Lec. Stochastic class-based hard example mining for deep metric learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7251-7259. 2019.
Kah-Kay Sung. Learning and example selection for object and pattern detection. 1996.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint a3Xiv: 2013.
12

---

Under review as a conference paper at ICLR 2021
Esteban G Tabak and Cristina V Turner A family of nonparametric density estimation algorithms. Communications on Pure and Applied Mathematics, 66(2):145-164, 2013.
Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. JEEE transactions on pattern analysis and machine intelligence, 30(1 1):1988-1970, 2008.
Jonathan Uesato, Jean- Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? arXiv preprint
139.,, 2019.
Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelenn decoders. In Advances in neural information processing systems, pp. 4790 4798, 2016.
Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2794-2802, 2015,
Chao-Yuan Wu, R Manmatha, Alexander J Smola, and Philipp Krahenbuhl. Sampling matters in deep embedding learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2840-2845 2017.
Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint 1044005504.67555 2015.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 03365, 2015.
Yuhui Yuan, Kuiyuan Yang, and Chao Zhang. Hard-aware deeply cascaded embedding. In Proceed- ings of the IEEE international conference computer vision, pp. 814-823, 2017.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.0714. 2016.
Runtian Zhai, Tianle Cai, Di He, Chen Dan, Kun He, John Hopcroft, and Liwei Wang. Adversarially robust generalization just requires more unlabeled data, arXiv preprint arXiv: /906. 1955005555 2019.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions pattern analysis and machine intelligence, 445-:444,,,, 2017.
13

---

Under review as a conference paper at ICLR 2021
Supplementary Material
Informative Outlier Matters: Robustifying Out-of-distribution Detection Using Outlier Mining
A THEORETICAL ANALYSIS
In this section, we provide theoretical analysis to the following two questions: 1) Why are auxiliary OOD data useful for training, even when they come from a different distribution than the test OOD distribution? 2) For practical auxiliary OOD data which contain non-informative samples, how can we make use of the auxiliary data to significantly improve the detection performance?
For the first question, we provide an error bound to justify the benefit of auxiliary OOD data. We notice that a key difference of OOD detection from typical classification is that the test OOD distribution is not accessible for training, so need use the auxiliary OOD data from a different distribution. This makes OOD detection more challenging. Our intuition is that even if the auxiliary data different from test OOD data, they can still calibrate detectors in quite general situations, then the detector can generalize to the test OOD data. To formalize this, we adopt the domain adaption framework for our analysis.
For the second question, we provide analysis in a generative model of the data and motivate the importance of careful selection of informative auxiliary OOD data (i.e., informative outlier mining). Intuitively, since the auxiliary OOD data are different from the test OOD data, they may not be all useful. However, is unclear why outlier mining can lead to significant improvements (see our experimental results in Section 5) and how to formalize this. Our intuition is that some of the auxiliary OOD data can non-informative or even harmful, and they can overwhelm the benefit of informative outliers, leading to drastic drop in detection performance. To formalize it, one needs distributional assumptions on the data. We thus use a Gaussian data model and derive concrete bounds to illustrate our intuition.
A.1 GENERALIZATION FROM AUXILIARY OOD DATA TO TEST OOD DATA
To see why detectors trained on the auxiliary OOD data Ux can generalize to the test OOD distribution Qx. we adopt the domain adaption framework (Ben-David et al., 2010). Recall that in domain adaptation there are two domains 8, 6, each being a distribution over the input space X' and label space 1). A classifier is trained on then applied on t. At high level, we view our OOD detection problem as classification, where the source domain s is Px with labels and Ux with labels - 11, and the target domain t is Px with labels 1 and Qx with label -1.
We focus on the FPR metric below; the argument for FNR is similar. Suppose we learn the OOD detector from hypothesis class g. Following Ben-David et al. (2010), define (a variant) of the divergence of Qx and Ux w.r.t. the hypothesis class 9 as
dg (xx,Ux)=
sup (GG -(G, G G.G'Eg
where
v(G, D) = D.0) FPR(G';
is the error difference of G and G' on the distribution D.
The divergence upper bounds the change of the hypothesis error difference between Qx and Ux. If it is small, then for any G, G" e G where G has smaller error than G' in Ux. we know that G will also have a smaller (or not too larger) error than G" in Qx That is, if the divergence is small, then the ranking of the hypotheses w.r.t. the error is roughly the same in both distributions. This rank- ponk-presgriing property thus makes sure that a good hypothesis learned in Ux will also be good for Qx-
Now we show that, if dg xQx. Ux) is small (i.c., Qx and Ux are aligned w.r.t. the class 9), then a detector G with small FPR on Ux will also have small FPR on Qx.
14

---

Under review as a conference paper at ICLR 2021
G,
Figure 3: An illustration example to explain why Ux helps to get good detector G., With Ux, we can prune away hypotheses G. for any 1.9. Thus, the resulting detector G, can detect OOD samples from Qx successfully and robustly.
Proposition 2. For any G G.
FPR(G; Qx R)) inf FPR(G": FPR(G:UX, dg(Qx, Ux).
Proof. For simplicity, we omit Ω from FPR(G: Q). For any G E 9G, we have
FPR(G; Qx) = =FPR(G";xx) Qx) FPR(G; - FPR(G";
(8)
=FPR(G";Xx) FPR(G;UX) FPR(G";UX)
(9)
+ [[FPR(G:(x) FPR(G"; (FPR(G;UX) FPR(G";UX))
(10)
The last term is
(IPR(G:Qx) :TPR(G":Qx)) - (FPR(G:UX) - FPR(G);XX)
(11)
"(GGG";; o(G,G";Ux)
(12)
s dg(Qx, Ux).
(13)
Therefore,
FPR(G;QN < FPR(G"; + FPR(G;UX) + dg Ux).
(14)
Taking inf over G E G completes the proof.
The error of the detector is bounded by three terms: the best error, the error on the training distributions, and the divergence between Qx and Ux. Assuming that there exists a ground-truth detector with small test error, and that the optimization can lead small training error, the error is then characterized by the divergence. So in this case, long as the rankings of the hypotheses (according to the error) on Qx and Ux are similar, detectors learned on Ux can generalize to Qx-
An illustration example. In this example, the on-ditrribution Px is uniform over the disk around the origin in I with radius 1, Ux is uniform over the disk around (0, 3) with radius 1, and Qx uniform over the disk around (3, 0) with radius Assume the adversary budget e = 0.1, i.e., Ooo. { bc < 0.1). J. The hypothesis class for the detector contains all functions of the form (z) = |1| < r] - 1 with parameter r. See Figure 3.
The example first shows the effect of the auxiliary OOD data: Ux helps prune away hypotheses G, for any r 2 1.9. Furthermore, it also shows how learning over Ux can generalize to Qx. Although Qx and Ux have non-overlapping supports, Ux helps calibrate the error of the hypotheses, any good detector trained on Px and Ux can be used for distinguishing Px and Qx. Formally, the dg is small in Proposition 2.
15

---

Under review as a conference paper at ICLR 2021
The analysis also shows the importance of training perturbed instances from the auxiliary OOD data Ux. Not using perturbation is equivalent to using ! = {0). this case, the analysis shows that it only guarantees the enror unperturbed instances from Qx even if Qx and Ux has small divergence and the learned detector can have small training error on Ux.
A.2 IMPORTANCE OF OUTLIER MINING: ANALYSIS IN A GAUSSIAN DATA MODEL
To understand how the outlier training data affect the generalization, we study a concrete distributional model, which is inspired by the models in (Schmidt al., 2018; Carmon al., 2019). In this model, we establish in Section A.2.2 a separation of the in-distribution sample sizes needed in the two cases: with and without auxiliary OOD data for training. We also demonstrate in Section A.2.3 the benefit of outlier mining when the auxiliary OOD data consists of uninformative outliers.
While the theoretical model is simple (in fact, much simpler than the practical data distributions), its simplicity is actually desired for our analytical purpose. More precisely, the separation of the sample sizes under this simple model suggests the same phenomenon can happen in more complicated models. This then means the auxiliary OOD data not only help training but are necessary for obtaining detectors with reasonable performance when in-distribution data is limited.
Gaussian Model. To specify distributional model for our robust OOD formulation, we need in-distribution Px. family of OOD distributions Q, and the hypothesis class H for the OOD) detector G. When auxiliary OOD data is available, also need to specify their distribution Ux. Let J Rd be the mean vector, > 0 the variance parameter, and 0 be parameter. our (n, - model:
Px is (u,
TRd, ||l|22 S v. Q = u,
(0x) x) : Rd. Here Gg(x) 1 means it predicts æ to be an in-distribution example, and Gx(x) 1 means it predicts an OOD example.
(Ga(z) = sign(0
We are interested in the False Negative Rate FNR(G) and worst False Positive Rate suPQxeR FPR(G; Qx (x)) over Qx E Q under Eoo perturbations of magnitude For sim- plicity, we denote them as FNR(G) and FPR(G; Qx) in our proofs.
Parameter Setting. The model parameters are set such that:
1. There exists a classifier that achieves very low errors FPR and FNR.
2. We need in-distribution data from Px learn classifier with non-trivial robust errors.
3. Using 7g in-distribution examples from Px and n' auxiliary OOD data from Ux where no is much smaller than n, we can learn classifier with non-trivial robust errors.
Here No. n, n' are sample sizes whose values are specified later in our analysis.
The family of instances of the Gaussian data model used for our analysis is as follows. First, fix an integer no O, and an e (0, 1/2). then set the following parameter values:
>> na/c + log² d, 11 E
99
11d
10
o² =²= /dna,
|1/12/44
(15)
10'
To interpret the parameter setting, one can view no, as fixed and d/no large number.
A.2.1 EXISTENCE OF ROBUST CLASSIFIERS
We give closed forms of the errors, and show that using 0 gives small errors under the chosen setting in equation 15. The calculation largely follows that in (Carmon et al., 2019) with some slight modification.
16

---

Under review as a conference paper at ICLR 2021
Closed Forms of the Errors. By definition, the FNR of detector Gø (on Px) is:
FNR(Ga) = Px 10 x S 0J = Px N
1
s 0
= +
(16)
o
where
f(x) = 1
e 12
(17)
22 x
the Gaussian error function.
Given a test OOD distribution Q = (- (- + 22 0)), the robust FPR Go on Qu is:
FPR(Go; Qc) = Px
inf T + +6) 2
(18)
[6
[0 + <|101| 2 0]
(19)
Px~Qu
[N[(u + 0 (a|10112) > 01]
(20)
uu++
<|1011
+ 0|1012
(21)
Then the worst robust FPR of Go on is:
(1u + v) To
< <|
sup FPR(Go; Qo) Q.EQ
sup + Bulles
(22)
o 0100|2
o ||0
1
V
<|101|
+
(23)
0
0101 2
To
V
cvd
< Φ
(24)
0
Small Errors of Cy.. Given the closed forms, now show that G₁, achieves small FNR and FPR in our parameter setting.
FNR(G,)) =
|1112
11
d
1/4
Se
Va/no
(25)
10
no
|A
evd
sup FPR(G,, Qv) S + Q.
(26)
1/4
S e da da/no
<
(27)
10
Therefore, the regime d/no > the detector Gj, achieves both small FNR on Px and robust FPR on any test OOD distribution in Q.
A.2.2 BENEFIT OF AUXILIARY OOD DATA
We will first consider the case when auxiliary OOD data are not available, and give lower bound (Proposition 3). We will then consider the case when auxiliary OOD data are used, and give an upper bound (Proposition 4). Comparing Proposition and Proposition will then justify the benefit the auxiliary OOD data.
Learning Without Auxiliary OOD Data. the detector Gg, given by
Given in-distribution data X1, X2,..., Xp,, we consider
0,
(28)
i=
17

---

Under review as a conference paper at ICLR 2021
Next we show a lower bound of the in-distribution data needed for the case without auxiliary outliers.
<² <vd/noo necessary for all algorithms to obtaín both non-trivial That is, sample size of order nig logd
robust FPR and FNR. We emphasize that this lower bound is information theoretic, i.c, it holds without restriction on the computational power of the learning algorithm and the hypothesis class used for the OOD detector.
Proposition 3. (Bound without auxiliary data) Consider the same family of instances in Preposi-
2 Lyddno
tion I, without any auxiliary OOD data. If n S no 16 d then for any algorithm bas there exists an instance in the family with
I ENR(A. + sup (S): Qx, (x)) (11 -- QXE
(29)
Proof. The key for the proof is the observation that robust classification is special case of our robust OOD problem. More precisely, consider the following robust classification problem. The data (x, y) with and then draw x from (3 jt, I² Given training data {(x,, y:) the goal is to find classifier fo(x) = sign(0" x) with small robust classification error
Rd and y E [11,+1] is generated as follows: first draw y uniformly at random,
(fo) = Exx..) max [[f((x + 8) + [J Sc
under for perturbation of magnitude e. has been shown that (Theorem in (Schmidt al., 2018)
x² /d/dooo and with the or Theorem in (Carmon et al., 2019)) that when NN 7) and n S no "llod"" d parameter setting equation 15, for any learning algorithm An
e (An (5)) 2 (1 - )
(30)
Now consider the following variant of the robust OOD problem in the proposition. Suppose μ (0 /). Suppose besides the data from Px, we also have i.id. samples from test OOD distribution Qo (-,,, I)) Then the above robust classification problem can be reduced to this variant of robust OOD, by viewing the in in-distribution data as with label +1 and viewing outliers as
with label 1. Furthermore, is clear that the sum of the FNR and FPR larger than the robust classification error. Then
E (FNT(A,, (S)) FPR(A, Qn)) 2 (1-d")
(31)
When d is sufficiently large, we have satisfies the condition in equation 15 with probability at least 9/10. 10. Then
|| (9d/10, 11d/10)) 2 1 ( (1-d-)). (32)
(FNR(A,(S) + (S):A.
After conditioning, this variant can be reduced to the original robust OOD problem in the proposition and furthermore Qu E Q. Furthermore, the fact that the expectation over the conditional distribution
of is large implies that there exist an instance of with a large error. The statement then follows. L
Learning With Auxiliary OOD Data. Assuming we have access to auxiliary OOD data from a distribution Ux where:
Ux is defined by the following distribution: first draw o uniformly at random from the ball {u Rd || S v), then draw x from from NN
v, 12).
Roughly speaking, Ux is uniform mixture of distributions in Q.
Given in-distribution data x1 X2. Xn from P' and auxiliary OOD data X1, X2,..., x,, from Ux. we consider the detector G given by
nn
(33)
==1
18

---

Under review as a conference paper at ICLR 2021
We will show that with ng and sufficiently large n', the detector has small errors. Again, as shown in the closed form solutions, the key factor determining the errors is 1 '' The following lemma bounds this term.
al10m,nt
Lemma 1. There exist numerical constants Co. C1 C2 such that under parameter setting equation 15 and d/no co.
0110n.''11 ll2
H On
10
( d
no
n n¹
no
1 11+0 c1
100
)² d
1/8
-1/2
(34)
with probability 2 1 -d((d/m)) minnnnta (idn(u+a"(dd/o))"/ 1/4 egn
Proof. The proof follows the argument of Lemma 1 in (Carmon et al., 2019) but needs some modifications accommodating the difference learning 0. Recall the generation of x,: first draw vs uniformly at random from the ball (B()) = {u v E Red ||il|2 S v), then draw x', from N(u, o² T),
and finally let x = X' So we have
On
Σ X1
X
Σ U;
(35)
n + n
n + n
5 + 6,
(36)
where
8
1
Σ
n
x; +
Σ x's
- p
N(0.
o²
I),
(37)
n2 + n
n2 + n'
8
Ui
(38)
n + n
To lower bound the term
upper bound its squared inverse:
10 n... la"
110 m0
|| + 8 + 5
(39)
(1 ))
+ 21 + 1t 8,)²
110 + 60.12 1 (μ T + p T 8.)²
(40)
|
(1121 + F. 8 + JA 5,)²
2 2| + 2 5,
< 1
(41)
(1] - 4 8 + 8z)
For 5, we have
To
o²
||812 + ni Xa and
N 0,
(42)
||
+
So standard concentration bounds give
IP [1011
²² n + n
d 1 1)
Se 4//8/ and IP
2
(01111)1/22
s 2e (n+n')llella/2
|All1
(43)
For 8,, by subguassian concentration bounds, we have
Cv een
P [18,1122 2
(44)
for some numeric constants and C. Suppose the event 11ó.112
Cv is true. Then Vn
Cullulla
| T del < 11l12211801122 $
(45)
Vn'
19

---

Under review as a conference paper at ICLR 2021
Plugging the concentration bounds in equation 39 and doing the same manipulation leads to the bound. To finish the proof, we also need to show J On,n' O, which can be shown by similar argument as above.
We then get the following guarantee. Again, the error bound 1 is chosen for simplicity of the statement, but it can be made to arbitrarily small values. Proposition 4. (Error bound with ideal auxiliary data) Consider the same family of instances as in Preposition I, with n' auxiliary OOD data from Ux specified above. If n. 2 no and n' 2 70 462 n/o then
1 B FNR(G 0 1000
BB
Bg
sup ExEQ
F ix,, Sxo,r S 10-3
(46)
Proof. The proposition comes from Lemma 1, the parameter setting equation 15, and the closed form expressions equation 16 and equation 22 of the errors.
A.2.3 BENEFIT OF OUTLIER MINING
The above Gaussian example shows the benefit of having auxiliary OOD data for training. All the auxiliary OOD data given in the example are implicitly related to the ideal parameter for the detector 8r = μ and thus informative for learning the detector. However, this may not be the case in practice: typically only part of the auxiliary OOD data are informative, while the remaining are not very useful or even be harmful for the learning. In this section, study such example, and shows that how outlier mining can help to identify informative data and improve the learning performance.
Suppose the algorithm gets n in-distribution data {x1, x....., OOD data {x1, x X,,' data are i.i.d. from the distribution U Unix-
..... Xn i..dd from P'x and n' auxiliary } for training. Instead of from Ux specified above, the auxiliary OOD
Umi is uniform mixture of 021- and N o²1) for uoo 10p.
That is. the distribution is defined by the following process: with probability 1/2 sample the outlier from the informative part (N I²)), and with probability 1/2 sample the outlier from the unin- formative part (NHo: o² I). We also note that Ao = 10p is chosen for simplicity of analysis. μıg can also be eu for some sufficiently large > 1, or even to cy + d' for sufficiently large e > 1, a small c' and unit vector 1 perpendicular to j.. Our analysis still go through with such assumptions.
Naïve Method Without Outlier Mining, It is clear that naively applying the method in the previ- ous section can lead to high errors: with in-distribution examples from Px and n' = n auxiliary OOD data from Umix, when n + oo, we have 0n + -7u/4 which has the worst errors among all detectors.
With Outlier Mining. Here we analyze the following algorithm using the outlier mining approach. The algorithm is simpler than what we used in Section 4 but shares the same intuition.
First, we use the in-distribution data points to get an intermediate solution:
0mn
X
(47)
We define the confidence score of point x being in-distribution as:
0m
f(x) - o = T where 1(x)
(48)
d
Here (t) 1 is the sigmoid function. We then select outlier training data whose confidence fall into an interval (a. b] and use them to learn the final solution:
En' (- (x.)((( E [a, 6
dom
(49)
1 ((f(x)) [a,
where TT.) is the indicator function.
20

---

Under review as a conference paper at ICLR 2021
Proposition 1. (Error bound with outlier mining.) For any (0,1/2) and any integer no there exist a family of instances of the Gaussian data model such that the following is true. n' auxiliary OOD data from Umix specified above. There exist thresholds a and b for dmm and a universal constant O such that if the number of in-distribution data Z c(no log + vdno) and the number of auxiliary data n' 2 (d + 1a0 4e / /d/no, then Dom has small errors:*
B FNR(G ) 100
B sup FPR(G SQx, : (x,, 10- QxEQ
(3)
Proof. - By definition we have
6om : 0om p 1- 1 8.) E 6]
(50)
(()()) E [a, 6]}
By the closed form expressions equation 16 and equation 22 of the errors, it sufficient to lower bound the key term
Bemm which comes down to show that 6am is small.
[|damll2
First, let's consider Oan Let dint = Oant μ. Then
a² Xa and
1 T 6int
(0. (0,
0²
16mml
N
(51)
n
|11l1
So standard concentration bounds give
(16 1 112
²² d (+t1)
S Se-d/s00 d/880 and P
luTous Siue ||
2e -d/20²
(52)
So with probability 2 3e 3//8/ over the randomness of the in-distribution points, we have the In T ômeel
good event Gint: | | < e² n (d+ and
S SVA
|
Now, condition on a fix 0. satisfying Gint, and consider d Define
2; = -u &
(53)
Lo: :- 1x((x)) E (a, b]},
(54)
I I(x; is from ((-,,0²2))
(55)
I2 1(x; is from (Nuo, o²])].
(56)
For simplicity, let's omit the subseript and consider sample X from Umixx and the corresponding variables 2, To, I, and I2. Since I, + I12 = 1,
()( E [a, b|} = 21lo11 + 2IoI2.
(57)
Case 1. Let's first consider the case when & is from NV (f- I2 More precisely, we condition on fixed dmm and condition on 11 = 1. Then N(00 and it can be decomposed along the direction pm = / ||mmall follows:
0.. 22
(58)
where s ~ _N(0, 02) and 22 is Gaussian distribution the subspace orthogonal to 0 Then
x TTut
11 Trmm
*110mttll
((x)
(59)
d
d
Therefore, we have
E[ 1, )ml] E[s 0o1 1, 0mt] E[ [I] 0
(60)
"The error bound in the proposition can be made arbitrarily small and with high probability. The current bound presented for simplicity.
21

---

Under review as a conference paper at ICLR 2021
Clearly the second term is 0 since zıIu is symmetric. So
II 1 1, Aae)]
E[s I(( [a, b|} Brtt]
(61)
s.. Iis E [T' 1, 0,0m) Oine
(62)
E 18 11 la UJH Pint
(63)
where
d' 1 Omn
σ (b)
(64)
10mell2
110
22'' Oine + d
(65)
2110ueell2
-2p² Oint
2111113 + d
(66)
2|10ine 1l
1 Oint
o (a)d
6'
(67)
10mell2
110mel12
2/ Piue + +3d
(68)
2/|0ine ll2
2/ Oint 211m113 + 3d
(69)
2 2|[0mnt ll
By the bound on 1ee di we have
(12/10+4/V Pan
[R (s T(s E [1,
al
12
(70)
(8/10-4/v7)
22T
2o liat
d
2d
1
(71)
20 0intll
20
ll
Given the bound on || 122, we have
02 V
1
2
| 2/2 S || + [|0 ll2 S 1|a||2 +
d
+
I
||wi + V
(72)
n
n
02
1
2202d
|10mtll2 2 2 || 118me ll2 2 ||al|2
d
+
||
V
(73)
n
n
Since n 2 Org log and > nzno
log² for sufficiently large C, we have
0nt 1
d2d(11/10 + 2202 /n)²
Sno
12
(74)
d2
de
d
C log d
and thus
d
-
E [s.. 11. E [d'1 5
(75)
o
00
2
df
320 1
(76)
0
m0
(77)
d
Combining with E[ ||I = 0, 0ml] = O we get
E E [mml] C1 Ouut
(78)
22

---

Under review as a conference paper at ICLR 2021
for some satisfying lq - 1/dl. Furthermore, 21ا4 Out is truncated Gaussian and thus is sub-Gaussian with Gaussian norm bounded by a. Then by sub-Gaussian concentration bounds, we have
n'' Σ
n
P
p
Σ
TU 2 [mal 2 vn'd 6mm
I =e-oo/o
(79)
Σ
2 40Vn'd 2 Omm
S Se-dio²
(80)
for some constant с > 0. In other words, with probability 2 11-1 22 od/aa we have
n'
I vn'd+ n'
(81)
d3/27
2 Iooott11
$ 26oVn'd
(82)
2
Conditioned on I = 1, we also have
B| III
1, Omne
P(x E la', bJ)
(83)
Ja Pine (8/10-1\vm)
(84)
-8100+A/V7
22T
+00
1
1122 dt
2 1 - 2
(85)
22T
Ar 12
1/
2 1 2
c
(86)
Vollgd
22T
12
21 d'
(87)
Let m 2/ (1 (1-#) 2). Then by Chernoff's ' bound, we have
n'
n'
P
Io:ll
m
2 m
1
I Se-mm
(88)
2
for an absolute constant d' > O. That is, with probality 2 1 - cnn' we have Σ 1ol11 2 n' /5.
Case 2. Next, let's consider the case when x is from (Ho,, o²1). More precisely, we condition on fixed 6 and condition on I, 1. Similar to case we have
11 Oin
(89)
where 8 ~ o²) and ø2 is Gaussian distribution subspace orthogonal to 0 So
[[[ + 11pTToo222 1, Omt] [[ Ta 1, fm0m)) Omt.
(90)
For this,
[( E (T, 1, (m0)) 0m. E. [s T(s [a". "0TH O.nt
(91)
where
d" -20 dime 20 1 + d
(92)
2110 l|z
6" -20,²² dimt 20 10||011
3d
(93)
0m 1|
23

---

Under review as a conference paper at ICLR 2021
By the bound on 1 8iut and ||6mm I12,
we have
2o 2018mell2 (10-20/vD)
E [s- 18 E 1'H
01
12 12 12
(94)
221 vn)
2
201liine 12
8d
22d
1
- 14u
)²
20114im 12
(95)
20 1
1|
2x 00nt
112
2
44d
20
2 1
(96)
|| mm
l1
2 d2
(97)
We also have
Iolle II
1, mm)
E [Ifs b"[11
(98)
1inel2 (15-20/ vT)
1
12
(99)
(21+20/
2tT
2o 10mlll2
1
3ed
(6 46+40/VT)
2010mal22
(100)
20 1inll12
22T
(101)
d3
Combining the above, have
[ + 1144)[0I2 [mml] = c1 Oiat
(102)
for constant 01 satisfying |c1| s 1/d². Furthermore, (z + 114) Pmt is truncated Gaussian and thus is sub-Gaussian with sub-Gaussian norm bounded by a. Then by sub-Gaussian concentration bounds, we have
n'
P
Σ
p (z+ + 11pp
Σe
BB + 121 (mat)] 2 Vn'd 0ame
e =nd/a²
(103)
n' 1op
P
Σ
(2:
2 4o 2Vn'd
S Se-di <e-dfo²
(104)
for some constant e > 0. Also by Hoeffding's bound, we have
n''
n
P
T2:
Σ
E Dos T2 0ml]
2 Vn'djo2
1mm
1 2e -2d/0²
(105)
In other words, with probability 2 odd/0 we have
p T 22
Vn'd + 22dVn'do + n' d
Vd
22d
n'
Vn'd
1+
22d
n'
d//2
(106)
n
Σ 2, Il:
S < 22vd
n'd n' (V
S Vn'd
60 + (60 6o0
d
+
22n'
d5/2
(107)
24

---

Under review as a conference paper at ICLR 2021
Combining equation 79, equation 80, equation 88 and equation 106, equation 107 together, get with probability 2 1 Ceeca//"
d (
22d
C
|e T daml - CV rl
1
+
(108)
d3/2
d
C
| l|2 S
CV n
60 + 2
+
(109)
0²
d5/2"
Then H² a0mm can be lower bounded by 110mml1a
1 0om
+ dom
(110)
110ml
1e domn l2
/4 T /e + / dom
(111)
|| + |10mm l|
d(1 1/vd)
(112)
vd(1 + 1/
2 vd 1 Vdl
(113)
The proof is completed by plugging the above into the closed form expressions equation 16 and equation 22 of the errors.
B DETAILS OF EXPERIMENTS
B.1 EXPERIMENTAL SETTINGS
Software and Hardware. We run all experiments with PyTorch and NVIDIA GeForce RTX 2080Ti GPUs.
Number of Evaluation Runs. We run all experiments once with fixed random seeds.
In-distribution Datasets. We use SVHN (Netzer al., 2011), CIFAR-10 and CIFAR- 100 (Krizhevsky et al., 2009) as in-distribution datasets. SVHN has 10 classes and contains 73,257 training images. CIFAR-10 10 and CIFAR- 100 have 10 and 100 classes, respectively. Both datasets consist of 50,000 training images and 10,000 test images.
Auxiliary OOD Datasets. We provide the details of auxiliary OOD datasets below. For each auxiliary OOD dataset, we use random cropping with padding of 4 pixels to generate 32 x 32 images, and further augment the data by random horizontal flipping. We don't 't use any image corruptions to augment the data.
1. TinyImages. 80 Million Tiny Images (TinyImages (Torralba et al., 2008) is dataset that contains 79,302,017 images collected from the Web. The images in the dataset are stored as 32 x 32 color images. Since CIFAR- 10 and CIFAR- 100 are labeled subsets of the Tinylmages dataset, we need remove those images in the dataset that belong to CIFAR- 10 CIFAR-100. We follow the same deduplication procedure as in (Hendrycks et al., 2018) and remove all examples this dataset that appear in CIFAR-10 10 or CIFAR- 100. Even after deduplication, the auxiliary OOD dataset may still contain some in-distribution data if use CIFAR-10 or CIFAR-100 as in-distribution datasets, but the fraction of them low.
2. ImtgeNet-RC. We use the downsampled ImageNet dataset (ImageNet64 x 64) (Chrabaszez et al., 2017), which is downsampled variant of the original ImageNet dataset. It contains 1,281,167 images with image size of 64 x 64 and 1,000 classes. Some of the classes overlap with CIFAR-10 10 CIFAR-100 classes. Since we don't use any label information from the dataset, we can say that the auxiliary OOD dataset is unlabeled. Since we randomly crop the 64 x 64 images into 32 32 images with padding of 4 pixels, with high probability, the resulting images won't 't contain objects belonging to the -n-distribution classes even if the original images contain objects belonging to those classes. Therefore, still can have
25

---

Under review as a conference paper at ICLR 2021
lot of OOD data for training and the fraction of in-distribution data in the auxiliary OOD dataset is low. We call this auxiliary OOD dataset ImageNet-RC.
OOD Test Datasets. We provide the details of OOD test datasets below. All images are of size 32 32.
1. SVHN. The SVHN dataset (Netzer et al., 2011) contains color images of house numbers. There are ten classes of digits 0-9. The original test set has 26,032 images. We randomly select 1,000 test images for each class and form a new test dataset of 10,000 images for evaluation.
2. Textures. The Describable Textures Dataset (DTD) (Cimpoi et al., 2014) contains textural images in the wild. We include the entire collection of 5640 images for evaluation.
3. Places365. The Places365 dataset (Zhou et al., 2017) contains large-scale photographs of scenes with 365 scene categories. There are 900 images per category in the test set. We randomly sample 10,000 images from the test for evaluation.
4. LSUU (crop) and LSSUN (resize). The Large-scale Scene UNderstanding dataset (L.SUN) has testing set of 10,000 images of 10 different scenes (Yu et al., 2015). We construct two datasets, LSUN-C and LSUN- R, by randomly cropping image patches of size 32 x 32 and downsampling each image size 32 32, respectively.
5. iSUN. The iSUN (Xu et al., 2015) consists of subset of SUN images. We include the entire collection of 8925 images in iSUN.
6. CIFAR-10. We the test set of CIFAR-10, which contains 10,000 images.
7. Gaussian Noise. The synthetic Gaussian noise dataset consists 10,000 random 2D Gaussian noise images, where each RGB value of every pixel is sampled from i.i.d Gaussian distribution with mean 0.5 and unit variance. We further clip each pixel value into the range [0.1].
8. Uniform Noise. The synthetic uniform noise dataset consists 10,000 images where each RGB value of every pixel is independently and identically sampled from uniform distribu- tion on [0.1].
Architectures and Training Configurations. We use the state-of-the-ar neural network architecture DenseNet (Huang et al., 2017) and WideResNet (Zagoruyko & Komodakis, 2016). For DenseNet, we follow the same setup as in (Huang et al., 2017), with depth L = 100, growth rate k 12 (Dense-BCC and dropout rate For WidcResNet, we also follow the same setup as in (Zagoruyko & Komodakis, 2016), with depth of 40 and widening parameter k = 4 (WRN-40-4). All neural networks are trained with stochastic gradient descent with Nesterov momentum (Duchi al., 201 1; Kingma & Ba, 2014). We set momentum 0.9 and l weight decay with coefficient of 10 for all model training. Specifically, for SVHN, we train the networks for 20 epochs and the initial learning rate of 0.1 decays by 0.1 at 10, 15, 18 epoch; for CIFAR-10 and CIFAR-I00,
train the networks for 100 epochs and the initial learning rate of 0.1 decays by 0.1 at 50, 75, 90 epoch. ATOM and NTOM, use batch size 64 for in in-distribution data and 128 for out-of-diitribution data. To solve the inner max of the robust training objective in ATOM, we use PGD with e 8/255, the number of iterations of 5, the step size of 2/ /255, and random start.
B.2 AVERAGE RUNTIME
We run our experiments using a single GPU on a machine with 4 GPUs and 32 cores. The estimated average runtime for each method is summarized in Table 3.
B.3
OOD DETECTION METHOPS
We consider eight common OOD detection methods listed in Table and describe each method in detail below.
Maximum Softmax Probability (MSP). Hendrycks Gimpel propose to use max; Fx(x)) as confidence scores to detect O0l) examples, where F(x:) is the softmax output of the neural network.
26

---

Under review as a conference paper at ICLR 2021
Method
Training
Evaluation
MSP
2.5 h
4 h
ODIN
2.5 h
h
Mahalanobis
2.5
20
SOFL
14 h
4 h
OE
5
4 h
ACET
17 h
4
CCU
6.7
hh
ROWL
24 h
4 h
ATOM (ours)
21 h
4
Table 3: The estimated average runtime for each result. We use DenseNet network architecture. h means hour. For MSP, ODIN, and Mahalanobis, we use standard training. The evaluation includes four OOD detection tasks listed in Section 2.
ODIN. Liang al. computes calibrated confidence scores using temperature scaling and input perturbation techniques. In all of our experiments, we set temperature scaling parameter T 1000. We choose perturbation magnitude 17 by validating on 1000 images randomly sampled from in- distribution test Diest and 1000 images randomly sampled from auxiliary OOD dataset Dousiliury which does not depend on prior knowledge of test OOD datasets. For DenseNet, we set 0.0006 for SVHN, 1] = 0.0016 for CIFAR-10. and 7j = 0.0012 for CIFAR- 100. For WideResNet, we set 0.0002 for SVHN, 1 = 0.0006 for CIFAR-10, and 17 = 0.0012 for CIFAR-100.
Mahalanobis. Lee al. propose to use Mahalanobis distance- based confidence scores to detect OOD samples. Following Lee et al., we use 1000 examples randomly selected from in in-distribution test set Diest and adversarial examples generated by FGSM (Goodfellow et al., 2014) on them with perturbation size of 0.05 to train the Logistic Regression model and tune the noise perturbation magnitude v)- 1) is chosen from {0.0, 0.01, 0.005, 0.002, 0.0014, 0.001, 0.0005), and the optimal parameters chosen to minimize the FPR FNR 5%.
Outlier Exposure (OE). Outlier Exposure (Hendrycks et al., 2018) makes use a large, auxiliary OOD dataset Demsillary to enhance the performance of existing OOD detection. We train from scratch with A = 0.5, and use in-distribution batch size of 64 and out-distribution batch size of 128 in experiments. Other training parameters are specified in Section B.
Self-Supervised OOD Feature Learning (SOFL). Mohseni et al. add auxiliary head to the network and train in for the OOD detection task. They first use a full-supervised training to learn in-distribution training data for the main classification head and then a self-supervised training with OOD training set for the auxiliary head. Following the original setting, we set À = 5 and use an in-distribution batch size of 64 and an out-distribution batch size of 320 in all of our experiments. In SVHN and CIFAR-10, we use 5 reject classes, while in CIFAR-100, we use 10 reject classes. We first train the model with the full-supervised learning using the training parameters specified in Section B. and then continue to train with the self- self-supervised OOD feature learning using the same training parameters. We use the large, auxiliary OOD dataset D Dauxiliarry as out-of-die dfsttiiiubutio training out
dataset.
Adversarial Confidence Enhancing Training (ACET). Hein et. propose Adversarial Confidence Enhancing Training to enforce low model confidence for the OOD data point, as well as worst-case adversarial example in the neighborhood of an OOD example. We use the large, auxiliary OOD dataset Dauxiliary as an OOD training dataset instead of using random noise data for fair comparison. In all of our experiments, we set À = 1.0. For both in-distribution and out-distribution, we use a batch size of 128. To solve the inner max of the training objective, we also apply PGD with e = 8/255 the number of iterations of 5, the step size of 2/255, and random start to half of minibatch while keeping the other half clean to ensure proper performance on both perturbed and clean OOD examples for fair comparison. Other training parameters are specified in Section B.
Certified Certain Uncertainty (CCU). Certified Certain Uncertainty (Meinke & Hein, 2019) gives guarantees on the confidence of the classifier decision far away from the training data. We use the same training set up as in the paper and code, except we use our training configurations specified in Section B.
27

---

Under review as a conference paper at ICLR 2021
MSP
ODIN
Mahalanobis
SOFL
OE
ACET
CCU
ROWL
ATOM
(ours)
Natural OOD
/
/
Loe OOD
X
X
X
/
Corruption OOD
X
X
X
X
X
X
X
X
Comp. OOD
X
X
X
X
X
X
X
X
/
Table 4: Common OOD detection methods and a family natural and perturbed OOD examples we considered.
Robust Open-World Deep Learning (ROWL). Sehwag et al. propose introduce additional background classes for OOD datasets and perform adversarial training on both the in- and out-of- distribution datasets to achieve robust spen-world classification. When an input is classified as the background classes, is considered as an OOD example. Thus, ROWL gives binary OOD scores (either 1) the inputs. In our experiments, sample data points from the large, auxiliary OOD dataset muxiliary to form the OOD dataset. To ensure data balance across classes, we include 7,325 OOD data points for SVHN, 5,000 OOD data points for CIFAR-10 10 and 500 OOD data points for CIFAR-I00 During training, we mix the in-distribution data and OOD data, and use batch size of 128. To solve the inner max of the training objective, we use PGD with = 8/255, the number of iterations of 5, the step size of 2/255, and random start. Other training parameters are specified in Section B.
only have one background class and randomly
B.4 ADVERSARIAL ATTACKS FOR OOD DETECTION METHOΟS
We propose adversarial attack objectives for different OOD detection methods. We consider a family of adversarial perturbations for the OOD inputs: (1) Leo -nom bounded attack (white-bon attack); (2) common image corruptions attack (black- box attack); (3) compositional attack which combines common image corruptions attack and T. norm bounded attack (white-box attack).
Lox norm bounded attack. For data point x E Rd, the Lox norm bounded perturbation is defined as
(x) = {8 E Rd
|| S A x + 6 is valid),
(114)
where is the adversarial budget. x + 8 is considered valid if the values of x + 8 are in the image pixel value range.
For MSP, ODIN, OF, ACET, and CCU methods, we propose the following attack objective to generate adversarial OOD example on clean OOD input x:
x' arg max
Σ log F(x');
(115)
K
where F(x) is the softmax output of the classifier network.
For Mahalanobis method, we propose the following attack objective to generate adverasrial OOD example on OOD input x:
x' = arg max - llog
(116)
(2 Me(x))
x' (x)
where Mx(x') the Mahalanobis distance-based confidence score of x' from the f-th feature layer, { and b are the parameters of the logistic regression model.
For SOFL method, propose the following attack objective generate adversarial OOD example for an input x:
K+R Σ F(x');
x' arg max
log
(117)
x' (x)
i=K+1
where Fxx) is the softmax output the whole neural network (including auxiliary head) and the number of reject classes.
28

---

Under review as a conference paper at ICLR 2021
For ROWL and ATOM method, we propose the following attack objective to generate adverasrial OOD example on OOD input x:
arg max - log Fx*)x+1 x0
(118)
where F(x) is the softmax output of the (K+1)-way neural network.
Due to computational concerns, by default, we will use PGD with 8/255, the number of iterations of 40, the step size of 1/255 and random start to solve these attack objectives. We also perform ablation study experiments the attack strength for ACET and ATOM, see Appendix B.10.
Common Image Corruptions attack. We use common image corruptions introduced in (Hendrycks Dietterich, 2019). We apply 15 types of algorithmically generated corruptions from noise, blur, weather, and digital categories to each OOD image. Each type of corruption has five levels of severity, resulting in 75 distinct corruptions. Thus, for each OOD image, we generate 75 corrupted images and then select the one with the lowest OOD score (or highest confidence score to be in in-distribution). Note that we only need the outputs of the OOD detectors to construct such adversarial OOD examples; thus it is a black-box attack.
Compositional Attack. For each OOD image, we first apply common image corruptions attack, and then apply the Lx -nom bounded attack generate adversarial OOD examples.
B.5 VISUALIZATIONS OF FOUR TYPES OF OOD SAMPLES
We show visualizations of four types of OOD samples in Figure 4.
B.6 HISTOGRAM OF OOD SCORES
In Figure 5, we show histogram of OOD scores for model snapshots trained on CIFAR-10 (in- distribution) using objective (4) without informative outlier mining. We plot every ten epochs for model trained for total of 100 epochs. We observe that the model quickly converges to a solution where OOD score distribution becomes dominated by easy examples with scores closer to 1. This is exacerbated as the model is trained for longer.
B.7 CHOOSE BEST Q USING VALIDATION DATASET
We create validation OOD dataset by sampling 10,000 images from the 80 Million Tiny Images (Torralba et al., 2008), which is disjoint from our training data. We choose q from {0, 0.125, 0.25, 0.5, 0.75). The results on the validation dataset are shown in Table 5. We se- lect the best model based on the average FPR at 5% FNR across four types of OOD inputs. Based on results, the optimal g is for SVHN, 0.125 for CIFAR- 10 and 0.5 CIFAR-100.
B.8 EFFECT OF AUXILIARY OOD DATASETS
We present results in Table 6, where we alternative auxiliary OOD dataset ImageNet-RC The details of ImageNet-RC is provided in Section B. We use the same hyperparameters as used in training with Tinylmages auxiliary data. For all three in-distribution datasets, find that using 0 results in the optimal performance.
B.9 EFFECT OF NETWORK ARCHITECTURE
We perform experiments evaluate different OOD detection methods using WideResNet, see Table For both ATOM and NTOM, we use the same hyperparameters those selected for DenseNet and find that it also leads to good results.
1.10 EFFECT OF PGD ATTACK STRENGTH
To the effect of using stronger PGD attacks, we evaluate ACET (best baseline) and ATOM Loc attacked OOD and compositionally attacked OOD inputs with 100 iterations and random restarts. Results provided in Table 8. Under stronger PGD attack, ATOM outperforms ACET.
29

---

Under review as a conference paper at ICLR 2021
(a) Natural OOD
(b) Loe OOD
(c) Corruption OOD
(d) Comp. OOD
Figure 4: Examples of four types of OOD samples.
B.11 EVALUATION ON RANDOM NOISE OOD DATA
We report the performance of OOD detectors using DenseNet random noise OOD test datasets in Table 9 (SVHN), Table 10 (CIFAR-10) and Table (CIFAR-100).
12 PERFORMANCE OF OOD DETECTOR AND CLASSIFIER ON IN-DISTRIBUTION DATA
We summarize the performance of OOD detector G(x) and image classifier f(x) on in-distribution test data. See Table 12 for DenseNet and Table 13 for WideResNet. From the results, we can see that ATOM improves the OOD detection performance while achieving in-distribution classification accuracy that is par with pre-trained network.
B. COMPLETE EXPERIMENTAL RESULTS
We report the performance of OOD detectors using DenseNet on each of the six natural OOD test. datasets in Table 14 (SVHN), Table 15 (CIFAR-10) and Table 16 (CIFAR-100).
30

---

Under review as a conference paper at ICLR 2021
AUROC
AUROC
(5% FNR)
(5% FNR)
(5% FNR)
(5% FNR)
Method
Natural OOD
Corruption
OOD
OOD
Comp. OOD
ATOM (q=0.0)
0.01
99.97
4,02
98.50
8,08
98. 14
96.79
43.77
(g=0.123
2.16
62.38
73.20
99
SVHN
ATOM (q=0.25)
1.87
99.38
41.86
94.20
75.85
57.75
100.00
1.26
ATOM (g=0.5)
2.73
99.18
45.02
93.85
83.01
49.38
99 99
1,96
ATOM (q=0.75)
4.97
98.8 83
56.51
92.13
85,98
40.89
100.00
1.16
ATOM (q=0.0)
5.34
98.35
43.61
91.85
22.08
92.81
59.17
83.56
ATOM (g=0.125)
4.77
98.31
27.49
5,42
93.76
CIFAR-
ATOM (q=0.25)
5,70
98.11
28.13
93.68
19.28
95.71
40.68
91.17
ATOM (q=0.5)
8.83
97.66
39.74
91.53
9.80
97.42
44.10
90.45
ATOM (q=0.75)
12.42
97.02
45.85
13.40
96.83
90.42
ATOM (q-0.0)
44.85
91.58
98.76
64.78
53.17
85.26
98.95
58.38
CIFAR-
ATOM (q=0.125)
36.75
92.90
96.22
73.33
38.79
91.42
96.33
71.74
ATOM (q=0.25)
34,66
92.62
94.13
73.86
35.84
91.24
94.33
70.35
ATOM (q=0,5)
35.04
91.36
89.28
71.78
36.76
90.21
89.57
70.62
(q=0.75)
87.95
59.34
78.00
92.94
$8.53
Table 5: Evaluate models on validation dataset. We use DenseNet as network architecture. 1 indicates larger value is better, and indicates lower value is better. All values are percentages and are averaged over six different OOD test datasets mentioned in section 5.1. Bold numbers superior results.
FPR (5% FNR)
AUROC
FPR
AUROC
FPR
AUROC
FPR
AUROC
Dist
Method
(5% ENR)
(5% FNR)
(5% ENR)
Natural OOD
Corruption OOD
Loc OOD
Comp. OOD
MSP
38.84
93.57
99.68
68.48
99.89
L.39
100.00
0.19
31.45
97.11
63.21
0.61
SVHN
Mahalanobis
22.80
95.57
93.14
60.78
97.33
8.
99.89
0.23
SOFL
0.02
99.99
5,93
98.57
58.53
68.85
67.34
61.42
OE
97.51
ACET
0.31
99.94
29.02
95.65
2.37
99.51
30.58
95.20
CCU
0.17
99.96
18.64
96.94
45.38
69.14
92.30
20.88
ROWL
2.04
98.87
55.03
72.37
77.24
61.27
99.79
$0.00
NTOM (ours)
0.02
99.99
6.39
98.54
59.27
68.53
58.
72.08
ATOM (ours)
0.02
99.99
7.03
98.38
0.14
99.95
7,30
98.32
MSP
91.79
58,35
100.00
100.00
ODIN
21.65
94.66
99.37
51.44
99.99
0.18
100.00
0.01
CIFAR.
Mahulunobis
26.95
90.30
91.92
43.94
95.07
12.47
99
1.58
10
SOFL
99.74
OE
9.70
98.35
49.84
91.76
91.30
43.88
98.82
31.12
ACET
10.72
98.01
53.85
90.19
17.10
96.01
55.21
89.78
CCU
92.34
ROWL
25.03
86.96
94.34
$2.31
99.98
49.49
100.00
49,48
NTOM (ours)
4.01
99.17
15.13
97.14
96.89
11.42
99.87
3.18
ATOM
4.08
99.14
96.94
7.46
98.50
18.35
96.60
MSP
78.05
76.11
100.00
30.04
100.00
2.25
100.00
2.06
ODIN
56.77
83.62
100.00
36.95
100.00
0.14
100.00
0.00
CIFAR-
Mahulanobis
42.63
42.96
100
SOFL
20.95
96.06
73.33
83.31
93.41
12.90
99.98
3.36
OE
18.52
95.27
86.83
66.95
96.27
18.79
99.97
4.88
ACET
19.79
94.76
70.04
26.23
81.95
CCU
19.44
95.05
84.11
69.09
84.89
35.85
99.61
15.67
ROWL
93.35
53.02
100.00
49.69
100.00
49.69
100.00
49.69
NTOM (ours)
18.09
87.97
ATOM (ours)
15.49
97.18
57.79
89.49
18.32
96.57
58.49
89.36
Table Comparison
competitive OOD detection methods. We use ImageNet-RC as auxiliary OOD dataset (see section B.1 for the details) for SOFL, OE, ACET, CCU, NTOM and ATOM. We use DenseNet as network architecture for all methods. We evaluate on four types of OOD inputs: (1) natural OOD, (2) corruption attacked OOD, (3) Loe attacked OOD, and (4) compositionally attacked OOD inputs. f indicates larger value better, and J. indicates lower value is better. All values are percentages and averaged over six natural OOD test datasets described in section 5.1. Bold numbers are superior results.
31

---

Under review as a conference paper at ICLR 2021
Histogram (Epoch 10)
Histogram of OOD Score (Epuch 20)
60000
130000
Number of Data Points
50000
Number Data Points
125000
40009
30000
75000
20009
50000
10000
25000
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
OOD Score
OOD Score
Histogram of OOD Score (Epoch 30)
Histogram of OOD Score (Epoch 40)
Number of Data Points
120000
Num ber Data Points
150000
100000
125000
60000
75000
40000
30000
20000
25000
0.9
0.4
8.6
0.3
1.0
0.2
0.4
0.6
0.5
1.0
OOD Score
OOD Score
Histogram of OOD Score (Epoch 50)
Histogram OOD Score (Epoch 60)
200000
300000
Number of Data Points
Number of Data Points
250000
150000
100000
150000
100000
50000
50000
0OD Score
OOD Score
Histogram (Epach 70)
Histogram (Epach 80)
300000
Number of Data Points
250000-
Number of Data Points
300000
200000
200000
150000
100000
100000
50000
50000
OOD Score
OOD Score
Histogram of 0OD Score (Epach
Histogram of OOD Score (Epoch 100)
350000
350000
Number of Data Paints
300000
Data Points
250000
250000
200000
150000
250000
100000
200000
50000
50000
0.8
OOD
Figure 5: On CIFAR-I0, we train the model with objective (4) for 100 epochs without informative outlier mining, For every 10 epochs, we randomly sample 400,000 data points from the large auxiliary OOD dataset and use the current model snapshot to calculate the OOD scores.
32

---

Under review as a conference paper at ICLR 2021
FPR (5% FNR)
AUROC
FPR (5% FNR)
AUROC
FPR FNR)
AUROC
FPR AUROC
FNR)
Method
Natural OOD
Corruption OOD
Tox OOD
Comp. OOD
MSP
42.78
100.00
ODIN
43,65
89.37
99.85
44.43
100.00
0.06
100.00
0.00
SVHN
Mahalanobis
6,94
98.47
80.91
78.92
93.44
39.42
99.70
2.69
SOFL
0.02
99.99
2.41
99.36
85.25
39.32
99.28
OE
0,58
99.87
21.97
96.00
70.66
47,36
96.63
5.87
ACET
0.43
99.92
18.
97.11
20.75
89.55
99
1.80
CCU
0.71
30.96
94.40
73.37
41.82
ROWL
1.46
99.13
45.18
77.28
86.95
56,39
99.94
49,90
NTOM (ours)
0.02
99.99
2,46
99.27
70.38
52.43
99.79
1.01
ATOM (ours)
98.65
95.50
MSP
53.81
91.10
100.00
50.30
100.00
5,48
100.00
5.43
ODIN
36.25
91.18
99.95
31.78
100.00
0.04
100.00
0.00
CIFAR-
Mahalanohis
10
SOFL
2.15
99.14
38.03
93.35
99.99
0.43
100.00
0.24
OE
3,03
98.83
56.00
91.19
99.97
0.26
100.00
0.04
ACET
3.19
98.74
35.78
94.25
40.22
89.52
56.08
87.76
CCU
2.61
98.75
36.31
93.75
99.93
0.86
99.99
0.41
ROWI,
17.21
90.79
86.09
56.35
99 94
49,43
99 99
49,41)
NTOM (ours)
1.86
99.10
23.77
94.58
0.84
ATOM (ours)
1,50
98.87
13.17
96.36
33,66
90.89
40,07
88,99
MSP
81.92
74.32
100.00
10
10000
3.20
100.00
3.14
ODIN
68.76
79.13
26.44
100.00
100.00
CIFAR-
Mahalanobis
43.48
86.02
95.78
42.25
95.83
17.41
99.92
3.55
SOFL
66.57
89.11
99.99
59.64
99 99
0.30
100.00
0.10
OE
46.14
58.55
ACET
46.79
91.67
99.76
62.48
95.66
41.38
99.99
36,65
CCU
48.35
91.41
99.95
$7.94
100.00
0.44
100.00
0.38
ROWL 96.93
51.34
100.00
49.81
100.00
49.81
100.00
49.81
NTOM (ours)
37,94
92.14
97.09
66.93
99.99
1.04
99.99
0.13
ATOM (ours)
30.52
93.90
94.46
74,96
44.30
89.68
95.06
74.25
Table 7: Comparison with competitive OOD detection methods. We WideResNet network architecture for all methods. evaluate on four types of OOD inputs: (1) natural OOD, (2) corruption attacked OOD, (3) Lro attacked OOD, and (4) compositionally attacked OOD inputs. f indicates larger value is better, and indicates lower value better. All values are percentages and are averaged over six different OOD test datasets described in section 5.1. Bold numbers superior results.
FPR
AUROC
FPR
AUROC
(5% FNR)
(5% FNR)
Diest
Method
Loo OOD
Comp. OOD
ACET
33.64
83.89
99.90
2.84
SVHN
ATOM (ours)
8.33
97.37
98.56
35.32
CIFAR-
ACET
76.66
76.29
97.08
51.35
ATOM (ours)
26.88
83.78
42.86
83.56
CIFAR-
ACET
87.13
48.77
99.82
33.67
100
ATOM (ours)
45.11
82.32
93.74
65.88
Table 8: Evaluation on La attacked OOD and compositionally attacked OOD inputs with strong PGD attack (100 iterations and 5 random restarts). We use DenseNet as network architecture for all methods. indicates larger value is better, and J. indicates lower value better. All values percentages and averaged over six different OOD test datasets described section Bold numbers are superior results.
33

---

Under review as a conference paper at ICLR 2021
FPR (5% KNR)
AUROC
FPR
AUROC
FPR
AUROC
FPR
AUROC
Poa
Method
KNR)
KNR)
ENR)
Natural OOD
Corruption OOD
Loc OOD
Comp. OOD
MSP
56.30
90.16
100000
55.70
100.00
100.00
Gaussan Noise
ODIN
58.49
88.24
100.00
44.91
100.00
0.87
100.00
0.01
Mahalanobis
0.00
99.92
97.34
77.03
1.51
98.73
100.00
0.00
SOFL
100.00
99.93
99.98
100.00
OE
0.00
100,00
0.93
99.74
0.00
100,000
98.71
2.20
ACET
0.00
100.00
0.00
99.98
0.00
99,
17.98
CCU
0.00
99.94
34.14
ROWL
0,00
99.89
3.83
97.97
0.00
99.89
91.98
53,90
NTOM (ours)
0.00
100.00
0.03
99, 94
0.00
100.00
100.00
0.72
ATOM (ours)
MSP
53.63
90.75
100.00
52.67
100.00
0.05
100.00
0.00
ODIN
59.07
87.96
100.00
39.84
100.00
0.00
100.00
0.00
Uniform
Noise
Mahalamobis
99.73
SOFL
0.00
100.00
0.35
99.74
39.53
91.58
100.00
0.01
OE
0.00
100.00
5.83
98.88
0.09
99.98
99.97
0.16
100.00
99.70
99,99
100.00
CCU
0.00
100.00
4.04
99.19
0.00
99.97
99.82
0.18
ROWL
0.00
99 89
17.44
91.17
0.19
99.80
100.00
49,89
NTOM (ours)
0.00
100.00
99.86
0.22
99.87
100.00
ATOM (ours)
0.00
100.00
0.00
99.92
0.00
100,00
99.79
74.58
Table Comparison with competitive OOD detection methods. We use SVHN as in-distribution dataset use DenseNet as network architecture for all methods. We evaluate the performance on all four types of OOD inputs: (1) natural OOD, (2) corruption attacked OOD, (3) Los attacked OOD, and (4) compositionally attacked OOD inputs. f indicates larger value is better, and 1. indicates lower value is better. All values percentages. Bold numbers are superior results.
FPR (5% FNR)
AUROC
FPR
AUROC
FPR
AUROC
(5% AUROC
FPR
(5% FNR)
FNR)
Method
Natural OOD
Corruption OOD
Loc OOD
Comp.
OOD
100.00
100.00
73.29
100.00
ODIN
97.30
89.00
100.00
81.48
100.00
71.12
100.00
34,99
Caussian
Noise
Mahalanobis
0.00
100.00
0.00
99.95
0.00
100.00
85.03
23.21
OE
0.00
99.91
0.05
98.48
0.00
99.
99.65
32.44
ACET
0.00
99.94
0.00
99. 74
0.00
99.87
0.04
99.40
CCU
0.00
98.84
98.09
97.21
84.47
69.56
ROWL
0.71
99.13
100.00
49.48
99.99
49.48
100.00
49,48
NTOM (ours)
0.00
99.97
0.08
9%,
0.00
99.87
37.97
ATOM (ours)
0,00
99.65
MSP
99.67
86.55
10000
76.54
100,00
71.48
10000
36.82
Uniform
ODIN
91.48
90.73
100.00
79.19
100.00
51.93
100.00
13.86
Mahalanobis
0.00
100.00
99.94
100.00
99.84
Noise
SOFL
0,00
99.75
95.82
86.22
100.00
69,89
100.00
4.39
OE
0.00
99.89
0.02
98.62
33.05
92.41
99 98
29.62
36.48
CCU
0.00
98.50
0.87
97.78
94.68
86.27
99.70
48.18
ROWL.
97.14
50.91
100.00
49.48
100.00
49.48
100.00
49.48
NTOM (ours)
0.00
1.42
98.08
28.41
95.94
100.00
ATOM (ours)
0,00
99.93
0.00
99.07
0.00
99.93
91.81
45,48
Table 10: Comparison with competitive
detection methods. We use CIFAR-10 as in-distribution dataset and use DenseNet as network architecture for all methods. We evaluate the performance on all four types of OOD inputs: (1) natural OOD, (2) corruption attacked OOD, (3) Los attacked OOD, and (4) compositionally attacked OOD inputs. indicates larger value is better, and & indicates lower value is better. All values percentages.
34

---

Under review as a conference paper at ICLR 2021
FPR (5% FNR)
AUROC
FPR (5% ENR)
AUROC
FPR
AUROC
EPR
AUROC
(5% FNR)
(5% FNR)
Method
Natural OOD
Corruption OOD
Lox OOD
Comp. OOD
18.62
11.90
100.00
100.00
Gaussian Noise
ODIN
10000
16.53
100.00
12.57
100.00
2.57
100.00
1.55
Mahulunobis
0.00
100.00
0.00
99.82
0.00
100.00
99. 18
81.79
SOFL
100.00
70.93
100.00
OE
3.69
97.53
100.00
64.71
67.00
92.45
100.00
1.60
ACET
0.00
99.02
22.38
95.49
0.14
97.68
99.71
75.87
CCU
ROWL
0.54
99.42
100.00
49.69
99.37
50.01
100.00
49.69
NTOM (ours)
0.00
99.95
30.60
95.57
0.00
99.92
100.00
30.10
ATOM (ours)
0.00
99.98
99.06
0.00
99.98
86.43
$2.84
MSP
99.99
37,86
100.00
16.39
100.00
2.45
100.00
1.67
Uniform
ODIN
100.00
31.77
100.00
20.56
100.00
0.48
100.00
0.40
Noise
Mahulunobis
100.00
99.69
100.00
32.70
SOFL
0.00
99.99
100.00
68.28
99.98
79.63
100.00
0.59
OE
0.19
99.21
100.00
$7.39
100.00
55.40
100.00
0.62
ACET
0.00
93.48
98.47
CCU
30,96
95.93
100.00
71.51
100.00
46.37
100.00
5.29
ROWL
98.45
50.46
100.00
49.69
100.00
49.69
100.00
49.69
NTOM (ours)
0.00
99.79
94.08
ATOM (ours)
0.00
99.98
0.04
98.44
0.00
99.98
89.90
41.34
Table 11: Comparison
competitive OOD detection methods. We use CIFAR in-distribution dataset and use DenseNet as network architecture for all methods. We evaluate the performance on all four types of OOD inputs: (1) natural OOD, (2) corruption attacked OOD, (3) Lox. attacked OOD, and (4) compositionally attacked OOD inputs. indicates larger value is better, and 1. indicates lower value better. All values percentages.
35

---

Under review as a conference paper at ICLR 2021
Diss
Method
FNR
Pred.
End-to-end.
Pred. Acc.
MSP
5.01
95.83
93.00
ODIN
5.01
95.83
92.44
Mahalanobis
5.01
95.83
91.50
SOFL
5.01
96.45
92.81
SVHN
OE
5.01
95.93
93.11
ACET
5,01
95.58
92.79
CCU
5.01
95.85
92.95
ROWL
0.22
95.23
95.23
NTOM (ours)
5.01
95.75
91.99
ATOM (ours)
5.01
96.09
91.95
MSP
5,01
94.39
91.76
ODIN
5.01
94.39
91.00
Mahalanobis
5.01
94.39
89.72
SOFL
5.01
95.11
91.60
CIFAR-10
OE
5.01
94.79
91.86
ACET
5.01
91.48
88,61
CCU
5.01
94.89
91.88
ROWL
1.04
93.18
93.18
NTOM (ours)
5.01
95.42
91.57
ATOM (ours)
5.01
95.20
91.33
MSP
5.01
75.05
73.87
ODIN
5.01
75.05
73.72
Mahalanobis
5.01
75.05
71.12
SOFL
5.01
74.37
72.62
CIFAR-100
OF
5.01
75.28
73.74
ACET
5.01
74.43
72.72
CCU
5.01
76.04
74.60
ROWL
0.62
72.51
72.51
NTOM (ours)
5.01
74.88
72.47
ATOM (ours)
5.01
75.06
72.72
Table 12:
The performance of OOD detector and classifier in-distribution test data. We use DenseNet for all methods. We three metries: FNR, Prediction Accuracy and End-to-end Prediction Accuracy. We pick the threshold OOD detectors such that 95% of in-distribution test data points are classified as in-distribution. Prediction Accuracy measures the accuracy of the classifier on in-distribution test data. End-to-end Prediction Accuracy measures the accuracy of the open world classification system (detector+classffeer where example classified correctly if and only if the detector treats in-distribution and the classifier predicts its label correctly.
36

---

Under review as a conference paper at ICLR 2021
Diss
Method
FNR
Pred.
End-to-end.
Pred. Acc.
MSP
5.01
95.61
92.91
ODIN
5.01
95.61
92.49
Mahalanobis
5.01
95.61
91.34
SOFL
5.01
95.65
92.34
SVHN
OE
5.01
95.76
92.91
ACET
5,01
95.96
93.13
CCU
5.01
95,83
93.05
ROWL
0.27
95.87
95.87
NTOM (ours)
5.01
95.76
91.98
ATOM (ours)
5.01
95.96
91.82
MSP
5,01
94.92
92.38
ODIN
5.01
94.92
91.60
Mahalanobis
5.01
94.92
90.27
SOFL
5.01
95.98
92.45
CIFAR-10
OE
5.01
95.51
92.35
ACET
5.01
95.33
92.47
CCU
5.01
95.63
92.46
ROWL
1.21
93.91
93.91
NTOM (ours)
5.01
95.89
91.97
ATOM (ours)
5.01
95.89
92.20
MSP
5.01
76.61
75.46
ODIN
5.01
76.61
75.16
Mahalanobis
5.01
76.61
72.61
SOFL
5.01
75.12
73.78
CIFAR-100
OF
5.01
75.58
74.47
ACET
5.01
75.61
74.60
CCU
5.01
76.36
75.34
ROWL
0.39
75.94
75.94
NTOM (ours)
5.01
76.30
74.04
ATOM (ours)
5.01
77.51
75.32
Table 13: The performance of OOD detector and classifier in-distribution test data. We use WideResNet for all methods. We three metrics: FNR, Prediction Accuracy and End-to-end Prediction Accuracy. We pick the threshold OOD detectors such that 95% of in-distribution test data points are classified as in-distribution. Prediction Accuracy measures the accuracy of the classifier on in-distribution test data. End-to-end Prediction Accuracy measures the accuracy of the open world classification system (detector<classfee where example classified correctly if and only if the detector treats in-distribution and the classifier predicts its label correctly.
37

---

Under review as a conference paper at ICLR 2021
FPR (5% FNR)
AUROC
AUROC
FPR (5% FNR)
AUROC
FPR (5% FNR)
AUROC
(5% FNR)
Dist
Method
Natural
Corruption
Comp.
MSP
33,90
94.01
99.59
68.76
99.94
0.82
100.00
0.11
LSUN.
ODIN
28.40
93.77
97.37
59.61
99.90
0.38
100.00
0.01
Mahalanobis
SOFL
0.00
100.00
2.10
99.38
68.89
54.15
99.98
0.31
OE
0.00
100.00
23.06
96.38
54,03
77.79
99.91
0.52
ACET
0.00
100.00
96.75
0.09
99.97
CCU
0,00
100.00
22.26
96.46
35.10
80.37
99.92
0.55
ROWL
2.50
98 64
75.48
62.15
96.22
51.78
99 99
49,90
NTOM (ours)
100.00
99.06
75.51
ATOM (ours)
0,00
100.00
16.71
96.91
0.00
100,00
97.85
39.64
MSP
44.34
92.75
99.93
66.82
100.00
1.22
100.00
0.11
62.91
LSUN-
Mahalanobis
18.35
96.64
98.92
54.27
99.04
10.01
100.00
0.00
SOFL
0.00
100.00
0.55
99.75
49.23
77.50
99.88
4.77
ACET
0.00
100.00
5.36
98.87
2.60
99.35
99.96
7.00
CCU
0.00
100.00
9.44
98.46
12.25
94.36
98.90
2.78
ROWL
0.01
99.89
22.57
88.61
39.54
80.12
NTOM (ours)
0.00
100.00
0.12
99.82
22.97
91.97
99.97
2.34
ATOM (ours)
0.00
100.00
0.13
99.67
0.09
99.97
96.08
63.20
MSP
40.78
93.57
99.85
100.00
ODIN
31.98
93.97
97.70
64.77
99.99
0.29
100.00
0.01
ISUN
Mahulunobis
19.73
96.25
98.35
56.95
98.73
10.21
100.00
0.01
0.00
OE
0.01
100.00
9.50
98.36
45.45
76.08
99.76
1.72
ACET
0.01
100.00
6.
98. 84
4,90
98.52
99 92
6.96
CCU
ROWL
0.04
99.87
24.43
87.68
44.62
77.58
99.29
50,24
NTOM (ours)
0.00
100.00
0.38
99.77
30.90
87.40
99 98
2.26
ATOM
0,00
100,00
0.24
99.59
0.25
99.94
95.20
61.70
MSP
41.91
92.08
99.33
61.77
99.45
2.84
100.00
0.44
ODIN
40.48
89.91
97.36
$2.69
99.29
2.14
100.00
0.27
Textures
Mahulunobis
24.34
94.15
62.00
17.73
99.38
SOFL
0.34
99.91
11.83
97.47
89.36
23.54
99.20
2.42
OE
2.73
99. 50
43.05
92.51
84.36
26.88
98.32
2.72
ACET
2.55
32.55
94.67
CCU
2.22
99.57
41.86
92.46
78.09
31.94
98.58
2.31
ROWL
7.85
95.96
71.17
64.30
89.73
55.02
99.77
50.01
(ours)
0.27
ATOM (ours)
0.41
99.83
11.31
97.66
28.81
90.97
91.61
49.35
MSP
36.14
94,37
99.60
71.71
99.99
T:03
100.00
0.14
Places365 Mahalanobis
34.83
93.95
98.14
$5.70
99.64
1.97
100.00
0.00
SOFL
0.00
99.99
3.46
99.09
94.58
26.96
99.96 96
1.98
26.54
ACET
0.15
99.96
19.28
97.06
49.09
83.63
99.99
2.65
CCU
0.27
99.95
29.12
95.65
83.
39.88
100.00
0.59
ROWL
0.97
$1.56
100.00
NTOM (ours)
0,00
99.99
2.39
99,21
87.35
46.68
99.99
0.48
ATOM (ours)
0.00
99.99
2.31
98.59
5.60
98.71
98.74
38.95
MSP
35.45
94.66
99.79
72.97
100.00
100.00
ODIN
24.15
95.54
95.45
70.89
100.00
0.28
100.00
0.01
Mahulunobis
39.35
92.80
98.28
55.52
99.70
2.47
100.00
0.01
10
OE
0.48
99.89
27.79
95.92
95.49
26.02
99.98
0.63
ACET
0.21
99.94
18.32
97.18
54.71
80.56
99.98
3.69
CCU
95.34
ROWL
0.84
99.47
69.28
65.25
96.68
51.55
100.00
49,89
NTOM (ours)
0.00
99.98
2.53
99.17
84.60
47.74
100.00
0.99
ATOM (ours)
0.00
99.98
98.72
7.39
98.42
98.51
44.28
Table 14: Comparison with competitive OOD detection methods. We use SVHN as in-distrihution dataset and
DenseNet network architecture for all methods. We evaluate the performance all four types of OOD inputs: (1) natural OOD, (2) corruption attacked OOD, Los attacked OOD, and compositionally attacked OOD inputs. t indicates larger value better. J. indicates lower value better. values are percentages. Bold numbers are superior results.
38

---

Under review as a conference paper at ICLR 2021
FPR (5% FNR)
AUROC
AUROC
FPR (5% FNR)
AUROC
FPR (5% FNR)
AUROC
(5% FNR)
Dist
Method
Natural
Corruption
Comp.
MSP
27.34
96.30
100.00
71.57
100.00
13.75
100.00
13,68
LSUN.
ODIN
1.89
99.50
98.94
71.86
100.00
0.06
100.00
0.00
Mahalanobis
93.31
46.23
SOFL
0.39
99.40
56.07
93.02
100.00
2.36
100.00
1.96
OE
0.97
99.52
40.85
93.99
99.99
0.36
100.00
0.19
ACET
1.76
99.42
31.46
95.40
45.54
90.57
88.09
70.84
CCU
0,62
99.65
33.44
94.56
99.97
0.31
100.00
0.05
ROWL
10.92
94.02
91.67
53.64
100.00
49.48
100.00
49.48
NTOM (ours)
0.27
100.00
ATOM (ours)
0,25
99.53
16,38
96.75
0.38
99.49
17.27
96,49
MSP
43.89
93.93
100.00
64.26
100.00
13.74
100.00
13.60
3.29
LSUN-
Mahalanobis
7.43
97.88
98.71
39.24
94.59
16.62
100.00
0.45
SOFL
1.67
99.2 99.29
56.41
90.62
100.00
0.55
100.00
0.43
OE
92.25
99
ACET
3.87
99.10
77.90
87.37
70.50
84.01
99.87
48.53
CCU
1.53
99.28
57.03
90.53
100.00
0.04
100.00
0.10
ROWL
36.03
81.47
98.72
50.12
100.00
49.48
100.00
49.48
NTOM (ours)
0,38
99.57
17.84
96.76
100.00
0.20
100.00
0.27
ATOM (ours)
0.41
99.33
16.77
96.63
52.76
71.84
50.33
75.18
MSP
93.58
100.00
62,76
100.00
ODIN
4.45
99.00
98.90
62.14
100.00
0.33
100.00
0.01
ISUN
Mahulunobis
8.58
98.00
98.
42,96
89 64
22.78
100.00
0.74
2.24
OE
1.14
99.40
48.25
92.46
99.97
0.13
100.00
0.02
ACET
6.16
98.59
75.36
87,00
78.59
79.84
99.63
47.11
CCU
ROWL
35.44
81.76
97.02
50.97
100.00
49.48
100.00
49,48
NTOM (ours)
0.63
99.59
16.25
96,90
99. 99
0.32
100.00
0.38
ATOM
0.66
99.34
15.10
96.79
52.48
68.44
51.94
72.08
MSP
64,66
87.64
100.00
$1.85
100.00
14.14
100.00
13.72
ODIN
52.45
84.81
99.56
38.20
99.95
0.56
100.00
0.07
Textures
Mahulunobis
25.39
71.42
99.27
SOFL
3.78
99.04
56.81
89.38
99.88
2.06
99.98
1.36
OE
6.24
98.43
53.32
88.83
99
1.40
99.95
0.68
ACET
11.74
90.52
77.66
94,26
CCU
5,83
98.45
54.61
86.33
99.49
1.68
99.84
0.99
ROWL
19.33
89.83 99.82
82.87
58.04
99.89
49.53
99. 98
49.49
NTOM (ours)
2.27
95.09
ATOM (ours)
L81
99.47
20.05
95.92
10.05
96.47
29.01
91.40
MSP
62.03
88.29
100.00
37.68
100.00
13.66
100.00
13.60
Places365 Mahalanobis
85.77
65.76
99.47
22.75
99.79
1.93
100.00
0.66
SOFL
7.73
97.81
61.66
88.07
100.00
0.47
100.00
0.27
87.39
ACET
18.63
95.97
79.42
85.00
93.09
66.83
99.61
46.19
CCU
8.49
97.63
67.68
85.75
100.00
0.03
100.00
0.03
ROWL
43.76
77.60
97.67
50.64
100.00
100.00
NTOM (ours)
6,63
97.94
37.01
92.75
100.00
0.06
100.00
0.04
ATOM (ours)
6.30
97.92
31.44
93.41
6.96
97.78
32.99
92.93
MSP
59.15
90.99
100.00
41.97
100.00
13.67
100.00
13.66
ODIN
23,96
95.00
100.00
19.78
100.00
0.00
100.00
0.00
SVHN
Mahulunobis
19.73
93.33
90.52
50.89
99.48
8.18
100.00
2.67
OE
1.55
99.16
75.62
89.02
100.00
0.01
100.00
0.02
ACET
31.50
94.99
83.04
85.25
94.49
69.42
99.81
$4.36
CCU
ROWL
4.73
97.12
98.09
50.43
100.00
49,48
100.00
49,48
NTOM (ours)
1.06
99.59
67.40
90.23
100.00
0.01
100.00
0.00
ATOM (ours)
0.69
99.63
51.84
92.22
0.69
99.63
51.81
92.18
Table 15: Comparison with competitive OOD detection methods. We use CIFAR-10 as in-distribation dataset and use DenseNet network architecture for all methods. We evaluate the performance on all four types of OOD imputs: (1) natural OOD, (2) corruption attacked OOD, Loe attacked OOD, and (4) compositionally attacked OOD inputs. indicates larger value is better, and J. indicates lower value better. All values are percentages. Bold numbers are superior results.
39

---

Under review as a conference paper at ICLR 2021
FPR (5% FNR)
AUROC
AUROC
FPR (5% FNR)
AUROC
FPR (5% FNR)
AUROC
(5% FNR)
Dist
Method
Natural
Corruption
Comp.
MSP
62,03
84.78
100.00
32.47
100.00
2.52
100.00
2.31
LSUN.
ODIN
15.47
97.34
100.00
42.25
100.00
0.20
100.00
0.01
Mahalunobis
47.44
98.80
58.23
SOFL
17.38
96.66
100.00
51.59
100.00
1.10
100.00
0.58
OE
14.75
97.33
99.91
54.29
100.00
1.61
100,00
0.64
ACET
14,60
98.65
67.81
95.35
98.88
$6.02
CCU
12.03
97.84
99.60
61.24
100.00
0.86
100.00
0.50
ROWL
88.67
55.35
100.00
49.69
100.00
49.69
100.00
49.69
NTOM (ours)
20.61
83.28
100.00
100.00
ATOM (ours)
21.40
96.31
79.72
87.67
21.50
96.26
79.98
87.11
MSP
77.48
76,40
100.00
32.21
100.00
194
100.00
1.78
34.81
LSUN-
Mahalanobis
14.87
97.06
99.89
31.14
94.81
24.18
100.00
0.37
SOFL
50.27
90.28
99.88
50.12
100.00
0.11
100.00
0.20
OE
56.25
41.17
ACET
56.35
88.17
99.50
51.85
98.67
18.64
99.74
22.45
CCU
38.44
91.83
99. 94
50.62
100.00
0.59
100.00
0.47
ROWL
88.25
55.57
49.69
100.00
100.00
NTOM (ours)
37.92
94.31
99.11
65, $5,00
100.00
0.36
100.00
0.06
ATOM (ours)
17.93
96.94
95.72
71.51
31.39
87.38
95.87
66.15
MSP
78.87
100.00
31.83
100.00
2.08
100.00
ODIN
38.92
92.15
100.00
43.43
100.00
0.31
100.00
0.00
ISUN
Mahulunobis
16.46
96.75
99.76
33,38
89.71
28.16
100.00
0.38
53.51
48.75
OE
61.59
81.51
99.99
39.94
100.00
0.81
100.00
0.54
ACET
60.49
86,80
99.70
49.61
98.54
19.92
99.95
23.51
CCU
99.
ROWL
90.42
$4.48
100.00
49.69
100.00
49.69
100.00
49.69
NTOM (ours)
43.66
93.24
99.3
61.40
100.00
0.41
100.00
0.02
ATOM
20.09
96.62
97.05
68.31
35.70
85.87
97.13
63.16
MSP
85.57
70.08
100.00
25.93
100.00
2.63
100.00
2.28
ODIN
83.58
70.71
100.00
27.30
100.00
0.11
100.00
0.01
Textures
Mahulunobis
34.59
89.85
78.35
62.36
89,27
20.33
SOFL
57.00
87.35
99.73
44.11
99.98
0.59
100.00
0.34
OE
59.86
86.17
99.88
42,95
99 99.98
1.45
100.00
0.73
ACET
62.02
86.26
63.61
CCU
60.80
86.34
99.88
44,90
100.00
1.39
100.00
0.62
ROWL
97.00
51.19
100.00
49.69
100.00
49.69
100.00
49.69
NTOM (ours)
38.21
ATOM (ours)
40.11
90.28
90,21
69.99
47.27
82,52
91.45
65.91
MSP
83.65
73.71
100.00
32.13
100.00
L.83
100.00
1.91
79.19
Places365 Mahalanobis
94.64
59.52
99.80
19.43
99.96
1.13
100.00
0.50
SOFL
60.49
87.57
100.00
40.25
100.00
0.06
100.00
0.15
50.83
ACET
56.26
86.75
99.66
$7.36
83.60
72.35
99.83
44.27
CCU
55.23
87.21
100.00
44.03
100.00
0.46
100.00
0.43
ROWL
96.86
$1.26
49.69
100.00
100.00
NTOM (ours)
56.54
84,27
99.33
57.32
100.00
0.01
100.00
0.02
ATOM (ours)
56.52
84.53
96.65
63.43
$8.68
82,50
96.68
62.36
MSP
80.71
76.00
100.00
100.00
100.00
ODIN
88.66
71.65
100.00
24.17
100.00
0.00
100.00
0.00
SVHN
Mahulunobis
47.78
90.54
98.93
53.25
99.
6.34
100.00
2.78
OE
44.47
92.58
100.00
40.86
100.00
0.53
100.00
0.57
ACET
55.86
90.36
100.00
49,51
70.55
86.86
99.99
44.93
CCU
40.01
ROWL 98.88
50.25
100.00
49.69
100.00
49.69
100.00
49,69
NTOM (ours)
24.67
96.20
99.83
60.37
100.00
0.01
100.00
0.00
ATOM (ours)
37.78
93.68
99.54
70.86
37.78
93.68
99.55
70.23
Table 16: Comparison with competitive OOD detection methods. We use CIFAR- 100 as in-distribution dataset and use DenseNet network architecture for all methods. We evaluate the performance on all four types of OOD imputs: (1) natural OOD, (2) corruption attacked OOD, Lie attacked OOD, and (4) compositionally attacked OOD inputs. indicates larger value is better, and 4. indicates lower value better. All values are percentages. Bold numbers are superior results.
40


Reviews:
[{"review_id": "p72j6tzOzfQ", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper attempts to formulate the task of robust OOD detection by expanding the usual definition of OODs, and then presents a theoretical analysis in the simple setting of a Gaussian data model by porting results from adversarial defense. An experimental comparison is also presented. \n\nPro:\n\n+ The paper addresses a very important problem which is growing in importance. A number of approaches have been recently proposed to address this problem that attempt statistical, explanation-based, and relational/semantic techniques. The paper takes a simple approach of class augmentation and then picking hard OODs to solve this problem. \n\nCons:\n\n- Why is it surprising that the \"majority of auxiliary OOD examples may not provide useful information to improve the decision boundary of OOD detector\"? This seems to be rather obvious both for the toy case of Gaussian model or in general. It is well known that hard examples help train. The triplet loss function and other methods have been used for years based on this insight. The paper appears to present this obvious observation as the central finding of the paper.  For e.g. in the context of embedding, see how finding hard examples has been found to be useful in https://openaccess.thecvf.com/content_ICCV_2017/papers/Yuan_Hard-Aware_Deeply_Cascaded_ICCV_2017_paper.pdf \n\n- The paper appears to be mixing a lot of different problems to formulate a very generic notion of OOD. Typically, OODs have been used for out of distribution datasets (SVNH for CIFAR10), new classes on which model was not trained (leave some classes out of CIFAR10), or transforms such as rotations or other transformations of inputs. The paper mixes adversarial examples to it and then considers even further compositions. While there is significant effort to break OOD by building a useful taxonomy of it to refine the sources of uncertainty and address these in a principled way, the paper creates a monolithic problem of detecting when inputs don't belong to the training distribution for any reason. This creates a rather ill-defined mathematical problem because modeling the exact distribution of the training data is quite infeasible. The theoretical treatment of the experiments in the paper fail to convince the reviewer that there is an advantage in merging these separate problems into a single problem.\n\n- The theoretical treatment are directly borrowed from http://papers.nips.cc/paper/9298-unlabeled-data-improves-adversarial-robustness.pdf where the treatment was for adversarial examples - since, the paper's definition of OOD includes these, the previous results are directly applicable. It would help to make this obvious in the main text instead of having this reference in the appendix. The connection is very strong. Also, this reviewer currently does not see the challenge in lifting the result from the reference to the theoretical analysis in this paper (please see question below). \n\n- The experimental evaluation is limited and does not meet the usual set of experiments reported for OOD detection which makes it difficult to understand the empirical benefit of the presented approach. \n\n- While there are other methods that also use \"exposure to outlier\" in OOD detection, it is debatable such an approach would be useful in practice, particularly, if one also included adversarial examples. But given past literature on this topic, the reviewer is less concerned about this aspect. \n\nIn summary, the idea of using hard examples is not new. The definition of OOD to include adversarial examples is not well motivated. The theoretical results are borrowed without significant extension from literature on adversarial defenses. The experimental results do not meet the expectation of a paper on this topic to judge its empirical value against existing approaches. \n\nQuestions for the author:\n\n- Why is it helpful to merge different problems of adversarial example detection, detection of data away from training distribution due to new classes, detection of corrupted inputs into a single problem? Does a uniform treatment yield some new insight or enable some particular approach? Does it beat the state of the art in any of these sub-problems? \n\n- Can you elaborate on the nature of \"natural OODs\" in the experiments so that it is easier to understand comparison with ODIN, Mahalanobis, etc methods? They perform differently when OODs are because of new datasets or because of held-out classes. This \"natural OOD\" is better split into at least these two different types before evaluating the methods. \n\n- How is the theoretical analysis presented here not a trivial application (from adversarial setting to a superset definition of OOD) of results in http://papers.nips.cc/paper/9298-unlabeled-data-improves-adversarial-robustness.pdf  and https://arxiv.org/pdf/1804.11285.pdf ? Could you help identify the challenge and the novelty in this extension? \n\nThe paper currently appears to be building on the premise that if we think of adversarial examples as a subclass of OODs, theoretical results from adversarial examples are applicable to OODs. A simple supervised approach using hard examples is then presented to solve this problem. Its empirical evaluation appears to behind the state-of-the-art in OOD detection as well as adversarial defense, but mixing both does not make this any stronger a paper. In its current form, the paper appears to need better motivation and identification of novelty. \n\nAfter discussion with authors\n------------------------------------------\n\nSome of the concerns of the reviewer have been addressed and the reviewer is raising the score to reflect it. The paper still has some major concerns preventing the reviewer from recommending acceptance of the paper: \n\n- Similarity of the theoretical analysis to https://proceedings.neurips.cc/paper/2019/file/32e0bd1497aa43e02a42f47d9d6515ad-Paper.pdf  and https://arxiv.org/pdf/1804.11285.pdf .  \n\n- Hard negative mining is pretty standard in many learning domains. \n\nBoth of these issues can be addressed by a better review of the related work and more accurate identification of the novelty in the paper. \n\nGiven the limited theoretical novelty, a more robust empirical evaluation establishing the value of the extreme outliers against state of the art approaches would have been useful. Also, if the authors want to investigate theoretical results on OODs, one challenge is the lack of a formal definition of OODs. \n\nThe paper is a good work in progress but not yet ready for publication. The reviewer is hopeful that the above suggestions will make the paper stronger.  ", "extra_scores": {}, "extra_text": {}}, {"review_id": "lMWbl8YAzI0", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. The paper presents a lot of theory but insufficient evidence. It only employs limited image data (SVHN, CIFAR variants). The paper should be clear that the scope is limited to well-known image datasets only. This is because the approach is dependent on auxiliary data which is available for the image datasets. It is not clear if the approach might be more generally applicable to (say) network traffic, credit card transactions, natural language, etc. It would be better to include other types of data and along with auxiliary data generated through more generic means.\n\n2. The presentation of the theory in Section 3 is not proper and makes it too simplistic. A better presentation should first present the general result first (rather than leave it to Appendix) and then show how that applies to a specific case such as Gaussian distribution.\n\n3. Propositions 1, 2, 3: There seems to be a disconnect: what is the significance of n_0 in Proposition 1? What is its connection to the algorithm?\n\nIf we try to do a ballpark estimate with reasonable range of values: n_0 ~ 100, d ~ 50, \\epsilon = 1/2, then the statement basically says that if n < 2 (optimistically), then we will have large errors. This type of limit on n is not very useful in practice as it is too loose. In fact, it seems that Proposition 1 would only be useful if d is very large. The paper should clarify that.\n\nBy and large, the Propositions 1, 2 and 3 are not helping much as the messages they are trying to convey (that auxiliary data helps and large number of samples are more accurate) are already well known. They do not add new knowledge. These might be moved to appendix.\n\n4. Appendix B2, the general error bound (Proposition 5): The assumption is that d(Q_x, U_x) is small. The proposition basically states that if Q_x is similar to U_x, then a small FPR on U_x will also have a small FPR on Q_x. This is rather trivial. It is *much* harder in practice to get a U_x that is similar to Q_x. A more interesting case would be to show that even if U_x was not very similar to Q_x, then is the FPR within reasonable limit?", "extra_scores": {}, "extra_text": {}}, {"review_id": "8xJIn6Nc-dn", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper provides theoretical analysis formalizing the intuition of mining hard outliers for improving the robustness of OOD detection. Motivated by the theoretical analysis, it proposes Adversarial Training with informative Outlier Mining (ATOM) and achieves SOTA results on both clean and perturbed OOD inputs with extensive experiments compared to a bunch of baselines. It also provides an ablation study on the effectiveness of adversarial training, which further proves its effectiveness. I think it's a good paper overall and should be accepted.", "extra_scores": {}, "extra_text": {}}, {"review_id": "81WVNyiYyXt", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper the authors propose a method for training a classifier to be more effective at OOD (out of distribution) detection. Many OOD detection methods work by utilizing an auxiliary dataset as examples of OOD-ness. This is the approach taken in this paper and OOD is trained as being a k+1 classification class.  When training the OOD class the proposed method allows for adversarial perturbation of the OOD examples to help improve training. This is a pretty common technique in deep learning, see for example \"Deep Robust One Class Classification.\"  Finally the main novelty of the method proposed by the authors is to sort a collection of OOD examples and sort according to the OOD score of the current model and use the \"qNth\" to be presented as OOD examples during the next epoch during training. The authors term this \"Informative Outlier Mining.\" The authors demonstrate that this method works well experimentally.\n\nThe authors present some theoretical justification for their approach via a very toyish analysis of a OOD detection with a Gaussian data model.\n\nWhile the proposed method seems to work reasonably well I find the novelty of this method to be too low to be interesting being more of a small trick rather than deserving of being the topic of an entire paper. I do think, however that this idea, in combination with some other OOD tricks could be an interesting paper.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors studied a robust method for detecting out-of-distribution (OOD) instances. OOD instance detection is an important practical problem, and multiple reviewers recognized the proposed approach is interesting. However, it was the common opinion of several reviewers that the main theoretical analysis was imported from existing studies, and the novelty is not sufficiently high. It was also observed that the relationship between the proposed method and closely related studies was not properly discussed. Although this point has been improved in the revision, a reviewer and area chair still concern that enough evidence is not provided for some of the points the authors claim as advantages over existing studies. Although the proposed method is interesting and could be an important contribution to the ICLR community, the current paper needs non-trivial revision before publication.
Decision: Reject