Provably Adversarially Robust Detection of Out-of-Distribution Data (Almost) for Free
Authors: Alexander Meinke
Julian Bitterwolf
Matthias Hein
Venue: NeurIPS 2022 Accept (2022)
Keywords: adversarial robustness
out-of-distribution detection

Abstract: The application of machine learning in safety-critical systems requires a reliable assessment of uncertainty.
However, deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data.
Even if trained to be non-confident on OOD data, one can still adversarially manipulate OOD data so that the classifier again assigns high confidence to the manipulated samples.
We show that two previously published defenses can be broken by better adapted attacks, highlighting the importance of robustness guarantees around OOD data.
Since the existing method for this task is hard to train and significantly limits accuracy, we construct a classifier that can simultaneously achieve provably adversarially robust OOD detection and high clean accuracy.
Moreover, by slightly modifying the classifier's architecture our method provably avoids the asymptotic overconfidence problem of standard neural networks.
We provide code for all our experiments.

Provably Adversarially Robust Detection of Out-of-Distribution Data (Almost) for Free
Alexander Meinke"
Julian Bitterwolf University of Tübingen
Matthias Hein University of Tübingen Tübingen Al Center
University of Tübingen Tübingen AI Center
Tübingen AI Center
Abstract
The application of machine learning in safety-critical systems requires reliable assessment of uncertainty. However, deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. Even trained to be non-confident on OOD data, so that the classifier again assigns high confidence to the manipulated samples. We show that previously published defenses can be broken by better adapted attacks, highlighting the importance robustness guarantees around OOD data. Since the existing method for this task is hard train and significantly limits accuracy, we construct a classifier that can simultaneously achieve provably ad- versarially robust OOD detection and high clean accuracy. Morcover, by slightly modifying the classifier's 's architecture our method provably avoids the asymptotic overconfidence problem of standard neural networks. We provide code for all our experiments
still adversarially manipulate OOD data
Introduction
Deep neural networks have achieved state-of-the-art performance in many application domains. However, the widespread usage of deep neural networks in safety-critical applications, c.g. in healthcare, autonomous driving/aviation, manufacturing, raises concerns as deep neural networks have problematic deficiencies. Among these deficiencies, overconfident predictions non-task related imputs 44 21 have recently attracted lot of interest. Even theoretically derived weaknesses like ReLU networks provably being overconfident far away from the training data (20 are yet to be fixed. Meanwhile, reliable confidences the classifier on the classification task (in-distribution) [19 as well as on the out-distribution [21] 20) are important to be able to detect when the deep neural network is working outside of its specification, which can then be used to either involve human operator or to fall back into a "safe state". Thus, solving this problem is of high importance for trustworthy ML systems. Crucially, detection method needs to generalize to novel test out- distributions that are not available during training, since one does not know which unknown inputs can be expected.
Many approaches have been proposed for OOD detection, 21 31 32 22 48 20 41 46 35 36].
In this paper we focus on confidence based OOD detection, i.e. the probability of the predicted class used to decide whether to reject or accept the sample, because of its straightforward interpretation been shown perform no worse than other scores for OOD detection [7]. One of
and because the currently best performing methods enforces low confidence during training ("outlier exposure" (OE) on a large and diverse set out-distribution images [22) which leads to strong separation of in- and out-distribution based the confidence of the classifier.
"Coresponding author www.:aawnnee.iiiieeeee--eeeeeeee httpssss//aaaiu...o.oo//eeeeeeee com www.////wwweeeiie////aaaaee----e Detection
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Table 1: ProoD combines desirable properties of existing (adversarially robust) OOD detection methods. It has high test accuracy and standard OOD detection performance (as (22)) and has worst- case guarantees if the out-distribution samples adversarially perturbed in an 1o nneihhoorhood to
maximize the confidence (see Section 4.2). Similar to CCU [41] it avoids the problem of asymptotic
overconfidence far away from the training data.
OE [2
CCU 41
ACET/ATOM 20 9
GOOD
g
ProoD
High accuracy
High clean OOD detection performance Adv. OOD ls robusttesss
(
Adv. OOD certifficass
Provably not asympt. overconfident
A remaining robustness problem of standard OOD detection methods is that they vulnerable to adversarial perturbations, i.e. small modifications of OOD inputs can lead to large confidence of the classifier on the manipulated samples [44 20 [50]. Of course, OOD input, which by definition is semantically far away from the in-distribution, should not be able to be moved into a region that is considered in-distribution by the detection model if the movement imperceptibly small. On the other hand, slightly perturbed in-distribution input can still be considered in-distribution for some perturbations (e.g. if the perturbation resembles standard camera noise), but for other perturbations it might be highly atypical and therefore should arguably rather be seen as OOD. Furthermore, adversarial robustness on the in-distribution known to come the cost of clean accuracy [53] which hinders the adoption of such methods in practice. We aim to provide method that does not harm the in-distribution performance in any way and thus, like previous DOD-detection methods, we do not consider adversarially manipulated in-distribution samples and focus on ensuring that OOD samples remain OOD under adversarial attacks.
While different methods for adversarially robust OOD detection have been proposed
20 50
For the standard empirical evaluation of adversarial robustness, for each input one runs an array of different attacks that conform the assumed threat model and records the output for the worst found perturbation. This means that there is no guarantee that a more malign perturbation does not exist, as only a lower bound on the adversarial robustness established. Provable adversarial robustness on OOD data, which we provide in this paper, yields a mathematically deduced upper bound on the worst-case confidence around each OOD sample. For our guaranteed upper bounds on the confidence of an OOD sample, it is certain that no applicable manipulation raises the confidence above the certified value.
there is little work on provably adversarially robust OOD detection 41 25
In (25 they apply randomized smoothing obtain guarantees wrt. la-perturbations for Dirchlet- based models (39, 40 (51) which already show quite some gap in terms of AUC-ROC to SOTA
OOD detection methods even without attacks. Interval bound propagation (IBP) [17 42 56 23
has been shown to be one of the most effective techniques in certified adversarial robustness on the in-distribution when applied during training. In GOOD [8] they use IBP to compute upper bounds on the confidence in an -eighhbooooo of the input and minimize these upper bounds on a training mitdiistriutton. This yields classifiers with pointwise guarantees for adversarially robust OOD detection even for "close" out-distribution inputs which generalize to novel OOD test distributions. However, the employed architectures of the neural network are restricted to rather shallow networks as otherwise the bounds of IBP are loose. Thus, they obtain low classification accuracy which is far from the state-of-the-an e.g. 91 % on CIFAR 10, and their approach does not scale to more complex tasks like ImageNet. In particular, despite its low accuracy the employed network architecture is quite large and has higher memory consumption than a ResNet50. The authors of (5) use SOTA verification techniques [15) and get guarantees for OOD detection wrt. lo0 pertuurbationn for ACET models 20 that were not specifically trained to be verifiable but the guarantees obtained by training the models via IBP in g are significantly better.
different type of guaranteed low confidence on OOD data pertains to the asymptotic behavior of classifier. Since standard ReLU networks provably have increasing confidence in almost all directions far away from the training data (20). one has to modify the architecture in order to solve this problem. In CCU 41 the authors append density estimators based on Gaussian mixture models for in- and out-distribution to the softmax layer. By also enforcing low confidence on a training out-distribution, they achieve similar OOD detection performance to [22] but can guarantee that the classifier shows

---

High confidence
Classilier
p("dog" ()), 99.9%
In-Distribution R.mggNer
dood 98.4%
Certilie
Discrininator
p(faa) 98. 9%
Guaranteed low confidence adversarial
Worst- Case Dut-Distributio Not R. R.tmgNer
Be
max
maac ply
35.47
Certilied Discriminetor vs. Out
p(3l2 ) 49.8%
Figure 1: ProoD's Architecture: Combining the output of a classifier and a certified discriminator, Eq. (2), we achieve high confidence the in-distribution sample of dog (R.ImgNet). The certified discriminator, see Eq. (4). yields an upper bound on the confidence in a neighhooooo of the shown OOD sample not belonging to any classes of I.ImgNet. ProoD achieves provable guarantees on adversarial OOD detection without loss in accuracy or clean OOD detection.
decreasing confidence as one moves away from the training data. However, for close in-distribution inputs this approach yields no guarantee the Gaussian mixture models powerful enough for complex image classification tasks. In [27 28 similar asymptotic guarantees are derived for Bayesian neural networks but without any robustness guarantees.
In this paper we propose ProoD which merges a certified binary discriminator for in-versus out- distribution with classifier for the in-distribution task in principled fashion into joint classifier. This combines the advantages CCU [41] and GOOD (8] without suffering from their downsides. In particular, ProoD simultancously achieves the following:
Guaranteed adversarially robust OOD detection via certified upper bounds on the confidence in loo -balls around OOD samples.
Additionally, provably prevents the asymptotic overconfidence deep neural networks.
It can be used with arbitrary architectures and has no loss in prediction performance and standard OOD detection performance.
Thus, we get provable guarantees for adversarially robust OOD detection, fix the asymptotic over- confidence (almost) for free we have (almost) no loss in prediction and standard OOD detection performance. We qualitatively compare the properties of our model to prior approaches in Table
2 Provably Robust Detection of Out-of-distribution Data
In the following we consider feedforward networks for classification, f .dd + R with K classes defined with input (0) = x and layers = 1, as
x = =0() (w(0 +b²)
f(x) = (44) +(4)
(1)
where W(() and bit) are weights and biases and ( is either the ReLU or leaky ReLU activation
function of layer 1. We refer to the output of f as the logits and get a probability distribution over
the classes via p(ylz) E elute
efa(x)
for 1..... ......K. K. We define the confidence as Conf(f(x))
maxy=1,... p(y)2)).
2.1 Joint Model for OOD Detection and Classification
In our joint model we assume that there exists an in- and out-distribution where the out-distribution samples are unrelated to the in-distribution task. Thus, we can formally write the conditional distribution the input
p(y[x) plylæ, ()p(i(z) + o((ozz)
(2)
3

---

where p()[z)) is the conditional distribution that sample x belongs to the in on-distribution and p(y) 2 i) the conditional distribution for the in-distribution. We assume that OOD samples are unrelated and thus maximally in-informative to the in-distribution task, i.e. we fix p(y|z, K. so that the classifier can be written as
1 [1 ((1-p(i)))) K
pulzz - P(y |z ()](()))) +
(3)
We train the binary classifier p(ia.) in certified robust fashion wrt. an -threat model so that even adversarially manipulated OOD samples are detected. In order to avoid confusion with the multi-class
classifier, we will refer to p(ilz)) as a binary discriminator. In an -bal of radius around E R and for all we get the upper bound on the confidence of the final classifier in Eq. 33)
K-1 K
max plylz') ||x aer-rllas
max p(1(=') + K (1 p(ilz') lar |a'-
max -e
p(ila)) + K
(4)
where we have used that p(y) < 1Vx, y, so we can defer the certification "work" to the binary discriminator. Using particular constraint on the weights of the binary discriminator, we get similar asymptotic properties as in [1 but additionally get certified adversarial robustness for close out-distriuution samples as in
contrast (8, this comes without loss in test accuracy or non-adversarial OOD detection performance since in our model the neural network used for the in-distribution classification task p(yl i) is independent of the binary discriminator. Thus, we have the advantage that the classifier can use arbitrary deep neural networks and ís not constrained to certifiable networks. We call our approach Provable out-of-Distribuue visualize its components in Figure ! The intuitive idea of why ProoD can achieve adversarially robust OOD detection without loss in clean OOD detection can be explained with the behavior the predicted probability distribution provided in Equation (3)
detector (ProoD) and
the classifier p(y|.; i) (trained similar to Outlier Exposure) already en- forces low confidence on out-of-distribution points and thus irrespective of the values p()) the resulting output of (zy) will be close to uniform as well and thus ProoD performs similar to Outlier Exposure.
For clean OOD:
For adversarial OOD: the classifier confidence maxy P(y] i) is potentially corrupted but now the binary discriminator p(ilz)) kicks in and ensures that the resulting prediction p(yl is close to uniform.
This explains why the combination of certified discriminator and classifier works much better than the individual parts and the use of this "redundancy" is the key idea of ProoD.
Certifiably Robust Binary Discrimination of In- versus Out-Distribution The first goal is to get a certifiably adversarially robust OOD detector pi)] We train this binary discriminator independently of the overall classifier as the training schedules for certified robustness are incompatible with the standard training schedules of normal classifiers For this binary classification problem use logistic model p( (z) = I+e where Rd > IR are logits of a neural network (we denote the weights and biases of g by W, and by in order to differentiate it from the classifier f introduced in the next paragraph). Let (ær, yr )r be our in-distribution training data (we use the class encoding +1 for the in-distribution and -1 for the out-distribution) and (z.)az be our training out-distribution data. Then the optimization problem associated to the binary classification problem becomes:
min w(4
log ( e -loa)))
1 M
Σ
log + eB(ze),
where we minimize over the parameters of the neural network I under the constraint that the weights of the output layer W((4)) componentwise negative and g(2) max,een,(s.)) g(u) is an upper bound on the output of around OOD samples for given 4p-threat model Bp(z,e) e) = {u E [0, 1] ||u - =Ilp < zj.. In this paper we always use an -trraa model. This upper bound could, in principle, be computed using any certification technique but we will use interval bound propagation (IBP) since simple, fast and has been shown to produce SOTA results [17] Note that this is not standard adversarial training for binary classification problem as here we have

---

an asymmetric situation: we want to be (certifiably) robust to adversarial manipulation on the out-disribbution data but not on the in-distribution and thus the upper bound is only used for out- distribution samples. The negativity of the output layer's weights W is enforced by using the parameterization (W/" = e componentwise and optimizing over hy. In Section we show how the negativity of W: allows to control the asymptotic behavior of the joint classifier.
For the reader's s convenience we quickly present the upper x(() and lower x0() bounds on the output of layer in a feedforward neural network produced by IBP:
F = (w (1-1)) + ( x (1-1) + +b0), z
(0 + w + b(0)). (6)
where W+ ma W) and W- = min(0. W) (min/max used componentwise). For an S0 threat model one starts with the upper and lower bounds for the input layer xz(0) x + e and _0()) = x
and then iteratively computes the layerwise upper and lower bounds (0()), (0) which fulfill
x00
min
(x')
max x: |
(x') S z(()
|| lar/=zll.e
While in S they also used IBP to upper bound the confidence of the classifier this resulted in a bound that took into account all O(K2) logit differences between all classes. In contrast, our loss in Eq. (5) is significantly simpler as we just have a binary classification problem and therefore only need a single bound. Thus, our approach easily scales to tasks with a large number of classes and training the binary discriminator with IBP turns out to be significantly more stable than the approach in [8]
(Semi)-Joint Training of the Final Classifier Given the certifiably robust model p(i]x)) for the binary classification task between in- and out-distribution, we need to determine the final predictive distribution p(y|z) in Eq. (2) On top of the provable OOD performance that we get from Eq. 4. we also want to achieve SOTA performance on unperturbed OOD data. In principle we could independently train model for the predictive in-distribution task p(yla: i), e.g. using outlier exposure (OE) [22] or any other state-of-the-ar OOD detection method and simply combine it with our ((llz)) While this does lead models with high OOD performance that also have guarantees, completely ignores the interaction between p(ilæ:) and p(y x., i) during training. Instead we propose to train p(y| i) by optimizing our final predictive distribution p(ylær). Note that in order to retain the guarantees of p(ilz)) we only train the parameters of the neural network f Rd > EK and need to keep (i( resp- fixed. Because g stays fixed we call this semi-joint training. We use OE [22 for training p(y(z)) with the cross-entropy loss and use the funnttin in order to obtain the fvle)
predictive distribution by (yl
Σ. Ext from f:
N
M
K
1
min
log (p(u)lz.))
log (p((lz)))
M
K
8=1
1=1
1
min
log (y i)p((xx,)+++ マ (1 p(ilz.))
N
M log (0]1)) + K (1 (1- ((llza)
(8)
M
where the first term is the standard cross-entropy loss on the in-distribution but now for our joint model for p(y) z) and the second term enforces uniform confidence on disttriiibution samples. In App. B we show that semi-joint training leads to stronger guarantees than separate training.
The loss in Eq. (5) implicitly weighs the in in-distribution and worst-case out-distribution equally, which
amounts to the assumption p(t) = 2 = p(o). This highly conservative choice simplifies training the binary discriminator but may not reflect the expected frequency of OOD samples at test time and in effect means that $ (z) tends to be quite low. This typically yields good guaranteed AUCs but can have a negative impact on the standard out-distribution performance. In order better explore the trade-off of guaranteed and standard OOD detection, we repeat the above semi-joint training with different shifts of the offset parameter in the output layer
b' (L,)) A.
(9)
5

---

where A 2 0 leads to increasing p(i(z). This shift has direct interpretation in terms of the probabilities p(i) and p(o). Under the assumption that our binary discriminator g perfect, that is
p(z())(())
p(ilx)) p(z)))((() + p(z(o)p(o)
(10)
1+e-9(x)
p(alolp(o)' A change of the prior probabilities p(i) and p(o) without changing p(all)) and p(rlo)) then corresponds a novel classifier
then it holds that e p(ah)p(()
e(a) p(2)))p(i) p(z))p(1) p(o)((t)
(11)
p(z(o)p(o) p(z(o)((o) p())(())
with log
plo)p(s) p(oplo)
Note that p(i) (ii) corresponds to positive shifts. In practice, this
parameter can be chosen based on the priors for a particular application. Since such priors are available in our case we determine a suitable shift by evaluating on the training out-distribution (see Section 4.2) Note that we explicitly do not train the shift parameter since this way the guarantees would get lost as the classifier implicitly learns a large A in order to maximize the confidence on the in-distribution, thus converging to normal outlier exposure-type classifier without any guarantees.
Guarantees on Asymptotic Confidence
In this section we show that our specific construction provably avoids the issue of asymptotic overconfidence that was pointed out in (20] Note that the resulting guarantee (as stated in Theorem different from and in addition to the robustness guarantees discussed the previous section (see Eq. The previous section dealt with providing confidence upper bounds on neighborhoods around OOD samples whereas this section deals with ensuring that a classifier's confidence decreases asymptotically one moves away from all training data.
We note that a ReLU neural network f Rd + RK as defined Eq. using ReLU or leaky ReLU as activation functions, potential max-or average pooling and skip connection yields piece-wise affine function 22 i.e. there exists finite set of polytopes Qr C IRd with r -I..... R such that UR Rd and f restricted to each of the polytopes is an affine function. Since there are only finitely many polytopes some of them have to extend to infinity and on these ones the neural network is essentially an affine classifier. This fact has been used in (20 to show that ReLU networks are almost always asymptotically overconfident in the sense that if one moves to infinity the confidence of the classifier approaches 1 (instead of converging to the desirable K as in these regions the classifier has never seen any data). The following theorem now shows that, in contrast to standard ReLU networks, moves away from the training data which a desirable property of any reasonable classifier.
proposed joint classifier gets provably less confident in its decisions one
Theorem 1. Let x E TRd with x + O and let RRd + R be the Rel.U-network of the binary discriminator (with the last activation being a non-leaky ReLU)). Denote by {Qr) R the finite set of polytopes on which g is affine (exists by Lemmal 7 in App. C Denote by Qi the polytope such that Bx E Qu for all B and let x! (((--))(2) U2 d with U E ERL-1X0 1xd output of the pre-logit layer of g gor E Q If + 0, then limg-soo fly(3x) K
and E R be the
The proof App. C In App. A we show that the condition Ux + is not restrictive, as this property holds in all cases where we checked it. The negativity condition on the weights w of the output layer of the in-vs. out-distribution discriminator g is crucial for the proof. This may seem restrictive, but we did not encounter any negative influence of this constraint on test accuracy, guaranteed or standard OOD detection performance. Thus, the asymptotic guarantees come essentially for free.
4 Experiments
4.1 Training of ProoD
We provide experiments on CIFAR10, CIFARI00 (29 and Restricted Imagenet ImgNet) [53
The latter consists of ImageNet images (ILSVRC2012) [16 49 belonging to 9 types of animals.

---

Training the Binary Diseriminator We train the binary discriminator between in-and out- distribution using the loss in Eq. ( with the bounds over an -e-bal of radius e 0.01 for the out-distribution following [ We use relatively shallow CNNs with only 5 layers plus pooling layers, see App. D For the training out-distribution, we could follow previous work and use 80M Tiny Images [52] for CIFAR
However, there have been concerns over the use of this dataset [6] due to offensive class labels. Although we do not use any of the class labels, we choose to use OpenImages (30) as training OOD instead. In order to ensure a fair comparison with prior work we also present results that were obtained using 80M Tiny Images in App. D For ImmgNett we use the ILSVRC2012 train images that do not belong to R. ImgNeet as training out- distribution INotR.ImgNer
Semi-Joint Training For the classifier we use ResNet 18 architecture on CIFAR and a ResNer50 on ImggNet. Note that the architecture of our binary discriminator is over an order of magnitude smaller than the one in ( (1IMB instead of 135MB) and thus the memory overhead for the binary discriminator is less than a third of that of the classifier. All schedules, hardware and hyperparameters are described in App. D As discussed in Section 1 when training the binary discriminator one implicitly assumes that in- and (worst-case) out- aut-distribution samples are equally likely. It seems very unlikely that one would be presented with such a large number of OOD samples in practice but as discussed Section 2.1 we can adjust the weight of the losses after training
the discriminator (but before training the classifier) by shifting the bias bg ((4g) in the output layer of the binary discriminator. We train several ProoD models for binary shifts in {0, 1, 3, 4, 5, 6] and then evaluate the AUC and guaranteed AUC (see 4.2) on subset of the training out-distribution Openlmages (resp. NotR. ImgNet.. For all bias shifts we use the same fixed provably trained binary discriminator and only train the classifier part. As our goal is to have provable guarantees with minimal or no loss on the standard OOD detection task, among all solutions which have better AUC than outlier exposure (OE) 22 we choose the one with the highest guaranteed AUC on OpenImages (on CIFAR RO/CIIIAAIIIIIII 100) resp. NotR. ImgNet (on I.mgNeet). If none of the solutions has better AUC than OE on the training out-distribution take the one with the highest AUC. We show the trade-off curves for the example in App. D
4.2 Evaluation
Setup For OOD evaluation for CIFARIO/100 we use the test sets from CIFAR100/10, SVHN 43
the classroom category of downscaled LSUN (55] (LSUN CR) as well as smooth noise as suggested
in [20 and described in App. D For R. Imggett we use Flowers 455, FGVC Aireraft [38, Stanford Cars 26 and smooth noise as test out-distriuttions. Since the computation of adversarial AUCs (next paragraph) requires computationally expensive adversarial attacks, we restrict the evaluation on the out-distribution to a fixed subset of 1000 images (300 in the case of L.SUN CR) for the CIFAR experiments and 400 for the R. ImgNet models. We still use the entire test set for the in-distribution. We also show the results additional test out-distributions in App. G
Guaranteed and Adversarial AUC We use the confidence of the classifier as the feature to discriminate between in- and out-distriuution samples. While in standard OOD detection one uses the area under the receiver-operator characteristic (AUC) to measure discrimination of in- from out-distribution, several prior works also study the worst-case AUC (WCAUC) [41 4 is defined as the minimal AUC one can achieve if each out-distribution sample is allowed to be perturbed to reach maximal confidence within certain threat model, which in our case is an 4o0 ball of radius c. Note that an alternative formulation of worst-case AUC as the worst-case across all samples from the distribbibbutonn would turn out to be uninteresting, since would necessarily be close to zero even if only single sample gets assigned high-confidence, so we do not consider this notion here. Formally, the AUC and WCAUC of feature h R2 > IR are defined as:
& 2 5) which
AUCM(PI.P2) = E [[()))()))
(CCUUC((112)) P2) = 1M))> max
h(=*)
(12)
aNp. 2--p2
p-pi 2o-p
where PI: Pa are in-resp. out-distribution and the indicator function 1 returns 1 if the expression in its argument is true and otherwise.
For all but one of our baselines, the OOD detecting feature the confidence of the classifier. Since the exact evaluation of the WCAUC is computationally infeasible, we compute upper bound and
7

---

Table 2: OOD performance: For all models report accuracy the test set of the in-distribution and AUCs, guaranteed AUCs (GAUC), adversarial AUCs (AAUC) for different test out-distributions. The radius of the & -bal for the adversarial manipulations of the OOD data is 0.01 for all datasets. The bias shift A that was used for ProoD is shown for each in-distribution. The AAUCs and GAUCs for ProoD tend be very close, indicating remarkably tight certification bounds. Models with accuracy drop of >> 3% relative to the model with highest accuracy are grayed out. Of the remaining models, we highlight the best OOD detection performance.
In: CIFARIO
CIFARI00 AUC GAUC AAUC
SVHN AUC GAUC AAUC|
LSUN CR
Smooth
Acc
AUC
GAUC AAUC| AUC GAUC AAUC
Plain
95.01
90.0
0.0
0.7
93.8
0.0
0.3
93.1
0.0
0.5
98.0
0.0
0.7
OE
94.91
91.1
0.0
0.9
97.3
0.0
0.0
100.0
0.0
2.7
99.9
0.0
1.5
ATOM
93.63
78.3
0.0
21.7
94.4
0.0
24.1
79.8
0.0
20.1
99.5
0.0
73.2
ACET
93.43
86.0
0.0
4.0
99.3
0.0
4.6
89.2
0.0
3.7
99.9
0.0
40.2
GOOD 80
87.39
76.7
47.1
57.1
90.8
43.4
76.8
97.4
70.6
93.6
96.2
72.9
89.9
GOOD 100 *
86.96
67.8
48.1
49.7
62.6
34.9
36.3
84.9
74.6
75.6
87.0
76.1
78.1
ProoD- Disic
62.9
57.1
57.8
72.6
65.6
66.4
78.1
71.5
72.3
59.2
49.7
50.4
ProoD 3
94.99
89.8
46.1
46.8
98.3
53.3
54.1
100.0
58.3
59.7
99.9
38.2
38.8
In: CIFARI00 100
CIFARIO
SVHN
LSUN CR
Smooth
Acc
AUC GAUC AAUC
AUC
GAUC
AAUC|
AUC
GAUC AAUC| AUC GAUC AAUC
Plain
77.38
77.7
0.0
0.4
81.9
0.0
0.2
76.4
0.0
0.3
86.6
0.0
0.4
OE
77.25
77.4
0.0
0.2
92.3
0.0
0.0
100.0
0.0
0.7
99.5
0.0
0.5
ATOM
68.32
78.3
0.0
50.3
91.1
0.0
67.0
95.9
0.0
75.6
98.2
0.0
80.7
ACET
73.02
73.0
0.0
1.4
97.8
0.0
0.7
75.8
0.0
2.6
99.9
0.0
12.8
ProsD-Disc
56.1
52.1
52.3
61.0
58.2
58.4
70.4
66.9
67.1
29.6
26.4
26.5
ProoD A 5
77.16
76.6
17.3
17.4
91.5
19.7
19.8
100.0
22.5
23.1
98.9
9.0
9.0
In: ImgN
Flowers AUC GAUC AAUC|
FGVC
Cars
Smooth
Acc
AUC
GAUC
AAUC|
AUC
GAUC AAUC|
AUC GAUC AAUC
Plain OE
96.34
92.3
0.0
0.5
92.6
0.0
0.0
92.7
0.0
0.1
98.9
0.0
8.6
97.10
96.9
0.0
0.2
99.7
0.0
0.4
99.9
0.0
1.8
98.0
0.0
1.9
ProoD-Disc
81.5
76.8
77.3
92.8
89.3
89.6
90.7
86.9
87.3
81.0
74.0
74.8
ProoD A 44
97.25
96.9
57.5
58.0
99.8
67.4
67.9
99.9
65.7
66.2
98.6
52.7
53.5
*Uses different architecture of classifier, see "Baselines" in Section
4.2
a lower bound on the WCAUC by finding 11(x) & =al he()) < h(2). We find the upper bound on the WCAUC the adversarial AUC (AAUC) by maximizing the confidence using an adversarial attack inside the I ball (i.e. finding an h). We compute lower bound on the WCAUC the guaranteed AUC (GAUC) by computing upper bounds on the confidence inside the L ball (i.e. h) via IBP. For poo-palvable methods, no non-trivial upper bound h < oc is available so their GAUC's are always O. Note that our threat model is different from adversarial robustness on the in-distribution which neither our method nor the baselines pursue. Since practical OOD detection scenarios require the selection of a threshold, we also evaluate the false positive rate (FPR) at 95% true positive rate and show the results in App. F
Vanishing gradients 47 3 are significant challenge for the evaluation of AAUCS
g even more
than in the evaluation of adversarial robustness on the in-distribution as the models are trained to be "flat" on the out-distribution. Thus we use an ensemble of variants of projected gradient descent (PGD) B7 as well as the black-box SquareAttack [ with 5000 queries. We use APGD [13] (except on RmmgNet, due to a memory leak) with 500 iterations and 5 random restarts. We also use a 200-step PGD attack with momentum of 0.9 and backtracking that starts with step size of 0.1 which is halved every time a gradient step does not increase the confidence and gets multiplied by 1.1 otherwise. As stated above, the models are trained to be flat and thus the gradients can be exactly zero which renders gradient-based optimization impossible. Therefore is important to use a variety different starting points. For PGD we start from: i) decontrasted version of the image, i.e. the point that minimizes the Loc diittance to the grey image 0.5 within the threat model, ii) 3 uniformly drawn

---

samples from the threat model, and iii) 3 versions of the original image perturbed by Gaussian noise with o 10-4 and then clipped to the threat model. We always clip to [O. 1] each step of the attack. For all attacks and models we directly optimize the final score used for OOD detection.
Baselines We compare to a normally trained baseline (Plain) and outlier exposure (OE), both trained using the same architecture and hyperparameters as the classifier in ProoD. For both ATOM and ACET we found the models' OOD detection to be much less adversarially robust than claimed in [ (see App. H so we retrained their models using the our architecture, threat model and training out- distribution with their original code (for CIFAR CIFAR10/100). Running these adversarial training procedures on ImageNet resolution infeasibly expensive. For GOOD we also retrain using OpenImages as training OOD dataset with the code from [ (comparisons with their pre- pre-trained models can be found in App. D Since they are only available on CIFARIO, we tried to train models on CIFAR 100 using their code and same hyperparameters and schedules as they used for CIFARIO. This only lead to models with accuracy below 25%, so we do not include these models in our evaluation. Since CCU already shown not provide benefits over OE on OOD data that is not very far from the in-distribution (e.g. uniform noise) [41] [8] we do not include it as baseline. We also evaluate the OOD- D00-performance of the provable binary discriminator (ProoD-Disc) that we trained for ProoD.
Note that this is not a classifier and is included only for reference. All results are in Table 2
Results ProoD achieves non-trivial GAUCs on all datasets. As was also observed in 81, this shows that the IBP guarantees not only generalize to unseen samples but even to unseen distributions. In App. [we we show that they even generalize to the larger threat model = 8/255. In general, the gap between GAUCs and AAUCs is extremely small. This shows that the seemingly simple IBP bounds can be remarkably tight, as has been observed in other works 17 23 It also shows that there would be very little benefit in applying stronger verification techniques like [10 24 15] in ProoD. Similarly, it demonstrates the strengths of our attack as there provably does not exist an attack that could lower the AAUCs on our ProoD model by more than 1.4% on any of the out-distributions. The bounds are also much tighter than for GOOD, which is likely due to the fact that for GOOD the confidence is much harder optimize during an attack because it involves maximizing the confidence in an essentially random class.
For CIFARIO, on 3 out of 4 out-distributions ProoD's GAUCs are higher than ATOM's and ACET's AAUCs, i.c. our model's provable adversarial robustness exceeds the SOTA methods' empirical adversarial robustness in these cases. Note that this is not due our retraining. because the authors' pre-trained models perform even more poorly (as shown in App. E On CIFAR100, ProoD's guarantees are weaker and ATOM produces strong AAUCs. However, we observe that training both ACET and ATOM can produce inconsistent results, i.e. sometimes almost no robustness is achieved. For the successfully trained robust ATOM model on CIFAR 100 we observe drastically reduced accuracy. Due to the difficulty in attacking these models, it is not unlikely that a more sophisticated attack could produce even lower AAUCs. Combined with the fact that both ACET and ATOM rely on expensive adversarial training procedures we argue that using ProoD is preferable in practice.
On CIFARIO, we that ProoD's GAUCs are comparable to, if slightly worse than the ones of both GOODgO and GOOD100- Note that although the presented GOOD models are retrained, the same observations hold true when comparing to the pre-trained models (see App. E However, point out that ProoD achieves this while retaining both high accuracy and OOD performance, both of which are lacking for GOOD. It is also noteworthy that the GOOD models' memory footprints are over twice as large as ProoD's. Generally, for ProoD the accuracy is comparable to OE and the OOD performance is similar or marginally worse. Thus ProoD shows that it is possible achieve certifiable adversarial robustness on the out-iistribution while keeping very good prediction and OOD detection performance. Note that all methods struggle on separating CIFARIO and CIFARIO0 when using OpenImages as training OOD (as compared to 80M Tiny Images in App. E
want to
To the best of our knowledge with I.ImgNet we provide the first worst case OOD guarantees on high-resolution images. The GAUCs are higher than on CIFAR, indicating that meaningful certificates on higher resolution are more achievable on this task than one might expect. FGVC and Cars may seem simple to separate from the animals in R. ImgNet but this cannot be said for Flowers which are difficult to provably distinguish from images of insects on flowers.
In summary, ProoD achieves our goal of maintaining high accuracy and clean OOD detection performance while providing provably adversarially robust OOD detection. In fact, out of all the

---

methods that do not significantly impair the in-distribution accuracy, ProoD is the only method providing such guarantees as well while simultaneously having the highest empirical robustness. Also note that for applications where adversarial robustness on the in-distribution is desired despite the induced reduction in accuracy, one can combine our ProoD model with a robustly trained classifier. In App.[K] K we demonstrate that ProoD in fact improves the clean as well as the robust OOD detection performance in this setting.
Limitations & Impact Like our baselines, ProoD depends on a suitable training out-distribution being available. Also, as noted throughout the paper, our method only focuses on adversarial robustness around OOD samples and does not by itself achieve adversarial robustness around the in-distribution. Furthermore, our considered threat model & and while the general architecture could also be applied to other threat models, this would require moving beyond IBP. We also show in App. that non-trivial bounds on the adversarial FPR are beyond all current methods, including ProoD.
Since our work aims to make ML models safer and more reliable, we do not anticipate any misuse of the technology. Additionally, since our method does not rely on any adversarial training. the environmental impact of training is lower than for most of our baselines. However, there have been concerns about any use of the 80M Tiny Images dataset, which is why we include these results only in App. E for back-compatibility.
Conclusion
We have demonstrated how to combine a provably adversarially robust binary discriminator between in- and out-distribution with a standard classifier in order to simultaneously achieve high accuracy, high clean OOD detection performance well as certified adversarially robust OOD detection. Thus, we have combined the best properties previous work with only a small increase in total model size and only single hyperparameter. This suggests that certifiable adversarial robustness on the out-distribution (as opposed to the in-distribution) is indeed possible without losing accuracy. We further showed how in our model simply enforcing negativity in the final weights of the discriminator fixes the problem of asymptotic overconfidence in ReLU classifiers. Training ProoD models is simple and stable and thus ProoD provides OOD guarantees that come (almost) for free.
Acknowledgements
The authors acknowledge support from the German Federal Ministry of Education and Research (BMBF) through the Tübingen AI Center (FKZ: 01IS18039A) and from the Deutsche Forschungsge- meinschaft (DFG, German Research Foundation) under Germany's 's Excellence Strategy (EXCnumber 2064/1, Project number 390727645), as well as from the DFG TRR 248 (Project number389792660), The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Alexander Meinke. We also thank Maximilian Augustin for helpful advice.
References
[1] M. Andriushchenko, F. Croce, N. Flammarion, and M. Hein. Square attack: a query-efficient black-box adversarial attack via random search. ECCV, 2020.
[2] R. Arora, A. Basuy, P. Mianjyz, and A. Mukherjee. Understanding deep neural networks with rectified linear unit. In ICLR, 2018
[3] A. Athalye, N. Carlini, and D. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
[4] M. Augustin, A. Meinke, and M. Hein. Adversarial robustness on in-and out-distribution improves explainability. In ECCV, 2020.
[5] L. Berrada, S. Dathathri, K. Dvijotham, R. Stanforth, R. Bunel, J. Uesato, S. Gowal, M. P. Kumar, et al. Make sure you're unsure: A framework for verifying probabilistic specifications. In NewrIPS, 2021.
10

---

[6] A. Birhane and V. U. Prabhu. Large image datasets: A pyrrhic win for computer vision? In WACV, 2021.
[7] J. Bitterwolf, A. Meinke, M. Augustin, and M. Hein. Breaking down out-of distriibitt detection: Many methods based on OOD training data estimate combination of the same core quantities. In JCML, 2022.
[8] J. Bitterwolf, A. Meinke, and M. Hein. Certifiably adversarially robust detection of out-of- distribution data. NeurIPS, 2020.
[9] J. Chen, Y. Li, X. Wu, Y. Liang, and S. Jha. Atom: Robustifying out-of-distribution detection using outlier mining. In ECML, 2021.
[10] C.-H. Cheng, G. Nühhenberg, and H. Ruess. Maximum resilience of artificial neural networks.
In International Symposium on Automated Technology for Verification and Analysis, 2017.
[11] M. Cimpoi, S. Maji. I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014.
[12] F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein. Robustbench: standardized adversarial robustness benchmark. In Thirty-fifhh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
[13] F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-Fe free attacks. In ICML, 2020.
[14] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le. Autoaugment: Learning augmentation strategies from data. In CVPR, 2019.
[15] S. Dathathri, K. Dvijotham, A. Kurakin, A. Raghunathan, J. Uesato, R. Bunel, S. Shankar, J. Steinhardt, I. Goodfellow, P. Liang, et al. Enabling certification of verification-agnosiic networks via memory-efficient semidefinite programming. In NeurIPS, 2020.
[16] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, 2009.
[17] S. Gowal, K. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, R. Arandjelovic, T. Mann, and P. Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv:1810.121
2018.
[18] S. Gowal, S.-A. Rebuffi, O. Wiles, F. Stimberg, D. A. Calian, and T. A. Mann. Improving robustness using generated data, NeurIPS, 2021.
[19] C. Guo, G. Pleiss, Y. Sun, and K. Weinberger. On calibration of modern neural networks. In ICML, 2017.
[20] M. Hein, M. Andriushchenko, and J. Bitterwolf. Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In CVPR, 2019.
[21] D. Hendrycks and K. Gimpel. A baseline for detecting misclassified and out-of-distribution examples neural networks. ICLR, 2017.
[22] D. Hendrycks, M. Mazeika, and T. Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019.
[23] N. Jovanović, M. Balunovic, M. Baader, and M. Vechev. Certified defenses: Why tighter relaxations may hurt training? arXiv:2102.06700, 2021.
[24] G. Katz, C. Barrett, D. Dill, K. Julian, and M. Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In CAV, 2017.
[25] A.-K. Kopetzki, B. Charpentier, D. Zügner, S. Giri, and S. Günnemann. Evaluating robustness of predictive uncertainty estimation: Are dirichlet-based models reliable? arXiv:2010. 14986, 2020.
11

---

[26] J. Krause, M. Stark, J. Deng, and L. Fei-Fei. 3d object representations for fine-grand Lineggriiedd categorization. In ICCV vision workshop. 2013.
[27] A. Kristiadi, M. Hein, and P. Hennig. Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks. In ICML, 2020.
[28] A. Kristiadi, M. Hein, and P. Hennig. Fixing asymptotic uncertainty of bayesian neural networks with infinite relu features. arXiv:2010.02709, 2020.
[29] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
[30] A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, A. Kolesnikov, et al. The open images dataset v4. International Journal of Computer Vision, 2020.
[31] K. Lee, H. Lee, K. Lee, and J. Shin. Training confidencc--calibraeed classifiers for detecting out-of-distribttio samples. In ICLR, 2018.
[32] K. Lee, K. Lee, H. Lee, and J. Shin. A simple unified framework for detecting disttiibbtittto samples and adversarial attacks. In NeurIPS, 2018
[33] Liang, Y. Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In ICLR, 2018.
[34] W. Liu, X. Wang, Owens, and Y. Li. Energy-based out-of-distribution detection. Advances in Neural Information Processing Systems, 2020.
[35] D. Macêdo and T. Ludermir. Enhanced isotropy maximization loss: Seamless and high- performance out-of-distribtiion detection simply replacing the softmax loss. arXiv:2105.14339, 2021.
[36] D. Macêdo, T. Ren, C. Zanchettin, A. L. Oliveira, and T. Ludermir. Entropic out-of-distribution detection: Seamless detection of unknown examples. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[37] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Valdu, Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
[38] S. Maji, E. Rahtu, Kannala, M. Blaschko, and Vedaldi. Fine-grained visual classification of aircraft.
[39] A. Malinin and M. Gales. Predictive uncertainty estimation via prior networks. In NeurIPS, 2018.
[40] A. Malinin and M. Gales. Reverse kl-divergence training of prior networks: Improved uncer- tainty and adversarial robustness. In NeurIPS, 2019.
[41] A. Meinke and M. Hein. Towards neural networks that provably know when they don't 't know. In ICLR, 2020.
[42] M. Mirman, T. Gehr, and M. Vechev. Differentiable abstract interpretation for provably robust neural networks. In ICML, 2018.
[43] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. In NeurIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
[44] A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
[45] M.-F. Nilsback and A. Zisserman. Automated flower classification over large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, 2008.
12

---

[46] A.-A. Papadopoulos, M. R. Rajati, N. Shaikh, and J. Wang. Outlier exposure with confidence detection. Neurocomputting, 2021.
control for
[47] N. Papernot, P. McDaniel. I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami. Practical black-box attacks against machine learning. In ACM ASIACCS, 2017.
[48] J. Ren, P. J. Liu, H. Fertig, J. Snoek, R. Poplin, M. A. DePristo, V. Dillon, and B. Lakshmi- narayanan. Likelihood ratios for out-of-distributiin detection. In NeurIPS, 2019.
[49] O. Russakovsky. J. Deng, H. Su, Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 2015.
[50] V. Sehhwag, A. N. Bhagoji, L. Song, C. Sitawarin, D. Cullina, M. Chiang, and P. Mittal. Better the devil you know: An analysis of evasion attacks using outrof-distribution adversarial examples. preprint, arXiv: 090599957072 0726, 2019.
[51] M. Sensoy, L. Kaplan, and M. Kandemir. Evidential deep learning quantify classification uncertainty. In NeurlPS, 2018.
[52] A. Torralba, R. Fergus, and W. T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. JEEE transactions on pattern analysis and machine intelligence, (0(11))1958-199)) 2008.
[53] D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry. Robustness may be at odds with accuracy. In ICLR, 2018.
[54] Xu, K. Ehinger, Y. Zhang, A. Finkelstein, S. Kulkarni, and Xiao. Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv:/504.000 2015.
[55] F. Yu, Y. Zhang, S. Song, A. Seff, and J. Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. CoRR, htt/1506.033360 2015.
[56] H. Zhang, H. Chen, C. Xiao, S. Gowal, R. Stanforth, B. Li, D. Boning, and C.-J. Hsieh. Towards stable and efficient training of verifiably robust neural networks. In ICLR, 2020.
[57] B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba. Places: 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6))1452-1444 1464, 2017.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] In order to be mindful of our resource consumption, we limited the measurement of error bars to single in m-diisriibutinn, see App. [
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes)
(b) Did you mention the license of the assets? [Yes] See App.D)
(c) Did you include any assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "wrNNo95O8Ni", "reviewer": "Reviewer_7nMZ", "summary": "This paper proposes a new OOD detection method that is provably robust against perturbations to OOD inputs. This method, called ProoD, has several desired properties including high detection performance as well as OOD detection certified robustness. ProoD makes use of a classifier and a certified binary discriminator for detection OOD samples and computes the class likelihoods as a product of the two functions. Theorem 1 in the paper guarantees that the network confidence would asymptotically converge to the uniform distribution and avoids overconfident predictions common in ReLU networks. ", "questions": "I do not have any additional questions. ", "limitations": "As it is also mentioned in the text, the proposed method only tackles one side of the problem which concerns adversarial robustness to OOD samples and by design does not consider the possibility of malicious perturbations to ID samples -- which is equally likely in principle. From that point of view the proposed method is severely handicapped. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**strengths** \n- well written paper \n- detailed explanation of the methods, theoretical background and contribution\n- thorough evaluations\n\n**weaknesses** \n- The main weakness of this work is the limited scope of the problem considered. As it is mentioned in the text, ProoD is designed to only consider the possibility of adversarial perturbations on the OOD samples (and not ID) and thus only seeks robustness in that regard. While this could be considered as a half-full half-empty kind of situation, I'm afraid in practice this is a critical downside which renders the method ultimately impractical.  \n- While the empirical results, reported in Table-2, are very thorough, the reported numbers are somewhat mixed. E.g. ProoD outperforms other baselines on most OOD sets in terms of the newly introduced GAUC. However, when considering the more common AAUC metric, it underperforms in most cases. Also related to the previous point, the evaluations seem to be always on OOD samples, so it limits the ability to assess if the method can additionally generalize its robustness  to ID samples. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "uhjmXxG2oF", "reviewer": "Reviewer_XeAG", "summary": "This paper studies a significant and interesting topic: robust Out-of-Distribution (OOD) detection issue. Specifically, the authors propose a new method which combines a provably adversarially robust binary discriminator and standard classifier. Then, authors show that their method can simultaneously achieve high accuracy, high clean OOD detection performance as well as certified adversarially robust OOD detection. Finally, the authors verify the effectiveness of their method through extensive experiments.", "questions": "see the weaknesses above.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths \n\n1，\tTraining Prood model proposed in this paper is easy and stable and thus Prood provides OOD guarantees that come for free.\n\n2，\tThe authors provide a theoretical guarantee for their proposed method.\n\n\nWeaknesses:\n\n1，\tRelated work is not presented clearly. It is hard to follow the intuition or motivation of authors.\n\n2，\tI would have preferred to see more authors’ comparison with previous related work\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "StcmqNMxy4q", "reviewer": "Reviewer_vBCS", "summary": "The paper proposes a noval approach, ProoD, to integrate provably robustness against adversary on OOD data into OOD detection and achieve high classification accuracy. In partcular, Prood first learns a certifably robust model for the binary classification task between in- and out-distribution, and enable its integration of with state-of-the-art OOD detection methods, like OE, via joint training. Prood demonstrated its effectiveness against asymptotic overconfidence of deep neural network both theoretically and empirically. The authors also perform comprehensive empirical study for Prood on benchmark datasets and against various SOTA OOD methods. \n", "questions": "- Since binary shift is a core hyper-parameter for joint training, can authors explain why AUC increases with bias shift whereas GAUC decreases with the bias shift?\n- Since the adversarial robustness is evaluated on OOD detection task instead of the naive classification, I am wondering if the authors explore the model-specific adaptive attack? It seems that currently the authors use an ensemble of variants of projected gradient descent. Particularly, I am not sure how robust the joint training procedure is against adaptive attacks. \n- I am wondering empirically what is the computation cost of the different variants of Prood, like Prood-S, Prood-disc, relative to other OOD detection methods, like OE?  \n", "limitations": "- As the authors mentioned in the paper, their approach only handles adversary around OOD samples instead of the in-distribution samples, and the studied adversary perturbation is generally limited to l_\\infty ball. \n- The type of allowed adversary, limited to small perturbation sets, could be unrealistic for OOD data, which can have more different forms of uncertainty for the adversary. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \n- The paper considers a novel setup of provably adversarially robust OOD, and the Prood is well-motivated with theoretical soundness.\n- The guarantee on asymptotic confidence is an interesting aspect with empirical validation.\n- The empirical study is comprehensive and evaluated on various benchmark datasets \n\nWeakness: \n- The proposed variation of the algorithms though well-motivated, is poorly presented. For better presentation, I suggest the author create a latex algorithm box and describe the different options: separate training, semi-joint training, Prood-disc, and Prood-\\Delta. Does Prood-disc corresponds to Prood-\\Delta = 0? \n- The empirical results require more clarification: \n\t- Highlight the best performing algorithm in each setup for table 2 and tables in the appendix\n\t- Explain specifically how the bias shift is chosen. I am assuming the authors take the bias shift that maximizes AUC+GAUC+Accuracy. Is there results for AAUC under different bias shift? \n\ntypos: \n- PDG should be PGD for line 289", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "De4NgyKjMrJ", "reviewer": "Reviewer_ejdX", "summary": "This paper proposes ProoD which merges a certified binary classifier for in-versus out-distribution with a classifier for the in-distribution task in a principled fashion into a joint classifier. It shows that ProoD simultaneously achieves three properties: 1) guaranteed adversarially robust OOD detection via certified upper bounds on the confidence in $l_\\infty$-balls around OOD samples; 2) ProoD provably prevents the asymptotic overconfidence of deep neural networks; 3) ProoD can be used with arbitrary architectures and has no loss in prediction performance and standard OOD detection performance. Extensive experiments are performed to show the promising performance of the proposed method ProoD.", "questions": "1. How to interpret the experimental results? Does the proposed method ProoD achieve a good trade-off between clean accuracy and adversarial robustness? Does ProoD outperform existing methods in terms of adversarial robustness? \n\n2. How to select a suitable threshold for the proposed method ProoD? Does ProoD have good performance under the False Positive Rate at 95% true positive rate (FPR) metric? ", "limitations": "The limitations and potential negative societal impact are properly addressed. ", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I think this paper has the following strengths:\n\n1. It derives the method ProoD in a principled way and proves that the method can prevent the asymptotic overconfidence of deep neural networks. Also, ProoD can be used with arbitrary architectures and has no loss in prediction performance and standard OOD detection performance;\n\n2. It performs extensive experiments to evaluate the proposed method ProoD and also compares ProoD to existing baselines;\n\n3. It is well-written and the related works are properly discussed.\n\nHowever, I think this paper has the following weaknesses:\n\n1. The idea of training a discriminator independently via interval bound propagation (IBP) to achieve certified robustness on out-of-distribution samples is not very novel since IBP is an existing technique that can ensure certified robustness;\n\n2. The performance of the proposed method ProoD is quite mixed across different in-distribution and OOD datasets. For example, in Table 2, on CIFAR-10 vs. Smooth, the performance of ProoD is worse than that of ATOM and GOOD in terms of AAUC metric, and in terms of GAUC metric, the performance of ProoD is also worse than that of GOOD; On CIFAR-10 vs. LSUN_CR, the performance of ProoD is also worse than that of GOOD in terms of GAUC metric. On CIFAR-100 as the in-distribution dataset, it seems the performance of ProoD is usually worse than that of ATOM in terms of the AAUC metric. In Table 7 (in the appendix), when evaluating the method on more OOD test datasets, the performance of ProoD is also mixed. Although ProoD usually has higher accuracy compared to the baselines, its adversarial robustness is worse when it achieves higher accuracy. So there is a trade-off here. For robust OOD detection, I think we should consider both clean accuracy and adversarial robustness. However, it seems ProoD doesn't achieve good performance when considering both clean accuracy and adversarial robustness (or achieve a good trade-off). \n\n3. In practice, we need to pick a threshold for OOD detection. Thus, the False Positive Rate at 95% true positive rate (FPR) metric is also very important. Some results under the FPR metric are reported in Appendix F. However, based on the results in Table 6 (Appendix F), it seems the performance of ProoD is not good, and ProoD doesn't have clear advantages over the existing methods under the FPR metric. Thus, it might be hard to select a suitable threshold for the proposed method ProoD. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept