Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?
Authors: Anna-Kathrin Kopetzki
Bertrand Charpentier
Daniel Zügner
Sandhya Giri
Stephan Günnemann
Venue:  (2021)

Abstract: Robustness to adversarial perturbations and accurate uncertainty estimation are crucial for reliable application of deep learning in real world settings. Dirichlet-based uncertainty (DBU) models are a family of models that predict the parameters of a Dirichlet distribution (instead of a categorical one) and promise to signal when not to trust their predictions. Untrustworthy predictions are obtained on unknown or ambiguous samples and marked with a high uncertainty by the models.

In this work, we show that DBU models with standard training are not robust w.r.t. three important tasks in the field of uncertainty estimation. First, we evaluate how useful the uncertainty estimates are to (1) indicate correctly classified samples. Our results show that while they are a good indicator on unperturbed data, performance on perturbed data decreases dramatically. (2) We evaluate if uncertainty estimates are able to detect adversarial examples that try to fool classification. It turns out that uncertainty estimates are able to detect FGSM attacks but not able to detect PGD attacks. We further evaluate the reliability of DBU models on the task of (3) distinguishing between in-distribution (ID) and out-of-distribution (OOD) data. To this end, we present the first study of certifiable robustness for DBU models. Furthermore, we propose novel uncertainty attacks that fool models into assigning high confidence to OOD data and low confidence to ID data, respectively. 
Both approaches show that detecting OOD samples and distinguishing between ID-data and OOD-data is not robust.

Based on our results, we explore the first approaches to make DBU models more robust. We use adversarial training procedures based on label attacks, uncertainty attacks, or random noise and demonstrate how they affect robustness of DBU models on ID data and OOD data.

Under review as a conference paper at ICLR 2021
EVALUATING ROBUSTNESS OF PREDICTIVE UNCER- TAINTY ESTIMATION: ARE DIRICHLET-BASED MOD- ELS RELIABLE?
Anonymous authors Paper under double-blind review
ABSTRACT
Robustness to adversarial perturbations and accurate uncertainty estimation are crucial for reliable application of deep learning in real world settings. Dirichlet- based uncertainty (DBU) models are a family of models that predict the parameters of a Dirichlet distribution (instead of categorical onc) and promise to signal when not to trust their predictions. Untrustworthy predictions are obtained on unknown ambiguous samples and marked with high uncertainty by the models.
In this work, we show that DBU models with standard training are not robust w.rt. three important tasks in the field of uncertainty estimation. First, evaluate how useful the uncertainty estimates are to (1) indicate correctly classified samples. Our results show that while they are good indicator unperturbed data, performance on perturbed data decreases dramatically. (2) We evaluate if uncertainty estimates are able detect adversarial examples that try to fool classification. turns out that uncertainty estimates are able to detect FGSM attacks but not able to detect PGD attacks. We further evaluate the reliability of DBU models on the task of (3) distinguishing between in in-distribution (ID) and
(OOD) data. To this end, we present the first study of certifiable robustness for DBU models. Furthermore, we propose novel uncertainty altacks that fool models into assigning high confidence to OOD data and low confidence to ID) data, respectively. Both approaches show that detecting OOD samples and distinguishing between ID-data and OOD-data is not robust.
Based on our results, we explore the first approaches to make DBU models more robust. We use adversarial training procedures based on label attacks, uncertainty attacks, random noise and demonstrate how they affect robustness of DBU models on ID data and OOD data.
INTRODUCTION
Neural networks achieve high predictive accuracy in many tasks, but they known have two substantial Weaknesses: First, neural networks are not robust against adversarial perturbations, i... semantically meaningless input changes that lead to wrong predictions (Szegedy et al. 2014] Goodfellow et al. 2015). Second, neural networks tend to make over-confident predictions at test time al. 2017). Even worse, standard neural networks are unable to identify samples that are different from the samples they were trained on. In these cases, they provide uninformed decisions instead of abstaining. These weaknesses make them impracticable in sensitive domains like financial, autonomous driving or medical areas which require trust in predictions.
(Lakshminarayanan
To increase trust in neural networks, models that provide predictions along with the corresponding uncertainty have been proposed. There are three main families of models that aim to provide meaningful estimates of their predictive uncertainty. The first family are Bayesian Neural Networks (Blundell al. 2015 Osawa et al.) 2019 Maddox et al. 2019), which have the drawback that they are computationally demanding. The second family consists of Monte-Cart -arlo drop-out based models et al., 2017) that estimate uncertainty by computing statistics such as mean and variance by aggregating forward passes multiple models. A disadvantage of all of these models is that uncertainty estimation at inference time is expensive.
(Gal & Ghahramani 2016) and ensembles (Lakshminaray

---

Under review as a conference paper at ICLR 2021
In contrast to these, the recently growing family of Dirichlet- based uncertainty (DBU) models
(Malinin & Gales, 2018a| 2019 Sensoy et al. 2018, Malinin et al. 2019 Charpentier et al. (020)
directly predict the parameters of Dirichlet distribution over categorical probability distributions. They provide efficient uncertainty estimates at test time since they only require a single forward pass. DBU models bring the benefit of providing both, aleatoric and epistemic uncertainty estimates. Alcatoric uncertainty irreducible and caused by the natural complexity of the data, such as class overlap or noise. Epistemic uncertainty results from the lack of knowledge about unseen data, e.g. when the model is presented an image of an unknown object. Both uncertainty types can be quantified using different uncertainty measures based on a Dirichlet distri- bution, such as differential entropy, mutual information, or pscudo-counts (Malinin & Gales, 2018a Charpentier et al. 2020). These uncertainty measures have been shown outstanding performance in, e.g., the detection of OOD samples and thus are superior to softmax based confidence (Malinin & Gales 2019 Charpentier et 2020).
(1))
Canfidencc
0.01
X 0.1
x 000D
[aage pert.
4
Inputspace
Neural networks from the families outlined above are ex- pected know what they don't know, i.e., notice when they are unsure about a prediction. This raises questions with regards adversarial examples: should uncertainty estimates detect these corrupted samples and abstain from making a prediction (i.e. indicated by high uncertainty in the prediction), or should they be robust to adversarial examples and produce the correct output even under perturbations? Using humans as the gold standard image classification and assuming that the perturbations are semantically meaningless, which is typically implied by small Lp norm of the corruption. we argue that the best option is that the models are robust to adversarial perturbations (see Figure [ Beyond being robust w.r.t. label prediction, we expect models to robustly know what they do not know. That is, they should robustly distinguish between ID and OOD data even if those are perturbed.
(OOD)
Figure Visualization of the desired uncertainty estimates.
In this work, focus DBU models and analyze their robustness capacity w.r.t. the classification decision and uncertainty estimations, going beyond simple softmax output confidence by investigating advanced measures like differential entropy. Specifically, we study the following questions:
1. Is high certainty a reliable indicator of correct predictions?
2. Can we use uncertainty estimates to detect label attacks on the classification decision?
3. Are uncertainty estimates such as differential entropy a robust feature for OOD detection?
In addressing these questions we place particular focus on adversarial perturbations of the input in order to evaluate the worst case performance of the models. We address question one by analyzing uncertainty estimation on correctly and wrongly labeled samples, without and with adversarial perturbations on the inputs. To answer question two, we study uncertainty estimates of DBU models on label altacks. More specifically, we analyze whether there is a difference between uncertainty estimates on perturbed and unperturbed inputs and whether DBU models are capable of recognizing successful label attacks by uncertainty estimation. Addressing question three, use robustness verification based on randomized smoothing and propose to investigate uncertainty attacks. Uncertainty attacks aim at changing the uncertainty estimate such that ID data is marked as OOD data and vice versa. Finally, we propose robust training procedures that use label attacks, uncertainty attacks or random noise and analyze how they affect robustness of DBU models on ID data and OOD data.
2 RELATED WORK
Recently, multiple works have analyzed uncertainty estimation and robustness of neural networks. (Snoek et al., 2019) compares uncertainty estimates of models based drop-out and ensembles under data set shifts. (Cardelli et al. 2019 Wicker et al.) (220)) study probabilistic safety of Bayesian networks under adversarial perturbations by analyzing inputs sets and the corresponding mappings
2

---

Under review as a conference paper at ICLR 2021
in the output space. In contrast, work focus DBU models and analyze their robustness w.r.t adversarial perturbations specifically designed to fool label or uncertainty predictions of the models. Furthermore, previous works attack defenses have focused on evaluating either robustness w.r.t. class predictions (Carlini & Wagner 2017: Weng et al. 2018) or label attack detection (Carlini & WWagner 2017). In contrast, our work jointly evaluates both tasks by analyzing them from the uncertainty perspective. Furthermore, in addition to label attacks, we study a new type of adversarial perturbations that directly target uncertainty estimation. Those attacks are different from traditional label attacks (Madry et al. 2018 Dang-Nion NNu et al. 2020
Different models have been proposed to account for uncertainty while being robust. (Smith & Gal 2018) and (Lee et al.) (011)) have tried to improve label attack detection based on uncertainty using drop-out or density estimation. In addition from improving label attack detection for large unseen perturbations, (Stutz et al., (020) aimed at improving robustness w.r.f. class label predictions on small input perturbations. To this end, they proposed a new adversarial training with softer labels for adversarial samples further from the original input. (Qin et al. 2020) suggested similar adversarial training where labels are soften differently depending on the input robustness. These previous works only consider the aleatoric uncertainty contained in the predicted categorical probabilities, i.e. the softmax output. They do not consider DBU models which explicitly account for both aleatoric and epistemic uncertainty. (Malinin & Gales 2019) proposed to improve a single type of DBU model for label attack detection by assigning them high uncertainty during training.
Please note that the works (Tagasovska & Lopez-Paz 2019 Kumar et al., 2020 Bitterwolf et al. 2020 Meinke & Hein 2020) study different orthogonal problem. (Tagasovska & Lopez-Paz 2019) propose to compute confidence intervals while (Kumar et al. 2020) propose certificates on softmax predictions. (Bittewool' et al. 2020) uses interval bound propagation to compute bounds on softmax predictions in the -bal around an OOD point and for ReLU networks, (Meinke & Hein 2020) proposes an approach to obtain certifiably low confidence for OOD data. These four studies estimate confidence based on softmax predictions, which accounts for aleatoric uncertainty only. this paper, we provide certificates on the OOD classification task using DBU models directly which is better suited to epistemic uncertainty measures.
3 DIRICHLET-BASED UNCERTAINTY MODELS
Standard (softmax) neural networks predict the parameters of a categorical distribution - - [p] (4) PC for a given input (11) E Rd where C is the number of classes. Given the parameters of a categorical distribution, we can evaluate its aleatoric uncertainty, which is the uncer- tainty on the class label prediction y( E {I.....Cr C). For example, when predicting the result of an unbiased coin flip, we expect the model to have high aleatoric uncertainty and predict p(head) = 0.5.
In contrast to standard (softmax) neural networks, DBU models predict the parameters of Dirichlet distribution the natural prior of categorical distributions - given input (n()) (i.e. g( ())(( where So (z)) = (6) E R$). Hence, the epistemic distribution g expresses the epistemic uncer- tainty on (0) i.e. the uncertainty on the categorical distribution prediction p From the epistemic distribution, follows an estimate of the aleatoric distribution of the class label prediction ()) where (o [p(*] = p(o) An advantage of DBU models is that one pass through the neural network is sufficient to compute epistemic distribution, aleatoric distribution, and predict the class label:
10 = Dir(a ())
9 P
(1) with 0o
Σ
(
y arg max [p]
PC
(1)
This parametrization allow to compute classic uncertainty measures in closed-form As an example, the concentration parameters ! can be interpreted as a pseudo-count of observed samples of class and, thus, are good indicator of epistemic uncertainty. Note that further measures, such as differential entropy of the Dirichlet distribution (see Equation 2 where I is the Gamma function and is the Digamma function) or the mutual information between the label e and the categorical p(* can also be computed in closed-form (App. A.2 (Malinin & Gales| (218a)). Hence, DBU models can efficiently use these measures to assign high uncertainty for unknown data making them specifically suited for detection of OOD samples like anomalies.
3

---

Under review as a conference paper at ICLR 2021
K MdiiE Σ - -nn ((o))
Σ -ac-1)- (cc(a))(P(aa) ((oo))
Several recently proposed models for uncertainty estimations belong the family of DBU models, such as PriorNet, EvNet, DDNet and PostNet. These models differ in terms of their parametrization of the Dirichlet distribution, the training, and density estimation. An overview of theses differences is provided in T We evaluate all recent versions of these models in our study.
Table 1: Summary of DBU models. Further details on the loss functions are provided in the appendix.
alraprraetrizztionn
Loss
OOD training data
Ensemble training
Density estimation
PostNet foia 1 (
Bayesian loss
No
No
Yes
PriorNet fola!!) = at
Reverse KL
Yes
No
No
DDNet
fsla(()) (2
Dir. Likelihood
No
Yes
No
EvNet
fsla!")
Expected MSE
No
Contrary to the other models, Prior Networks (PriorNet) (Malinin & Gales 2018a 2019) requires
OOD data for training to "teach" the neural network the difference between ID and OOD data. PriorNet is trained with loss function consisting of two KL-divergence terms. The fist term is designed to learn Dirichlet parameters for ID), while the second one is used to learn flat Dirichlet distribution (a 1) for OOD data. There a two variants of PriorNet. The first one is trained based on reverse KL-divergence (Malinin & Gales 2019), while the second one is trained with KL-divergence (Malinin & Gales] 2a) We include in our experiment the most recent reverse version of PriorNet, as it shows superior performance (Malinin & Gales 2019).
Evidential Networks (EvNet) (Sensoy et al., 2018 are trained with a loss that computes the sum of squares between the on-hot encoded true label y and the predicted categorical ( under the Dirichlet distribution. Ensemble Distribution Distillation (DDNet) (Malinin et al.) 2019) is trained in two steps. First, an ensemble of M classic neural networks needs to be trained. Then, the soft- Jm=1 provided by the ensemble of networks are distilled into a Dirichlet-based network by fitting them with the maximum likelihood under the Dirichlet distribution. Posterior Network (PostNet) (Charpentier et al.) 2020) performs density estimation for ID data with normalizing flows and uses a Bayesian loss formulation. Note that EvNet and PostNet model the Dirichlet parameters as fo(a:()) = + ( while PriorNet, RevPrionNet and DDNet compute them So (xx(1) (+))
(9) 1M
labels {pm
ROBUSTNESS OF DIRICHLET-BASED UNCERTAINTY MODELS
We analyze robustness of DBU models in the field of uncertainty estimation w.r.t. the following four aspects: accuracy, confidence calibration, label attack detection and OOD detection. Uncertainty quantified by differential entropy, mutual information and pseudo counts. A formal definition of all uncertainty estimation measures is provided in the appendix. Robustness of strichlet-based uncertainty models is evaluated based on label attacks and a newly proposed type of attacks called uncertainty allacks. While label attacks aim at changing the predicted class, uncertainty attacks aim at changing uncertainty assigned to a prediction. All existing works are based on label attacks and focus robustness w.r.t. the classification decision. Thus, are the first to propose attacks targeting uncertainty estimates such as differential entropy and analyze further desirable robustness properties of DBU models. Both attack types compute a perturbed input (i) close to the original input (+()) i.e. ( - ( < where is the attack radius. The perturbed input is obtained by optimizing a loss function ((xx) using Fast Gradient Sign Method (FGSM) or Projected Gradient Descent (PGD). We use also black box attack (Noise) which generates 10 Noise samples from Gaussian distribution with mean equal to the original sample. The pertrubed sample which fools the most the loss function is selected as an attack. To complement attacks, we propose the first study of certifiable robustness for DBU models, which is based on randomized smoothing (Cohen et al. 2019).
The following questions we address by our experiments have a common assessment metric. Distin- guishing between correctly and wrongly classified samples, between attacced input and attacked inputs or between ID data and OOD data can be treated as binary classification problems. To quantify the performance of the models on these binary classification problems, we compute AUC-PR.
4

---

Under review as a conference paper at ICLR 2021
Experiments are performed on two image data sets (MNIST (LeCun & Cortes 2010) and CIFARIO (Krizhevsky et al. 2209)), which contain bounded inputs and two tabular data sets (Segment (Dua & Graff 2017) and Sensorless drive (Dua & Graff 2017)). consisting of unbounded inputs. Note that unbounded inputs are challenging since it is impossible to describe the infinitely large OOD distribu- tion. As PriorNet requires OOD training data, we use two further image data sets (FashionMNIST (Xiao et al. (017) and CIFAR 100 (Krizhevsky et al. 2009)) for training on MNIST and CIFAR 10, respectively. All other models are trained without OOD data. To obtain OOD data for the tabular data sets, we remove classes from the ID data set (class window for the Segment data set and class 9 for Sensorless drive) and use them as the OOD data. See appendix for further details on the setup.
4.1 UNCERTAINTY ESTIMATION UNDER LABEL ATTACKS
Label attacks aim at changing the predicted class. To obtain a perturbed input with a different label, we maximize the cross-entropy loss ( R arg max, ((x) = CE(p()) eه under the radius constraint. For the sake of completeness to change class predictions and report their accuracy to show the effectiveness based on different radii (see Appendix, Table 7). As expected and partially shown by previous works, none of the DBU models is robust against label attacks. However, we noted that PriorNet is slightly more robust than the other models. This might be explained by the use of OOD data during training, which can be seen as some kind of robust training. From now on, we switch to the core focus of this work and analyze robustness properties of uncertainty estimation.
also analyze label attacks regarding their performance
Is high certainty reliable indicator of correct predictions?
Expected behavior: Predictions with high certainty are more likely to be correct than low certainty predictions. Assessment metric: We distinguish between correctly classified samples (label 0) and wrongly classified ones (label 1) based on the differential entropy scores produced by the DBU models (Malinin & Gales 2a) Correctly classified samples are expected to have low differential entropy, reflecting the model's 's confidence, and analogously that wrongly predicted samples tend to have higher differential entropy. Observed behavior: Note that the positive and negative classes are not balanced, thus, the use of AUC-PR scores (Saito & Rehmsmeier] 2015) are important to enable meaningful measures. While uncertainty estimates are indeed an indicator of correctly classified samples on non-perturbed data, none of the models maintains its high performance on perturbed data (see. Table 2). Thus, using uncertainty estimates as indicator for correctly labeled inputs is not robust to adversaril perturbations, although the used attacks do not target uncertainty.
Table 2: Certainty based on differential entropy under PGD label attacks (AUC-PR).
CIFARIO
Sensorless
Alt. Rad.
0.1
0.2
2.0
4.0
0.2
0.5
4.0
PostNer
98.7
38.6
56.2
7.8
1.2
0.4
0.3
99.7
8.3
3.9
3.6
7.0
9.8
11.3
PriceNet
92.9
37.6
24,9
3.0
10.5
0.7
2.2
DDNet
97.6
91.8
78.3
18.1
0.0
0.0
99.7
119
16
0.4
0.2
0.2
FvNet
97.9
85.9
57.2
10.2
4.0
2.4
0.3
99.9
22.9
1300
6.0
3.7
3.2
3.1
Can we use uncertainty estimates to detect label attacks on the classification decision?
Expected behavior: Adversarial examples are not from the natural data distribution. Therefore, DBU models expected to detect them OOD data by assigning them higher uncertainty. We expect perturbations with larger attack radius r to be easier to detect as they differ more significantly from the data distribution. Assessment metric; The goal of attack-detection is distinguish between unperturbed samples (label 0) and perturbed samples (label 1). To quantify the performance, we use the differential entropy (Malinin & Gales 2018a). differential entropy, reflecting the fact that they are from the distribution the models were trained on, while perturbed samples are expected to have a high differential entropy. Further results based on other uncertainty measures are provided in the appendix. Observed behavior: Table /hoows that the accuracy of all models decreases significantly under PGD label attacks, but none of the models is able to provide an equivalently increasing high attack detection rate (see Table 3). Even larger perturbations are hard to detect for DBU models. Although PGD label attacks do not explicitly consider uncertainty, they seem to provide adversarial examples with similar uncertainty as the original input. Such high certainty adversarial examples are illustrated in Figure [ where certainty is visualized based on the precision og that is supposed to be high for ID data and low for OOD)
contperdddeed samples are expected to have low
5

---

Under review as a conference paper at ICLR 2021
data. While the original imput (perturbation size 0.0) is correctly classified frog and ID data, there exist adversarial examples that are classified as deer or bird. The certainty on the prediction of these adversarial examples has similar even higher value than the prediction Using the differential entropy to distinguish between ID and OOD data results in the same ID/OOD) assignment since the differential entropy of the three right-most adversarial examples is similar or even smaller than on the unperturbed input.
the original input.
For the less powerful FGSM and Noise attacks (see Appendix), DBU models achieve mostly better attack detection rates than for PGD attacks. This suggests that uncertainty estimation is able to detect weak attacks, which consistent with the observations & Gales 2018b). Furthermore, PostNet provides bet- ter label attack detection rate for large perturbations on tabular data sets. An explanation for this observation is that the density estimation of the ID samples has been shown to work better for tabular data sets (Charpen tier et al. 2020). Standard adversarial training (based on label attacks targeting the crossentropy loss function) im- proves robustness w.r.t. class predictions (see Appendix, Table 32). but does not improve label attack detection performance of any model (see Table 40). Overall, none of the DBU models provides relibbee indicator for adversarial inputs that target the classification decision.
25
truck
dog
bird
automobile
30
(Malinin
perturbation
Figure 2: Input & corr. Dir.- Dir-parraeeters under label attacks (dotted: threshold to distinguish ID and OOD).
Table 3: Label Autack-Detectio by normally trained DBU models based on differential entropy under PGD label attacks (AUC-PR).
CIFARIO
Sensoeless
Atr. Rad.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
PostNer
63.4
66.9
42.1
33.9
31.6
31.2
47.7
42.3
36,9
48.5
85.0
90.0
PriceNer
53.3
56.0
55.6
49.2
42.2
35.4
38.8
33.6
51.4
33.1
40.9
53.5
DDN
33.9
46.3
46.3
41.7
4.2 ATTACKING UNCERTAINTY ESTIMATION
DBU models are designed to provide uncertainty esti-
mates (heyond softmax based confidence) alongside predictions and use this predictive uncertainty for OOD detection. Thus, in this section we focus on attacking these uncertainty estimates. We present result for attacks based on the differential entropy as loss function (œ(0) = arg max I(x) = Result
Diff-Ent(Dee (a) since is the most widely used metric for based further uncertainty measures, loss functions and details the uncertainty attacks provided in the appendix. Regarding uncertainty attacks, we analyze model performance w.r.t. two tasks. First, attacks are computed on ID data transform them in OOD data, while OOD data is left non-attacked. Second, we attack OOD data to transform it into ID data, while ID data is not attacked. Hence, uncertainty attacks aim at posing ID data as OOD data conversely.
Are uncertainty estimates a robust feature for OOD detection?
Expected bekavior: We expect Dirichlet-basee uncertainty models be able to distinguish between ID and OOD data by providing reliable uncertainty estimates, even under small perturbations. That is, we expect the uncertainty estimates of DBU models to robust under attacks. Assessment metric: We distinguish between ID data (label 0) and OOD data (label 1) based on the differential entropy as uncertainty scoring function (Malinin & Gales, 201 2018a). Differential entropy is expected to be small on ID samples and high OOD samples. Experiments on further uncertainty measure and results for AUROC are provided in the appendix. Observed behavior: OOD samples are perturbed as illustrated which is identified OOD. Adding adversarial
in Figure 3 The left part shows perturbations 2 0.5 to it changes the Dirichlet parameters such that the resulting images are identified ID, based precision differential entropy as uncertainty measure. Adding adversarial perturba-

---

Under review as a conference paper at ICLR 2021
truck ship honse
dog deer
bird automobile
14
airplane
0
perturbation norm
perturbation L2 norm
(a) OOD uncertainty attack
(b) uncertainty attack
Figure 3: ID and OOD input with corresponding Dirichlet-prrameterss under uncertainty attacks (dotted line: threshold to distinguish ID) and OOD).
tions to an ID sample (right part) results in images identified as OOD. OOD detection performance of all DBU models decreases rapidly with the size of the perturbation, regardless of whether attacks are computed on ID or OOD data (Table Thus, using uncertainty estimation to distinguish between ID and OOD data is not robust. PostNet and DDNet achieve slightly better performance than the other models. Further, PostNet provides better scores for large perturbations on tabular data sets which could again be explained by its density-based approach.
Table 4: OOD detection based on differential entropy under PGD uncertainty attacks against differen- tial entropy on ID data and OOD data (AUC-PR).
ID-Anack (nos-ttacked (OD)
(oon-attacked
Rud
2.0
0.5
CIFARIO SVHN
22.4
PricdNer
54.4
40.1
3000
17.9
15.6
15.4
15.4
54.4
20.7
30.7
19.5
16.5
15.7
15.4
DONet
828
71.4
59.2
25.9
16.0
15.4
15.4
82.8
72,0
57.2
20.3
15.6
15.4
15.4
80.3
23.9
Sens, Sens. class 10,
PostNer
36.0
82.6
PrionNet
32.3
26.6
26.5
26.5
26.6
28.3
32.3
26.7
26.6
26.6
27.0
30.1
DDNet
31.7
26.8
26.6
26.5
26.6
27.1
30.5
31.7
27.1
26.7
26.7
26.8
26.9
27.3
FvNer
66.5
30.5
28.2
27.1
28.1
31.8
37.5
66.5
38.7
36.1
30.2
28.2
28.8
32.2
4.3 ROBUST TRAINING FOR DBU MODELS & ID/OOD VERIFICATION
Our robustness analysis based label attacks and uncertainty attacks shows that neither the predicted class, nor the uncertainty corresponding to prediction, nor the differentiation between ID and OOD-data robust. Thus, we propose adversarial training procedures During training we augment the data set by samples computed based on (i) PGD attacks against the crossentropy loss or (ii) against the differential entropy function, which is used distinguish between ID and OOD data, or (iii) by adding random noise as proposed for randomized smoothing training.
enhance robustness.
Since attacks are used during robust training, we want avoid tying robustness evaluation to gradient based attacks. Instead, we propose the first approach that certifies robustness of DBU models based on randomized smoothing (Cohen al., 2019). Randomized smoothing was proposed to verify robustness w.r.t. class predictions and we modify it for ID/OOD-verifiatioo. As randomized smoothing treats classifiers as a black-box, we transform distinguishing between ID data (label 0) and OOD data (label !) into binary classification problem based on uncertainty measure, which requires to set a threshold for the uncertainty measure to obtain an actual decision boundary. This is in contrast to attack-based experiments where avoided setting thresholds by analyzing under the curve metrics. Thresholds for uncertainty measure are set for each model individually based on the validation set, such that the accuracy w.r.t. to ID/OOD-assigmmett of the model is maximized.
7

---

Under review as a conference paper at ICLR 2021
In the following we discuss results for (ID data) and SVHN (OOD data). Further results on other data sets, other uncertainty measures and results on the standard classification based randomized smoothing verification are shown in the appendix. Table shows the percentage of samples which are correctly identified as ID (resp. OOD) data and are certifiably robust within this type (cc; certified correct) along with the corresponding mean certified radius. The higher the portion of сс samples and the larger the radius the more robust DD/00D-distinguishin w.r.t. the corresponding perturbation size
based on differential entropy on CIFARIO
Table 5:
Randomized smoothing verification for different of CIFARIO (ID data) and SVHN (OOD data). Left part: percentage of samples that is correctly identified and certified as ID data (cc) and corresponding mean certified radius (R). Right part: same for OOD data.
Verification
Venification
0,1
0.2
0.5
0,1
0.2
0.5
cc
R
R
cc
cc
K
R
ads. train. loss: Nome
PriorNet
83.2
0.26
97.8
0.58
100.0
147
3,7
0.10
0.0
0.00
0.0
0.00
2366
0.00
99.3
1.37
DDNer
63.7
0.34
58.7
0.50
53.0
0.32
27.9
0.17
8.7
0.16
77.6
0.58
EvNet
53.2
0.15
58.3
0.20
13.1
0.34
$6.9
0.11
48.1
0.21
94,3
0.59
ady. train. loss: rand. smooth.
PriorNet
1.5
0,06
0.8
0.05
89.3
0.73
97.5
0.28
99.4
0.34
38.7
0.22
PostNer
63.3
0.46
0.36
95.4
0.26
92.9
0.48
73.2
0.63
DDNN
68.6
80.5
0.90
86.3
0.33
EvNet
38.9
0.27
56.6
0.45
63.9
0.98
92.9
0.27
74.4
0.46
85.6
0.81
adv. Irain. losa: crussmtropy
PrionNet
99.8
0.38
0.0
0.00
31.1
0.25
0.0
0.00
100.0
0.76
60.7
0.21
PownNer
22.2
0.:15
$1.2
0.21
0.0
0.00
99.4
11.2
44.9
0.18
100.0
1.44
DDNe
49.0
45.4
FoNet
29.4
0.12
84.2
0.26
2.4
0.09
96.6
0.16
8.4
0.10
100.0
0.55
adv. loss:
PriorNer
1.1
0.84
0.0
0.00
100.0
1.91
99.2
0.31
100.0
0.76
0.0
0.00
PostNer
30.5
0.17
6.1
0.13
0.0
0.00
94,9
0.17
99.8
0.55
100,0
1.17
DINNr
37.1
4.4
0.00
81.5
100.0
100.0
1,80
EoNer
38.6
0.15
0.11
77.9
91.8
59.8
For each model, we observe a performance jump between ID- and on ID data drops from high values to low ones while the ce percentage and radius on OOD-data increase. These jumps are observed for normal training as well as adversarial training based on the crossentropy or the differential entropy. Thus, either ID-verification or OOD-verification performs well, depending on the chosen threshold. Augmenting the data set with random noise perturbed samples (randomized smoothing loss) does not result in such performance jumps (except for PriorNet), but there also a trade-off between robustness on ID data versus robustness on OOD data and there is no parametrization where ID-verification and DOD-veeifiiation perform equally well.
where robustness
Table 6: Randomized smoothing verification for different a of CIFARIO 10 (ID data) and SVHN (OOD data): percentage of samples that is wrongly identified as ID/OOD and certifiably robust as this wrong type (cw) and corresponding mean certified radius (R). The lower cw, the more robust the model.
0.1
0.2
0.5
0.1
0.2
0.5
ew
R
CW
R
.W
CW
R
CW
R
cw
ads; train. lues: Nome
ads, truin. luss: rund, smooth,
PriorNet
15.9
0.13
1.9
0.18
0.0
0.00
98.2
(1.33
98.5
0.55
8,0
0.22
PostNet
76.9
0.17
73.5
0.21
100.0
1.30
35.7
0.16
16.7
0.34
32.3
0.17
DINNet
0.14
10.1
0.17
0.11
40.5
0.31
17.6
EvNet
45.0
0.09
37.2
0.15
$2.8
0.52
39.5
0.22
41.4
0.53
24.2
0.50
adv. traiin. crusscatropy
ady:
loss:
diffE
PriorNet
0.1
0.12
100.0
0.76
62.2
0.33
98.4
0.31
100.0
0.74
0.0
0.00
PowNNit
76.4
0.18
45.0
0.18
1000
1.28
68.0
0.15
93.5
0.42
100.0
1.10
DDNa
64.3
100.
0.19
95.5
0.57
100.0
ExNet
68.3
0.12
12.9
0.11
95.6
0.39
61.2
0.33
73,8
0.18
97.9
0.60
While Table the percentage of samples which are correctly identified and certified as ID/OOD data (cc), Table 6 shows the percentage of samples which are wrongly identified as ID/OOD data
We want to highlight again that attacks are here only used to enable robust training of the models. The robustness evaluation itself operates the original data (not attacked and, thus, seemingly easy); only smoothed via randomized smoothing. The verification provides us radius that guarantees rohustness around the sample.
8

---

Under review as a conference paper at ICLR 2021
and certifiably robust this wrong type (ew; certified wrong). These samples worse than adversarial examples. Neither robust training based on label attacks, uncertainty attacks nor noise perturbed samples consistently reduce the portion of certifiably wrong samples, worse it seems to increase the number of cw samples. Thus, although robust training improves DBU-model resistance against label attacks (see Appendix, Table 35), DD/OOD-verifictiion shows that each model is either robust on ID-data on OOD-data. Achieving robustness on both types is challenging, Our results rise the following question: How do we make DBU models robust w.r.t. class label predictions and ID/OOD-differema differrenniiinnnnn without favoring either performance on ID data or OOD data?
5
CONCLUSION
This work analyze robustness of uncertainty estimation by DBU models and answer multiple questions in this context. Our results show: (1) While uncertainty estimates are good indicator to identify correctly classified samples unperturbed data, performance decrease drastically on perturbed data- points. (2) None the Sirichlet-hased uncertainty models is able detect PGD label attacks against the classification decision by uncertainty estimation, regardless of the used uncertainty measure. (3) Detecting OOD samples and distinguishing between ID-data and OOD-data not robust. (4) Robust training based on label attacks uncertainty attacks increases performance of Dirichlet-basee uncertainty models w.r.. either ID data or OOD data, but achieving high robustness on both is challenging - and poses interesting direction for future studies.
REFERENCES
Julian Bitterwolf, Alexander Meinke, and Matthias Hein. Provable worst case guarantees for the data. Neural Information Processing Systems, 2020.
detection of
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. International Conference on Machine Learning, 2015.
Luca Cardelli, Marta Kwiatkowska, Luca Laurenti, Nicola Paoletti, Andrea Patane, and Matthew Wicker. Statistical guarantees for the robustness of bayesian neural networks. Conference on Artificial Intelligence, (IJCAI), 2019.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. Workshop on Artificial Intelligence and Security, 2017.
Nicholas Carlini and David Wagner. Towards evaluating the robustness neural networks. Sympo- sium Security and Privacy, 2017.
Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncertainty estimation without ood samples via density-based pscudo-counts. Advances in Neural Information Processing Systems, 2020.
Tarin Clanuwat, Mikel Bober- Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. Neural Information Processing Systems, 2018.
Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, 2019.
Raphaël Dang-Nhu, Gagandeep Singh, Pavol Bielik, and Martin Vechev. Adversarial attacks on probabilistic autoregressive forecasting models. International Conference on Machine Learning (ICML), 2020.
Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. Interna- tional Conference on Machine Learning, 2006.
Dheeru Dua and Casey Graff. UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences, 2017.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. Proceedings of Machine Learning Research, 2016.

---

Under review as a conference paper at ICLR 2021
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations, 2015.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10. Canadian Institute for Advanced Research, 2009.
Aounon Kumar, Alexander Levine, Soheil Feizi, and Tom Goldstein. Certifying Confidence via Randomized Smoothing. Advances in Neural Information Processing Systems, 2020.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems, 2017.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. National Institute of Standards and Technology. 2010.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Processing Systems, 2018.
Wesley J. Maddox, Pavel Izmailov, Timur Garipov, Dmitry P. Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. Advances in Neural Information Processing Systems, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Neural Information Processing Systems, 2018a.
Andrey Malinin and Mark Gales. Prior networks for detection of adversarial attacks. arXiv:1812.02555 [stat.ML], 2018b.
Andrey Malinin and Mark Gales. Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness. Advances in Neural Information Processing Systems, 2019.
Andrey Malinin, Bruno Mlodozeniec, and Mark Gales. Ensemble distribution distillation. Interna- tional Conference on Learning Representations, 2019.
Alexander Meinke and Matthias Hein. Towards neural networks that provably know when they don't 't know. International Conference on Learning Representations, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.
Kazuki Osawa, Siddharth Swaroop, Mohammad Emtiyaz E. Khan, Anirudh Jain, Runa Eschenhagen, Richard E. Turner, and Rio Yokota. Practical deep learning with bayesian principles. Advances in Neural Information Processing Systems, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vito,, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, nigh-performance deep learning library. Advances in Neural Information Processing Systems, 2019.
Yao Qin, Xuezhi Wang, Alex Beutel, and Ed H. Chi. Improving uncertainty estimates through the relationship with adversarial robustness. arXiv:2006.1637. 16375 [cs.LG], 2020.
Takaya Saito and Marc Rehmsmeier. The precision-recall plot more informative than the plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 2015.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. Advances in Neural Information Processing Systems, 2018.
10

---

Under review as a conference paper at ICLR 2021
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations, 2015.
Lewis Smith and Yarin Gal. Understanding measures of uncertainty for adversarial example detection.
Uncertainty in Artificial Interlligence, 2018.
Jasper Snoek, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan, Sebastian Nowozin, Sculley, Joshua Dillon, Jie Ren, and Zachary Nado. Can you trust your models uncertainty? Advances in Neural Information Processing Systems, 2019.
David Stutz, Matthias Hein, and Bernt Schiele. Confidence-calibratee adversarial training: Generaliz- ing unseen attacks. International Conference Machine Learning. 2020.
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations, 2014.
Natasa Tagasovska and David Lopez-Paz. Single-model uncertainties for deep learning. Advances in Neural Information Processing Systems, 2019.
Tsui- Wei Weng, Huan Zhang, Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. International Conference Learning Representations (CLR))2018,
Matthew Wicker, Luca Laurenti, Andrea Patane, and Marta Kwiatkowska. Probabilistic safety for bayesian neural networks. Proceedings of Machine Learning Research (PMLR), 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. Zalando SE, 2017.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 DIRICHLET-BASED UNCERTAINTY MODELS
In this section, we provide details on the losses used by each DBU model. PostNet uses a Bayesian loss which can be expressed as follows:
N Σ E,te,,,/CBB (0) H(g())
PesstNNet
(3)
where CF denotes the cross-entropy. Both the expectation term (i.e. F Eg(pt) (F y () and the
entropy term (i.e. H Hq(()) can be computed in closed-form (Charpentier et al. 2020). PriorNet uses loss composed of two KI. divergence terms for IDD and OOD) data:
NeriorNet
Σ
Σ an Dddaa
[KL[Di((a OOD Ill
(4)
N
( EID data
Both KL divergences terms be computed in closed-form (Malinin & Gales 2019). The precision ID and a QOD are aypra-parameter. The precision !D is usually set to le for the correct class and otherwise. The precision a OOD is usually to DDNet uses use the Dirichlet likelihood of soft labels produce by an ensemble of M neural networks:
M
LDDNet = N
Σ
Σ m==1
In ( (TTT)]
(5)
where im denotes the soft-label of mtth neural network. The Dirichlet likelihood can be computed in closed-form (Malinin et al., (019).. EvNet the expected mean square error between the one-hot encoded label and the predicted categorical distribution:
LEwNet = Ep (0) 1112
1 N
(6)
where (1) denotes the one-hot encoded label. The expected MSE loss also be computed in
closed form (Sensoy et al. 2018). For more details please have a look at the original paper on
PrionNet (Malinin & Gales 2018a), PostNet (Charpentier et al. 2020). DDNet (Malinin & Gales
2019) and EvNet (Sensoy et al. (018).
A.2 CLOSED-FORM COMPUTATION OF UNCERTAINTY MEASURES & UNCERTAINTY ATTACKS
Dirichlet-based uncertainty models allow to compute several uncertainty measures in closed form (see (Malinin & Gales] 2018a) for a derivation). As proposed by Maliiin & Gales (018a), we use precision mag differential entropy MAHHE and mutual information m to estimate uncertainty on predictions.
The differential entropy mair of DBU model reaches its maximum value for equally probable categorical distributions and thus, flat Dirichlet distribution. It is a measure for distributional uncertainty and expected to be low on ID data, but high on OOD data.
K
MaRIE InT(a)) (an((aa)-(n(((a))) In E(ae ())(()(()))) (aa))
where are the parameters of the Dirichlet-distributtinn I' is the Gamma function and Vi is the Digamma function.
The mutual information myi is the difference between the total uncertainty (entropy of the expected distribution) and the expected uncertainty on the data (expected entropy of the distribution). This uncertainty is expected to be low on ID data and high on OOD data.
K
MMI
at
In
ac
Va. + 1) (10 ((oo+1))
(8)
Onn
12

---

Under review as a conference paper at ICLR 2021
Furthermore, the precision ou to measure uncertainty, which is expected to be high ID data and low on OOD data.
K
Maa
0g
As these uncertainty measures are computed in closed form and it is possible to obtain their gradients, we use them (i.e. MAiEE, MMI, Ma.) are target function of our uncertainty attacks. Changing the attacked target function allows us a wide range of gradient-basee attacks such as FGSM attacks, PGD attacks, but also more sophisticated attacks such as Carlini- Wanner attacks.
A.3 DETAILS OF THE EXPERIMENTAL SETUP
Models. We trained all models with a similar based architecture. We used namely linear layers for vector data sets, 3 convolutional layers with size of 5 + 3 linear layers for MNIST and the VGG16[Simony & Zisserman (2015) architecture with batch normalization for CIFAR1O, 10. All the implementation are performed using Pytorch (Paszke et al. 2019).. We optimized all models using Adam optimizer. We performed early stopping by checking for loss improvement every 2 epochs and a patience of 10. The models were trained on GPUs (1 TB SSD).
We performed a grid-search for hyper-parameters for all models. The learning rate grid search was done in [lc le 31. For PostNet, we used Radial Flows with depth of 6 and a latent space equal to 6. Further, we performed grid search for the regularizing factor in [le_?, ?, le +]. For PriorNet, we performed grid search for the OOD loss weight in [1, 10). For DDNet, we distilled the knowledge
of 5 neural networks after a grid search in [2, 5, 10, 20] neural networks. Note that it already implied a significant overhead at training compare to other models.
Metrics. For all experiments. we focused on using AUC-PR scores since it is well suited to imbalance tasks (Satoo Rehmsmeier] (2015) while bringing theoretically similar information than AUC-ROC scores (Davis & Goadrich 2006). We scaled all scores from [0. 1] to [0. 100). All results are average over 5 training runs using the best hyper-parameters found after the grid search.
Data sets. For vector data sets, we use 5 diffeerent random splits to train all models. We split the data in training, validation and test sets (60%, 20%, 20%).
We use the segment vector data set Dua & Graff (2017), where the goal is to classify areas of images into classes (window, foliage, grass, brickface, path, cement, sky). We remove class window from ID training data to provide OOD training data to PriorNet. Further, We remove the class 'sky' from training and instead use it as the OOD data set for OOD detection experiments. Fach input is composed of 18 attributes describing the image area. The data set contains 2, 310 samples in total.
We further use the Sensorless Drive vector data set Dua & Graff (2017), where the goal is to classify extracted motor current measurements into 11 different classes. We remove class from ID training data to provide OOD training data to PrionNet. We remove classes 10 and 11 from training and use them as the OOD dataset for OOD detection experiments. Each input is composed of 49 attributes describing motor behaviour. The data set contains 58, 509 samples in total.
Additionally, we use the MNIST image data set LeCun & Cortes (2010) where the goal is to classify pictures of hand-drawn digits into 10 classes (from digit 0 to digit 9). Each input is composed of a 28 28 tensor. The data set contains 70, 000 samples. For OOD detection experiments, we use FashionMNISTT [iiao et al. (2017) and KMNIST Clannwatt et al. (2018) containing images of Japanese characters and images of clothes, respectively. MaST was used as training OOD for PrionNet while KMNIST is used as OOD at test time.
Finally, we use the CIFARIO image data set Krizhevsky et al. (2009) where the goal is to classify picture of objects into 10 classes (airplane, automobile, bird, cat, deer, dog, frog. horse, ship, truck). Each input is a 3 x 32 x 32 tensor. The data set contains 60, 000 samples. For OOD detection experiments,
use street view house numbers (SVHN) Netzer al. (011)) and CIFAR 100 (Krizhevsky et al.] (2009) containing images of numbers and objects respectively. CIFAR100 was used training OOD for PrionNet while SVHN is used as OOD at test time.
Perturbations. For all label and uncertainty attacks, we used Fast Gradient Sign Methods and Project Gradient Descent. We tried 6 different radii 0..0,0.1,.0.
0.1, 0.2, 0.5, 1.0, 2.0, 4.0). These radii operate on
13

---

Under review as a conference paper at ICLR 2021
the input space after data normalization. We bound perturbations by Lx -nom by Lg-norm, with
(x) max :=1....
aaa and (2(xx) xt)0..
(10)
For Lo norm it is obvious how to relate perturbation size with perturbed input images, because all inputs are standardized such that the values their features are between and 1. A perturbation of size e 0 corresponds the original input, while a perturbation of size corresponds the whole input space and allows to change all features to any value.
For T--norm the relation between perturbation size and perturbed input images is less obvious. To justify our choice for E w.r.t this norm, we relate perturbations size corresponding to L2-norm with perturbations size &oo corresponding -norm. First, we compute E2, such that the L2-norm is the smallest super-set of the - Let us consider perturbation of & The largest Ly-norm would be obtained each feature is perturbed by Ec. Thus, perturbation F2. such that L2 encloses Lee is E2 = - 2 = v DE For the MNIST-data set, with D = 28 x 28 input features L2-norm with E2 = 28 encloses L.w norm with &co 1.
Alternatively, E2 can be computes such that the volume spanned by L2-norm is equivalent to the one spanned by -norm. Using that the volume spanned by -nom is =2 and the volume g..52
spanned by L2-norm is r(0.5D+1) (where is the Gamma- Gamma-function) we obtain volume equivalence if 82 r00.5D 1) VT For the MNIST-data set, with D 28 x 28 input features -o-norm with E2 R 21.39 is volume equivalent to -orm with c 1.
14

---

Under review as a conference paper at ICLR 2021
A.4
ADDITIONAL EXPERIMENTS
Table 7 and X illustrate that no DBU model maintains high accuracy under gradient- -bsed label attacks. Accuracy under PGD attacks decreases than under FGSM attacks, since PGD is stronger. Interestingly Noise attacks achieve also good performances with increasing Noise standard deviation. Note that the attack is not constraint be with given radius for Noise attacks.
Table 7: Accuracy under PGD label attacks.
Rad.
0.1
2.0
CIFARIO
PouNerr
Priceher
99.3
99.1
98.8
97.4
93.9
75.3
88.2
77.8
68.4
54.0
57.9
17.5
5.1
DONer
99.4
90.1
98.8
97.5
91.6
48.8
0.2
86.
73.9
59.1
20.5
1.5
0.0
0.0
EvNer
40.9
Senselless
Segment
PostNit
PriceNet
99.3
16.5
1.2
0.4
99.5
47.6
0.0
0.4
DDNe
99.3
12.4
2.4
0.6
0.3
0.1
0.1
99.2
90.8
45.7
6.9
0.0
0.0
0.
EvNet
353
22.3
7.0
99.3
54.0
10.3
0.5
Table 8: Accuracy under FGSM label attacks.
AlL. Rod.
0.0
0.1
0.2
1.0
2.0
4.0
0.0
0.2
0.5
1.0
PostNet
99.4
99.2
98.9
97.7
95.2
90.1
79.2
89.5
72.3
54.9
31.2
21.0
163
15.6
PricuNerr
99.3
99.1
98.9
97.7
95.8
93.2
76.7
88.2
77.3
70.1
59.4
52.3
48.5
46.8
DINNr
99.4
97.8
94.7
75.0
6612
14.6
Sensorless
PostNet
98.3
10.9
10.9
11.9
12.5
98.9
79.6
18.4
20,6
PrionNer
99.3
24-7
11.8
3.6
8.5
8.1
8.1
99.5
85.5
40.5
8.9
0.4
0.3
0.2
DDNet
99.3
18.0
8.2
6.5
544
6.7
7.8
99.2
86.4
36.2
11.9
0.9
0.0
EvNet
Table 9: Accuracy under Noise label attacks.
Nesie Sud
0.0
0.1
0.2
0.5
1.0
2.0
4.0
0.0
0.1
0.2
0.5
1.0
2.0
4.0
MINIT
CIFARIO
PostNer
99.4
98.6
91.8
14.9
1,3
0.1
0.0
91.7
21.5
10.1
0.1
1.2
0.0
1.9
ProoNNet
95.7
14.4
0.0
0.0
87.7
28.1
9.0
ByNNet
99.3
96.9
81.6
11.7
0.3
0.0
0.0
89.5
2003
11.1
5.2
0.5
2.3
3.9
Senseeloss
Segment
PostNet
98.1
0.1
3.7
11.7
11.7
11.7
11.7
98.5
30.4
3.9
1.8
12.1
20,3
22.1
PriceNet
99.3
0.2
0.0
0.0
0.0
0.3
2.4
99.4
47.9
38
0.0
0.0
0.0
DDNN
EvNet
98.6
0.2
1.4
4.6
99.1
50.3
1.2
0.0
1.5
A.4.1 UNCERTAINTY ESTIMATION UNDER LABEL ATTACKS
Is high certainty reliable indicator of correct predictions?
On non-perturbed data uncertainty estimates are indicator of correctly classified samples, but if the input data is perturbed none of the DBU models maintains its high performance. Thus, uncertainty estimates are not robust indicator of correctly labeled inputs.
Table 10: Certainty based on differential entropy under PGD label attacks (AUC-PR).
Segment
Rod.
0.0
0.2
2.0
4.0
PowtNer
99.9
.99.9
09.8
98.7
88.5
43.5
9.0
99.9
77.6
31.6
11.1
5.3
4.4
87
Printter
160.0
15.2
DDNer
100.0
99.9
97.6
30.2
100.0
54.0
0.0
BvNer
99.6
09.3
98.7
96.1
88.8
63.1
31.7
100.0
95.9
44.3
599
088
0.6
07
Table 10 II and illluuttate that neither differential entropy nor precision, nor mutual information reliable indicator of correct predictions under PGD attacks. DBU-models achieve significantly
15

---

Under review as a conference paper at ICLR 2021
Table 11: Certainty based on precision dg under PGD label attacks (AUC-PR).
Art. Rad.
0.0
0.1
0.2
0.5
1.0
2.0
4.0
0.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIPAR10
PostNet
100.0
9939
99.7
96.2
87.9
39.1
6.9
98.77
53.6
56.2
7.8
0.4
0.3
PrionNer
37.6
24,9
97.6
18.1
EyNer
95.6
992
98.6
95.7
38.6
03.6
32.6
91.9
$3.9
57.2
10.2
4.0
2.4
0.3
Sensocless
Segment
PostNet
99.6
1.0
3.3
3.1
6.9
9.8
11.3
99.9
74.2
31.6
11.1
5.0
4,2
8.6
PriorNet
99.8
10.5
3.2
0.6
0.2
0.2
1.8
100.0
96.9
45.2
4.4
0.4
0.0
1.2
EvNet
99.9
23.2
132
6.0
3.7
100.0
95.7
44.5
0.6
0.7
Table 12: Certainty based on mutual information under PGD label attacks (AUC-PR).
Alt. Rod
0.0
0.1
0.2
0.5
1.0
2.0
4.0
0.0
0.1
0.2
0.5
1.0
2.0
40
MNIST
CIFARIO
PostNer
99.7
99.7
99.6
99.2
92.4
40.0
6.9
97.3
84.5
56.2
12.2
2.4
0.7
0.3
PriouNet
68.9
51.4
35.5
24.4
2.9
EvNer
97.8
97.0
95.7
92.6
S6.1
62.3
25.9
91.3
72.4
47.9
11.4
1.6
0.9
1.6
Senserless
Segment
PostNet
99.3
1.0
3.3
3.3
7.0
9.8
11.3
99.9
73.2
31.5
11.1
5.0
4,3
8.7
PrionNet
99.8
10.5
3.2
0.6
0.2
0.1
11.8
100.0
96.6
45.2
4.5
0.4
0.0
1.1
100.0
EvNet
99.1
22.0
12.6
59
3.7
100.0
90.5
41.0
0.6
0.7
Table 13: Certainty based on differential entropy under FGSM label attacks (AUC-PR).
Ast. Rad.
0.0
0.1
0.2
0.5
1.0
2.0
4.0
0.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIARRIO
RostNet
99.9
99.9
99.8
99.4
97.8
92.1
83.2
98.5
$8.7
68.9
31.0
18.6
13.5
16.7
PrionNer
90.1
46.1
38.5
37.3
3.1
EvNer
99.6
99.4
99.1
97.
95.8
90.4
76.8
95.0
96.2
67.4
32.7
19.9
18.2
19.7
Sensoriess
Segment
PostNer
99.7
11.7
7.3
9.3
11.8
12.5
12.5
99.9
73.6
40.6
23.7
17.2
19.8
20.2
PriorNet
21.4
10.4
8.5
9.0
10.3
100.0
93.7
37.7
1.1
0.9
0.8
100.0
EoNar
100.0
Table 14: Certainty based on differential entropy under Noise label attacks (AUC-PR).
Nosse Std
0.0
0.1
0.2
0.5
1.0
2.0
4.0
0.2
0.5
2.0
4.0
MNNITT
CIPARIO
PostNet
99.9
99.8
99.6
74,2
7.4
0.2
0.0
98.7
76.3
24.3
0.4
4.9
0.0
L.7
PriseNet
99.9
99.8
73.4
0.0
0.0
85.0
27.8
15.9
20.4
7.0
7.7
8.3
108.0
99.4
51.1
EvNet
99.3
98.4
33.5
20.2
0.9
0.0
97.5
66.1
21.4
7.2
2.3
3.8
Segment
PostNet
99.7
0.3
3,2
13.3
12.0
11.7
11.7
90.9
53.9
4.8
1.8
11.2
21.7
21.6
PrienNer
108.0
0.1
0.0
0.0
0.0
7.8
11.5
190.0
84.5
15.6
0.0
0.0
0.0
0.0
0.3
1.7
5.5
10.0
100.0
78.3
3.5
1.7
16

---

Under review as a conference paper at ICLR 2021
better results when they are attacked by FGSM-attacks (Table 13} but as FGSM attacks provide much weaker adversarial examples than PGD attacks, this cannot be seen as real advantage.
17

---

Under review as a conference paper at ICLR 2021
Can uncertainty estimates to detect attacks against the classification decision?
PGD attacks do not explicitly consider uncertainty during the computation of adversarial examples, but they seem to provide perturbed inputs with similar uncertainty the original input.
Table 15: Attack-Detection based on differential entropy under PGD label attacks (AUC-PR).
MNIST
Segment
Arr. Rad
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
100
2.0
4.0
PostNit
57.7
66.1
83.4
90.5
79.0
90.1
98.6
73.5
47.0
42.3
53.4
82.7
PrionNitt
67.7
83.2
97.1
96.7
92.1
82.9
86.7
83.3
38.0
31.3
30.8
31.5
EvNet
59.0
63.3
75.9
72.6
59.8
91.9
80.9
415
32.5
31.1
31.1
Table 16: Attack- based on precision ao under PGD label attacks (AUC-PR).
Att. Rad
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIFARIO
PostNer DDNet
92.6
95.1
63.4
66.9
42.1
31.6
52.7
55.7
64.7
78.4
91.9
50.9
55.8
50.5
57.3
38.7
32.3
31.4
EvNer
49.1
48.0
45.1
42.7
41.8
39.2
48.4
46.9
46.3
46.3
44.5
42.5
Semsorless
Segmeni
PostNer
30.8
35.8
35.4
52.0
18.2
99.0
94.6
70.3
46.3
42.6
54.9
$4.0
DDNe
40.3
35.3
33.8
80.0
86.0
43.3
31.0
3088
EyNet
45.4
39.7
36.1
34.8
34.7
36.0
90.9
72.4
80.4
32.4
31.1
31.1
Table 17: Attack-Detection based mutual information under PGD label attacks (AUC-PR).
Rad
2.0
M
CIFAR10
PostNict
PrizeNer
67.7
83.3
97.1
96.9
92.6
84.5
54.0
56.9
56.3
49.7
42.4
35.5
DONer
53.1
56.3
06.5
$1.0
94.0
82.9
56.0
60.8
57.4
38.2
52.1
31.3
ExNett
45.2
42.9
48.7
47.3
46.3
42.2
Sensorless
Segment
PostNet
73.5
32.0
82.7
308
DDNet 48.0
40.0
35.2
32.6
31.2
30.8
82.4
88.1
43.4
33.4
30.9
30.8
EvNett
45.5
39.7
36.1
34.8
34.7
36.0
91.7
72.9
40.5
32.4
31.1
31.1
FGSM and Noise attacks are easier to detect, but also weaker thand PGD attacks. This suggests that DBU models are capable of detecting weak attacks by using uncertainty estimation.
Table 18: Detect-ettcttin based on differential entropy under FGSM label attacks (AUC-PR).
At.. Rad
0.1
0.2
1.0
2.0
4.0
0.1
0.5
1.0
2.0
PostNer
55.0
61.8
74.8
84.0
88.9
89.9
62.1
67.2
65.7
63.1
65.4
73.8
PrindNrr
67.4
82.4
96.9
98.3
98.9
99.6
58.4
63.1
68.5
70.1
68.5
62.5
DUNe
68.3
62.9
76.5
57.88
Sensorless
Segment
PostNet
99.8
99.9
99.9
96.9
93.9
99.5
PriorNet
48.7
38.6
32.7
$2.9
38.6
44.3
89.0
80.8
46.7
37.2
33.7
32.4
DDNet
61.5
47.8
37.1
33.1
32.4
33.2
79.6
86.2
60.2
47.5
36.6
31.6
72.3
18

---

Under review as a conference paper at ICLR 2021
Table 19:
based on differential entropy under Noise label attacks (AUC-PR).
Noise
1.0
MNIST
CIRAR10
PowtNes
98.2
PriorNe
32.5
36.8
88.9
99.6
99.7
92.7
54.7
32.3
34.3
60.3
95.3
100.0
DINe:
60.7
87.6
99.8
100.0
99.9
99.8
90.1
62.6
81.5
98.6
99.8
98.7
66.9
78.6
Smmorles
Segment
PostNet
PriorNes
42.0
33.8
31.5
34.7
43.7
47.0
56.7
56.7
39.8
33.7
31.9
33.7
DLNNe
53.4
43.5
34.3
31.6
32.5
36.1
57.0
58.9
43.1
33.7
31.5
31.3
EvNet
78.8
8833
95.4
97.8
63.5
61.2
73.7
85.2
A.4.2 ATTACKING UNCERTAINTY ESTIMATION
Are uncertainty estimates robust feature for OOD detection?
Using uncertainty estimation to distinguish between ID and OOD data is not robust as shown in the following tables.
Table 20: OOD detection based differential entropy under PGD uncertainty attacks against differential entropy on ID data and OOD data (AUC-PR).
ID-Atiack (nom sltacked OOD)
OOD OOD-Atlac (nom-atlatiinn ID)
Alt.
0.1
1.0
2.0
0.5
PostNer
94.5
94.1
93.9
91.1
77.1
44.0
31.9
94.5
95.1
91.4
82.1
62.2
50.7
48.8
Prioateet
99.6
99.4
99.1
97.8
93.8
77.6
32,0
99.6
99.4
99.1
95.0
94.6
$5.1
73.9
DINer
97.8
Seg.
PostNet
99.0
80.7
53.5
38.0
34,0
41.6
49.5
99.0
88.4
69.2
45.1
36.4
42.6
75.4
PrionNet
34.8
31.4
30.9
30.8
30.8
30.8
30.8
34.8
31.8
31.0
30.8
30.8
30.8
32.1
DDNet
31.5
30.8
30.8
31.5
31,0
30.8
30.8
30.8
30.8
Table 21: OOD detection under PGD uncertainty attacks against differential entropy on ID data and OOD data (AUC-ROC).
ID-Anack (noo- (OD)
LDon-
Att. Rod.
0.1
2.0
4.0
0.0
0.5
MINST KMNIST
45.0
99.7
93.0
DDNe
99.2
98.9
98.6
97.3
92.1
58.2
1.2
99.2
99.0
98.5
97.9
95.8
89.1
69.3
EvNet
81.2
79.6
78.2
74.6
69.5
58.7
43.0
81.2
67.2
$4.8
35.4
25.5
20.7
18.5
CIFARIO SVHN
PostNer
56.3
30.2
71.0
$4.3
50.3
26.2
DDNet
$7.0
76.0
63.6
29.3
6.1
1.1
0.4
$7.0
78.1
66.1
26.2
5.1
0.7
0.1
EvNer
$8.0
69.1
51.7
24.6
15.5
9.5
4.2
88.0
72.0
661.7
47.9
42.1
33.3
24.0
Sens, Sens. class 10,
PostNer
85.3
49.1
38.1
7.8
8.2
8.2
8.2
85.3
57.2
54.0
27.3
31.5
86.7
99.5
DDNa
21.0
3.0
2.1
7.3
21.0
2.1
1.9
2.2
EvNNet
74.2
21.4
12.2
4.3
1.4
0.6
0.3
74.2
45.3
38.5
19.6
9.6
12.1
26.0
Seg.
Seg, class sky
PostNec
99.2
84.7
55.5
23.0
9.7
4.4
4.7
99.2
92.1
77.1
41.5
24.9
41.0
$0.8
PrionNer
4.4
0.1
17.1
1.5
DDNer
0.0
EvvNet
91.2
54.5
23.3
0.9
0.4
0.2
91.2
82.9
76.4
42.2
977
0.
0.6
19

---

Under review as a conference paper at ICLR 2021
Table 22: OOD detection (AU-PR) under PGD uncertainty attacks against precision do on ID data and OOD data.
OOD)
Alt. Rad
0.0
0.1
0.2
0.5
1.0
2.0
4,0
0.0
0.1
0.2
0.5
1.0
4,0
MNNST KMNIST
PostNer
34.3
DDNe
97.2
96.7
96.
91.8
86.4
53.3
31.0
97.3
96.7
96.3
94.5
91.1
82.9
64.6
EvNer
39.8
99.2
38.8
$7.9
57.1
36.3
35.4
99.8
34.5
32.5
31.2
31.0
30.9
31.0
CIFARIO SVIN
PostNet
82.4
22.3
82.4
61.8
41.5
19.8
17.5
25.9
DDNe
81.1
S8.4
36.0
16.7
13.4
81.1
71.2
99.9
27.8
16.5
15.5
15.4
LvVNet
34.7
27.4
25.4
22.0
19.7
18.1
17.1
34.7
19.4
18.1
17.1
16.8
16.2
15.7
Sens, Sens. class 10,
PostNer
77.4
39.6
35.9
51.7
44.4
44.4
44.4
77.4
40.3
38.6
29.5
34,0
79.4
97.4
26.8
27.7
DDNer
30.4
20.5
40.9
26.9
EvNet
66.3
33.3
29.7
27.0
27.1
29.2
33,9
66.3
39.3
37.1
31.3
28.3
28.4
29.7
Seg.
Seg, class sky
PostNet
98.4
14.8
51.0
37.2
32.8
43.5
49.9
98.4
81.7
66.1
42.4
34.8
40.9
71.2
PriouNer
30.8
303
DDNet
30.3
EvNet
98.3
83.0
60.5
34.0
31.0
30.8
30.8
98.3
94,4
83.8
65.6
37.0
31.1
30.9
Table 23: OOD detection (AUC-ROC) under PGD uncertainty attacks against precision dg on II) data and OOD data.
inom-attacked
Alt. Rod
0.0
0.1
0.2
0.5
1.0
2.0
4.0
0.0
0.1
0.2
0.5
100
2.0
4.0
MINIST KMNIST
PostNer
96.4
74.0
97.6
DONer
96.5
95.9
95.1
92.0
82.6
44.3
3.5
96.5
95.9
95.2
92.9
88.6
78.7
$9.4
EviNer
35.9
$4.1
32.8
30.1
27.4
34.6
21.4
35.9
18.7
10.4
3.7
2.0
2.0
CIFARIO SVHN
PostNet
87.4
71.2
54.8
29.2
19.0
14.0
9.4
87.4
71.4
54.1
30.1
25.8
17.5
5.8
32.2
DDNet
81.9
7338
61.8
30.2
9.3
3.0
0.8
$4.9
76.6
66.2
34.6
10.4
EvNet
6122
69.4
45.2
37.6
30.5
23.4
17.0
61.2
29.1
23.0
16.3
14.2
10.2
5.5
Sens, Sens. class 10,
PostNer
87.2
48.8
37.3
4.1
0.7
0.7
0.7
87.2
50.0
45.4
16.5
27.6
81.9
98.0
3.5
19.2
3.6
1.4
DDNer
FuNer
75.5
50.8
18.2
1.6
DU6
0.2
75.5
47.8
41.9
24.1
10.2
10.3
15A
Seg.
Seg, class sky
PostNer
98.6
77.7
50.8
28.3
8.2
1.3
0.5
98.6
88.9
73.4
36.2
19.4
36.7
752
PrionNet
8.5
1.3
0.0
0.1
0.0
E-Net
97.7
47.7
1.2
0.2
0.1
97.7
93.5
21.5
20

---

Under review as a conference paper at ICLR 2021
Table 24: OOD detection (AU-PR) under PGD uncertainty attacks against distributional uncertainty ID data and OOD data.
OOD)
Alt. Rad
0.0
0.1
0.2
0.5
1.0
2.0
4,0
0.0
0.1
0.2
0.5
1.0
4,0
MINST KMNIST
PostNer
66.6
65.4
80.5
72.1
30.9
DDNa
98.4
96.
97.7
95.8
89.5
56.3
30.9
98.4
98.1
97.8
96.5
93.8
86.1
67.7
EvNer
40.1
39.5
99.1
58.2
57.3
36.5
35.6
40.1
34.6
32.6
31.3
$1.0
31.0
31.1
CIFARIO SVIN
PostNet DDNe
64.2
51.1
28.5
19,3
64.2
16.4
16.5
1633
82.0
29.9
16.6
13.4
82.0
72.3
60.3
26.3
16.2
1544
15.4
EvVNtt
36.4
28.7
26.5
22.8
20.2
18.4
17.2
36.4
19.5
18.3
17.2
16.9
16.2
15.7
Sens, Sens. class 10,
PostNer
79.1
40.3
35.9
33.0
45.5
45.5
45.5
79.1
47.3
43.7
36.5
37.9
74.6
96.5
26.9
27.5
26.77
DDNer
28.4
EvNet
66.3
33.3
29.6
27.0
27.2
29.3
35.2
66.3
39.3
37.1
31.3
28.3
28.4
29.7
Seg.
Seg, class sky
PostNet
98.0
76.3
53.1
37.4
32.9
44.6
50.2
98.0
83.5
64.8
41.8
35.4
43.1
71.3
PricuNer
52.3
52.3
31.0
50.8
30.8
DDNet
EvNet
98.1
59.1
33.8
31.0
30.8
98.1
93.8
88.2
64.5
36.4
31.3
31.0
Table 25: OOD detection (AUC-ROC) under PGD uncertainty attacks against distributional uncer- tainty ID data and OOD data.
ID-Atack satceed
attackel
Alt. Rod
0.0
0.1
0.2
0.5
1.0
2.0
4.0
0.0
0.1
0.2
0.5
10
2.0
4.0
MNNST KMNIST
PostNer
82.2
84.5
DONer
98.1
97.7
97.2
94.8
$7.0
48.7
3.0
98.1
97.8
97.3
95.8
92.3
83.3
63.3
EviNer
36.8
35.0
33.7
30.9
28.2
25.3
22.1
36.8
19.3
10.7
3.9
2.1
2.2
CIFARIO SVHN
PostNet DDNet
82.9
67.7
59.1
51,3
47.7
24.2
82.9
$1,9
26.2
8.9
9.5
11.1
9.9
85.9
62.7
30.1
2.3
0.6
85.9
77.6
66.9
32.1
8.0
EvNet
63.3
$1.4
47.1
39.3
$2.1
24.9
17.9
63.3
31.1
24.4
17.7
15,0
10.7
577
Sens, Sens. class 10,
PostNer
87.1
50.9
37.8
5.5
4.5
.4.5
4.5
87.1
55.3
51.1
34.4
38.9
79.7
97.9
2.9
1.3
DDNer
FuNee
75.5
50.7
18.1
1.6
D.6
0.8
75.5
47.7
41.8
23.8
10.3
10.3
158
Seg,
Seg, class sky
PostNet
98.6
78.3
51.9
20.5
8.3
2.1
1.7
98.6
88.8
73.1
35.9
21.4
39,9
75.9
PrionNet
1.6
0.3
1.8
15.4
0.0
E-Net
0.2
0.1
97.4
92.9
86.1
20.4
21

---

Under review as a conference paper at ICLR 2021
Table 26: OOD detection (AU-PR) under FGSM uncertainty attacks against differential entropy on ID data and OOD data.
ID)
Adt. Rad.
0.0
0.2
0.5
1.0
4.0
0.0
0.1
0.2
1.0
2.0
4.0
MNIST
KMNIST
PostNer
87.6
DDNa
99.3
99.1
98.9
98.0
95.4
81.9
48.2
99.3
99.2
99.0
98.5
97.6
95.5
92.0
EvNer
69.0
67.4
66.2
64.0
61.9
99.8
$6.70
9.0
60.1
56.5
53.4
52.7
$2.9
53.5
CIFARIO SVIN
PostNet
81.8
66.2
65.7
61.3
43.4
46.5
77.0
PriurNet
DDNa
82.8
71.9
64.6
53.8
90.2
47.8
41.0
82.8
71.5
60.5
39.1
31.4
41.2
66.6
EvNet
80.3
67.8
64.0
61.9
61.6
57.4
49.6
$0.3
59.2
51.5
46.7
49.0
56.3
64.6
Sens, Sens. class 10,
PostNer
74.5
40.5
57.2
31.4
38.1
44.9
45.9
74.5
99.6
99.8
99.9
99.9
99.9
99.9
PrionNar
27.6
29.1
ExNet
66.5
45.7
46.8
42.3
42.0
41.4
41.8
66.5
$4.7
66.5
76.2
71.1
75.3
75.8
Seg.
Seg, class sky
PostNet
99.0
80.8
66.4
43.6
37.0
35.5
43.0
99.0
94.8
92.0
98.5
99.7
100.0
100,0
PrionNet
34.8
88.8
94.5
34.8
31.0
31.2
56.2
30.8
EvNet
92.5
64.9
$4.6
66.6
69.5
69.6
64.6
92.5
85.9
83.0
65.3
66.1
61.1
56.8
Table 27: OOD detection (AU-PR) under Noise uncertainty attacks against differential entropy on ID data and OOD data.
(nom-oitacked
(non-attacked
Noise Sid
0.1
0.2
0.5
10
2.0
4.0
0.0
0.1
0.2
0.5
1.0
2.0
4.0
MINIST KMNIST
95.0
34.4
31.6
92.2
93.5
PrionNer
100.0
72.4
DDNer
99.1
97.5
81.2
51.3
51.0
30.9
31.2
99.0
98.8
99.2
99.8
99.9
99.8
99.1
ExNee
65.5
60.5
51.4
35.3
34.5
35.5
35.0
62,5
47.2
40.9
35.1
34.6
33.5
34.9
CIFAR10- SVHN
PostNet
88.5
41.4
39.8
31.0
30.7
31.6
33.9
88.5
86.6
81.9
93.0
98.5
98.6
97.3
PrionNet
73.3
31.6
DDNe
87.3
69.3
78.4
55.2
31.6
30.7
31.4
87.3
55.8
57.9
73.9
97.3
99.5
97.2
EvNet
92.4
56.8
53.8
33.4
30.9
32.9
36.6
92.4
73.7
73.5
77.7
90.7
92.5
92.1
Sens. Sens, class 10,
PosNNer
85.3
30.8
39.4
50.0
50.0
50.0
50.0
85,3
98.9
100.0
100.0
160.0
100.0
100,0
PrinnNer
32.3
77.7
47.6
32.4
FuNar
80.3
30.
31.2
37.9
46.
50.0
50.0
80.3
34.5
38.4
$3.9
60.3
78.8
81.5
Sey.
Seg. class skay
PosNNet
90.9
41.8
30.8
34.5
49.1
50.0
50.0
99.9
97.4
96.6
99.5
100.0
100.0
100.0
PriorNet
31.0
30.8
30.8
32.77
69.0
31.0
30.8
30.8
30.8
32.4
30.8
EcNat
99.1
38.1
30.8
30.8
33.2
99.1
95.6
87.6
46.6
53.8
22

---

Under review as a conference paper at ICLR 2021
A.5
ROBUST TRAINING FOR DBU MODELS & ID/OOD VERIFICATION
Table 5 and 29 on adversarial training illustrate that there is a jump between ID-verification and OOD-verification, where robustness ID data drops while robustness OOD data increases. These jumps are observed for each model and each training (normal, noise-based, adversarial with label attacks, adversarial with uncertainty attacks). Thus, either IDverificatiinn OOD-verification perform well, depending on the chosen threshold.
In contrast that, adversarial training improves robustness w.r.t. the predicted class label for most pair model/data set (Fig. 7232
Table 28: Randomized smoothing verification of CIFARIO (ID data) and SVHN (OOD data) harmonic mean.
adv,
Nome
PriorNer 26.7
PostNet
35.9
34.1
0.0
DDNet
45.2
18.1
46.6
train, loss: crosseatrogy
PrionNer
PoaNNer
34.4
47.9
0.0
DINer
49.2
44.3
0.0
22.4
adv. train, loss: diffE
PrionNet
PostNer
41.9
11.4
0.0
DINet
46.2
8.4
0.0
EvNet
47.3
346
2.0
Table 29: Randomized smoothing verification of MNIST (ID data) and KMNIST (OOD data):
percentage of samples that is certifiably correct (cc) and mean certified radius (R).
I1 Verifeataiion
OUD Vesticatioon
0,1
0.2
11.5
0.1
0.2
0.5
cc
R
R
CC
CC
adv. train. loss: Nome
PriorNet
97.0
0.36
88.2
0.52
3.0
0.20
98.7
0.37
99.5
0.74
100.0
1.38
PostNet
93.2
0.32
68.1
0.31
0.8
0.11
98.4
0.36
99.5
0.68
100.0
1.55
EyNet
95.0
0.31
83.0
0.30
17.3
0.21
77.3
0.17
82.7
0.24
$3.6
0.39
erosscntrogy
PriorNet
97.0
0.36
94.3
0.58
1.0
0.15
99.8
0.38
99.5
0.74
100.0
1.89
PostNer
94.4
0.51
57.7
0.12
3.2
0.13
97.2
0.33
95.6
0.51
99.6
1.02.
0.75
70.1
0.27
85.2
0.24
PriorNet
98.0
0.37
83.4
0.49
99.7
0.38
100.0
0.76
100.0
1.90
PostNer
93.5
01.33
47.1
0:23
0.6
0.15
95.8
0.34
98.8
0.63
100.0
1.38
DDNet
93.6
0.36
52.7
0.43
97.7
0.37
99.7
0.75
100.0
99.1
23

---

Under review as a conference paper at ICLR 2021
Table 30: Randomized smoothing verification of MNIST (ID) data) and KMNIST (OOD data):
percentage of samples that is certifiably wrong (cw) and mean certified radius (R).
0.1
0.2
0.5
ew
DW
loss:
PriarNer
2.8
0.16
10.7
0.21
96.0
0.97
PostNet
64
0.17
78.8
0.22
99.0
1.15
DDN
9.1
0.23
46.3
crussentropy
PriorNet
2.9
0.20
0.24
98.4
1.05
PosiNet
5.3
0.17
38.8
0.23
95.2
0.93
DDNet
43.5
0.41
adv. loss: diff
PriotNer
2.0
15.7
98.8
PouNer
6.3
01.17
49.8
0.25
99.1
1.10
DENNr
6.2
00.22
46.2
0.42
100.0
1.81
Table 31: Randomized smoothing verification of MNIST (ID data) and KMNIST (OOD data)
harmonic mean.
0.1
0.2
0.5
ads. train, loss: Nome
PrioxNer
5.5
19.1
5.9
PostNer
12.
40.5
EvNer
877
25.6
25.4
adv. train, losa: crssentrogy
PriorNet
5.6
9.3
2.0
PostNet
10.0
46.4
6.2
EvNet
5.8
38.2
30,2
PriorNer
3.9
25.4
14
PostNer
11.8
48.4
12
11.6
49.2
Table 32: Adversarial training with CE: Accuracy under PGD label attacks (AUC-PR).
Alt. Rad
0.1
1.0
2.0
4.0
10
20
PostNer
99.1
98.7
96.7
59.3
62.4
148
72.1
50.5
12.8
2.4
0.2
0.1
PriatNet
99.1
98.3
97.6
94.8
91.1
79.5
69.5
63.9
55.8
46.4
30,9
11.1
DDNet
98.9
97.4
90.6
0.2
74.7
63.9
23.6
2.0
0.0
96.7
Sensorless.
Segmient
PostNee
PriomNet
31.3
15.8
0.2
0.0
0.3
5.3
94.0
65.2
19.1
0.6
0.0
0.0
DDNe
12.4
4.2
0.2
0.3
0.2
0.1
91.4
46.2
74
0.2
0.0
19.4
2.6
Table 33: Adversarial training with CE: Accuracy under FGSM label attacks (AUC-PR).
MNIST
CIFARID
95.3
71.0
PricnNer 99.1
93.9
97.9
96.1
94.7
90.0
69.7
65.3
SKN
52.1
38.9
22.2
DONer
99.1
98.9
97.8
94.6
79.1
27.9
73.7
64.7
34.8
15.5
8.0
4.
8033
48.0
17.3
Sensorless
Segment
PricNNet
20.8
91.2
0.7
DONet
16.4
9.7
7.0
4.6
6.4
7.5
85.5
36.4
10.5
0.8
0.0
0.0
EvNet
41.1
27.4
20.1
15.2
14.9
12.6
93.8
64.2
25.0
1.5
24

---

Under review as a conference paper at ICLR 2021
Table 34: Adversarial training with CE: Accuracy under Noise label attacks (AUC-PR).
0.5
0.2
4.0
MNIST
CIFAR10
Proehet
933
94.6
32.7
0.0
0.0
39.1
16.8
3.8
10.5
3.7
0.4
DONer
97.6
89.1
16.5
21
0.0
0.3
25.8
15.1
2.2
0.2
9.8
8.9
Evhet
7.1
Sensrless
Segmint
PriceNet
0.0
0.0
0.1
$6.4
2.1
0.0
0.0
DDNet 0.3
0.0
0.0
0.0
0.0
0.1
29.4
0.1
0.0
0.0
0.0
0.0
EvNet
0.9
0.2
3.5
512
10.6
0.3
0.0
0.6
Table 35: Randomized smoothing verification of CIFARIO: percentage of samples that certifiably
correct (cc) w.r.t. the predicted class label and mean certified radius (R) w.r.t. class labels.
0.1
0.2
0.5
22
CC
X
adv. train.
lusac Nome
PriorNet
42.8
0.25
21.2
0.42
11.8
1.30
PosiNet
35.0
0.22
12.3
0.51
9.4
0,12
10.8
1,91
RoNer
11.0
loss: crossentropy
PiorNet
56.2
0.25
25.4
0.48
13.0
0.35
PostNet
34.7
0.22
15.6
0.45
11.0
0.32
DDNet
41.7
1,30
adv,
ditfe:
PriorNet
48.1
0.23
28.0
0.40
0.22
PooNet
45.5
0.21
18.0
0.36
5.4
0.18
DDNet
49.2
0.25
26.3
04
9.6
0.27
Table 36: Randomized smoothing verification of MNIST: percentage of samples that certifiably
correct (cc) w.r.t. the predicted class label and mean certified radius (R) w.r.t. class labels.
0.2
0.5
CV
R
R
R
adv, train. loss: Nome
PriorNer
99.2
0.38
98.,
0.71
61.4
0.45
PostNer
99.2
996.
51.2
99.3
EvNer
98.9
0.37
96.2
0.56
$7.1
0.42
mdv, train, losa: crussenrropy
PriocNet
99.1
0.38
99.0
0.72
50.4
0.53
PostNet
99,4
0.38
97.4
0.62
28.5
0.51
99.3
EvNet
99.1
0.37
92.1
0.43
35.0
0.40
PriorNet
99.5
0.38
98.3
0.71
64.0
0.48
PoweNee
99.1
0..8
96.8
0.62
48,
0.44
DINNr
99.6
99.1
32.4
EvNNtt
99.1
96.7
89.5
0.93
Table 37: Adversarial training with CE: Certainty based on differential entropy under PGD label attacks (AUC-PR).
Rad
MNJST
CIFARIO
PriouNer
59.8
19.4
91.7
92.4
79.7
67.5
54.0
45.5
37.9
29.7
18.1
DUNet
100.0
99.9
99.7
96.9
46.1
0.1
92.1
83.1
24.8
1.1
0.0
0.0
ExNet
81.2
98.4
95.5
90.1
53.2
38.3
62.9
59.2
13.1
1.5
1.6
Semsrrless
Segment
PostNer
3588
DDNa
10.9
3.0
0.1
0.2
0.1
0.1
95.9
52.6
4.5
0.5
0.0
0.0
EoNer
21.4
11.0
4.4
3.1
1.7
1.4
94.8
42.7
8.9
1.3
1.3
25

---

Under review as a conference paper at ICLR 2021
Table 38: Adversarial training with CE: Certainty based on differential entropy under FGSM label attacks (AUC-PR).
0.5
0.2
4.0
MNIST
CIFARIO
PrionNet
99.8
99.5
98.2
94.6
39.3
83.5
54.7
48.1
42.3
36.8
27.3
19.8
DDNer
100.0
99.9
99.7
98.7
87.3
27.4
91.3
83.4
45.4
15.0
6.5
3.6
bvNet
28.6
15.1
20.5
Sansorless
Segment
12.5
PrionNet
30.0
22.2
0.7
0.5
96.1
13.5
0.0
DDNet
13.5
6.7
6.2
3.1
4.8
5.5
92.0
42.0
6.4
2.3
0.0
0.0
EvNett
416
25,1
189
132
14.6
13,9
93.1
55.1
15.5
0.0
1.3
Table 39: Adversarial training with CE: Certainty based differential entropy under Noise label attacks (AUC-PR).
Art. Rod.
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MINIST
CIFARIO
PosNNer
100.0
98.8
54.7
12.1
0.0
0.0
$2.1
22.7
6.6
0.0
9.8
9.0
PrionNeet
100.0
99,9
$8.5
0.0
0.0
34,9
11.2
7.8
10.0
0.4
8.6
EvNet
93.4
36.9
13.3
0.0
54.3
17.6
0.1
10.5
Sensorless
Segmtes
PostNet
0.6
5.1
12.2
11.7
11.7
11.7
36.7
2.0
3.6
17.2
20.8
21.3
PrintNer
8.5
0.0
0.0
0.2
0.0
2,0
90.5
32.
7.1
1.
0.0
0.0
DINNr
1.5
Table 40: Adversarial training with CE: Attack-Detectinn based on differential entropy under PGD label attacks (AUC-PR).
Alt. Rad.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIPAR10
PostNer DDNer
$4.1
62.5
66.7
36.6
544
57.4
60.9
96.4
96.1
86.3
56.7
62.4
608
39.3
32.9
$1
EvNer
52.9
59.7
07.7
71.9
66.5
58.5
52.4
59.0
48.9
41.7
40.5
40.7
Semsorless
Segment
PostNiet
43.7
41.1
38.4
53.0
83.5
98.7
96.2
73.5
47.7
42.7
56.8
70.7
30.8
DDNe
34.7
76.6
83.0
45.7
30.8
30.8
EvNet
48.3
42.1
37.7
36.6
39.3
48.5
95.9
79.5
43.3
33.4
31.3
31.2
Table 41: Adversarial training with CE: Attack-Detection based on differential entropy under FGSM label attacks (AUC-PR).
CIFARIO
PostNet
75.5
86.3
92.6
68.9
66.1
77.5
PriurNet
68.6
80.6
96.8
98.0
98.4
98.2
55.9
59.0
63.0
65.4
63.4
53.4
DDNet
56.4
57.4
69.5
$1.0
95.5
99.0
57.6
63.7
70.3
69.0
73.4
76.1
Sonsoeless
PostNer
100.0
PriorNet
78.6
68.1
37.6
32.0
30.7
49.1
68.5
60.0
42.9
31.4
30.7
32.3
DINer
60.9
55.5
41.0
34.6
31.7
32.7
61.3
51.7
39.0
33.8
31.5
32.5
67.5
76.4
Table 42: Adversarial training with CE: Attack-Detection based differential entropy under Noise label attacks (AUC-PR).
AlL. Rad.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4,0
MNIST
CIFAKII
PostNet
59.0
66.4
97.2
95.7
95.9
99.6
90.5
89.2
95.2
99.5
85.7
99.7
PrioNNet
33.8
61.3
100.0
52.2
5022
54.4
DUNNet
100.0
EvNet
51.7
58.5
85.3
84.9
66.3
100.0
46.9
68.1
93.4
94.9
71.4
77.8
Sensodless
Seyment
PostNot
99.9
100.0
100.0
100.0
99.9
100.0
93.2
99.3
99.9
100.0
100.0
100.0
PriorNet
56.7
42.9
32.0
30.8
30.7
68.5
60.0
42.9
31.4
30.7
32.3
EvNet
69.2
38.5
71.7
52.3
70.9
77.6
89.5
70.0
66.9
62.4
77.2
76.4
26

---

Under review as a conference paper at ICLR 2021
Table 43: Adversarial training with CE: OOD detection based on differential entropy under PGD uncertainty attacks against differential entropy ID data and OOD data (AUC-PR).
Rad.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.5
1.0
2.0
4.0
MNNST KMNIST
PostNer
46.4
41.2
99.3
DDNa
99.0
98.9
97.8
91.7
58.6
30.7
99.1
90.0
98.4
96.2
90.8
78.7
EvNer
71.3
66.9
60.6
64.4
50.4
42.7
66.3
47.8
37.4
46.7
57.3
33.3
CIFARIO SVIN
PostNet
45.6
21.0
17.7
63.8
41.1
17.0
16.1
DDNet
63.5
34.0
16.8
13.4
72.7
64.8
28.3
15.4
15.4
EvNet
33.9
43.7
24.2
16.6
16.1
15.5
35.8
34.7
29.6
21.5
22.0
22.5
Sens, Sens. elass 10,
PostNNr
40.5
37.3
43.8
46.7
47.3
45.8
42.6
41.7
31.7
38.5
$1.9
99.3
PrionNer
76.6
26.5
27.7
26.5
DDNer
26.6
26.6
26.6
26.7
EvNet
31.8
29.7
27.2
28.0
32.8
37,8
36.5
38.1
27.8
27.4
30.0
38.3
Seg.
Seg, class sky
PostNet
61.2
90.8
53.3
32.7
45.3
49.2
79.9
61.6
62.7
32.6
46.0
66.7
PrixeNet
30.8
$1.4
30.3
50.8
303
DUNet
EvNet
67.0
34.9
30.9
30.8
30.8
31.6
75.5
52.1
31.2
31.2
30.8
30.8
Table 44: Adversarial training with CE: OOD detection based on differential entropy under FGSM uncertainty attacks against differential entropy III data and OOD data (AUC-PR).
(non-altacked
OOD-Allack (tun-allacked ID)
All. Rad.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
10
2.0
4.0
MNIST KMNIST
PostNer
95.0
73.9
DINer
99.0
99.0
98.1
94.6
90.3
47.3
99.1
99.1
98.6
97.3
95.3
92.4
ExNet
71.4
67.7
62.6
68.6
56.9
$3.0
68.3
57.1
50.7
62.6
50.7
46.4
CIFARIO-SVIT
PostNet
67.4
61.4
62.9
70.77
65.0
64.1
65.6
51.6
39.0
45.1
62.7
77.6
PrionNet
16.9
22.3
DDNet
71.0
66.9
56.1
55.7
48.7
44.7
72,2
66.4
48.0
42.2
48.7
69L1
EvNet
$6.5
62.2
$1.6
53.9
61.4
46.6
55.9
42.1
35.0
37.4
56.2
68.7
Sens, Sens, class 10,
PostNer
41.2
37.9
34.1
41.4
45.7
45.7
99.2
99.8
99.8
100.0
99.9
100.0
PrinnNir
28.0
28.8
26.5
26.5
DINN
27.5
FsNer
44.3
47.2
47.7
46.3
38.8
40.0
$1.6
69.3
50.4
48.5
65.4
72.3
Seg,
Seg, class sky
PosNNe
61.9
54.3
57.4
34.0
37.5
43.1
92.9
92.2
91.1
99.9
99.9
100.0
PriorNet
30.8
30.8
30.8
36.9
31.2
30.8
30.8
30.8
30.8
ExNet
66.7
42.3
61.5
50.1
74.6
60.0
63.2
27

---

Under review as a conference paper at ICLR 2021
Table 45: Adversarial training with CE: OOD detection based on differential entropy under Noise uncertainty attacks against differential entropy on ID data and OOD data (AUC-PR).
ID Attack (son attacied OOD)
Ariack (non attacked ID)
At.. Rad.
0.1
0.2
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
KMNIST
PostNet
91.7
12.7
32.4
36.2
33.2
30.7
93.0
85.5
$9.9
90.5
72.1
99.1
PriorNet
99.7
98.4
71.5
36.3
37.3
38.7
99.2
96.8
95.8
99.3
100.0
77.4
100.0
EvNet
87.4
49.9
32,8
32.0
33.2
36.3
87.4
46.8
45.6
45.1
33.3
100.0
SVHN
PostNet
43.9
31.9
30.7
30.7
56.3
$1.5
85.1
84.8
85.2
97.0
$2.2
95.5
PriorNes
31.4
32.8
85.8
37.1
30.7
50.8
35.2
40.1
50.8
42.1
99.0
100,0
DDNe
42.8
82.3
63.9
PostNet
30.8
47.8
50.0
50.0
50.0
50.0
98.7
99.8
100.0
100.0
99.8
100.0
PriorNet
30.7
30.7
30,9
87.7
100.0
100.0
31.0
30.7
32.4
31.4
34.2
30.7
DDNet
30.7
30.7
47.4
75.0
79.7
30.7
30.7
30.7
34.3
29.6
Seg, class sky
PourNet
42.6
93.7
100.0
PriorNes
30.8
30.8
30.8
30.8
30.9
100.0
30.9
30.8
50.8
30.9
31.2
32.9
DDNe
30.8
30.8
30.8
31.3
77.8
93.3
30.8
30.8
30.8
30.8
30.8
32.3
Table 46: Adversarial training with Diff. Ent.: Accuracy based on differential entropy under PGD label attacks (AUC-PR).
Alr. Rad
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIFARID
99.1
98.8
65.9
0.4
Priehhit
9.6
DONer
99.1
98.9
97.4
91.9
45.7
0.3
78.9
63.1
22.0
0.0
0.0
EvNet
98.3
98.1
95.2
91.0
72.7
40.1
65.6
48.9
14.8
8.4
3.8
1.8
Sensorless
Segment
PostNet
16.1
7.4
5.8
7.5
12.5
81.7
47.1
22.3
10.8
38
PriceNet
33.3
0.0
EvNet
36.1
22.1
10.8
3.8
1.7
3.1
92.0
56.2
11.9
2.1
0.1
2.8
Table 47: Adversarial training with Diff. Ent.: Accuracy based differential entropy under FGSM label attacks (AUC-PR).
Au. Rad
0.1
0.2
1.0
2.0
4.0
0.1
1.0
2.0
MNIST
CIFAR10
PostNer
99.1
98.9
97.6
95.0
91.2
50.0
23.5
11.2
0.2
0.1
9.0
1.6
PriceNer
99.1
98.8
97.9
95.9
93.5
87.4
31.6
14.4
6.2
7.8
0.2
1.4
DDN
10,0
98.3
95.7
90
Segment
PostNer
21.4
10.6
10.3
12.3
12.4
12.5
83.9
53.7
27.8
15.4
20.1
19.1
PristNet
40.3
21.3
8.1
0.0
0.0
2.3
91.3
57.2
18.1
3.2
0.0
0.0
DDNet
17.6
5.0
8.0
7.0
5.7
85.1
39.2
12.1
0.4
0.0
Table 48: Adversarial training with Diff. Ent.: Accuracy based on differential entropy under Noise label attacks (AUC-PR).
Red.
0.1
2.0
4.0
MNIST
CIHAR10
PricnNiet
98.1
83.5
24.5
0.0
0.0
67.2
65.2
58.9
48.5
40.5
31.4
DUNet
97.3
92.5
6.4
2.2
0.1
0.1
78.1
63.8
34,2
16.0
8.0
6.0
ExNet
96.2
87.0
2.3
0.0
0.0
65.2
51.8
29.2
17.4
162
Sansorless
Segiteet
PostNer
1.0
20.9
0.0
0.9
DDNer
0.3
0.0
0.0
0.0
0.1
49.4
6.1
0.0
0.0
0.0
EoNer
0.9
0.0
0.2
3.5
3.1
51.2
10.6
0.3
0.0
0.
06
28

---

Under review as a conference paper at ICLR 2021
Table 49: Adversarial training with Diff. Ent.: Certainty based on differential entropy under PGD label attacks (AUC-PR).
0.5
0.2
0.5
4.0
MNIST
CIFARIO
PrionNit
59.7
99.6
98.0
91.4
76.2
54.8
51.0
44.7
36.3
23.7
13.8
5.5
DDNer
100.9
99.9
99.7
97.6
47.9
94.7
82.4
21.2
1.1
0.0
0.0
bvNet
60.8
22
Sensorless
Segment
PrionNet
22.6
10.3
388
0.0
97.9
11.4
0.0
DDNet
13.2
2.2
0.4
0.3
0.1
0.2
95.8
51.1
5.0
0.3
0.0
0.0
EvNett
22.6
12.8
59
2.0
2.9
94.5
44.3
7.0
2.0
Table 50: Adversarial training with Diff. Ent.: Certainty based differential entropy under FGSM label attacks (AUC-PR).
Art. Rod.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIFARIO
PostNer
90.9
99.9
90.4
97.1
92.1
82.2
88.3
67.8
50.9
19.2
15.0
16.7
Prionbiet
99.7
99.6
95.1
93.8
$7.0
79.5
52.9
44.1
36,3
28.6
20,9
15.4
EvNett
99.2
99.1
97.8
95.3
$7.8
67.0
80.2
61.1
32.5
21.9
21.6
22.2
Segmest
PostNet
13.8
7.2
9.7
12.3
12.4
12.5
74.4
37.9
2012
16.7
19.5
19.9
PrintNer
32.3
19.1
9.1
0.0
0.0
5.2
96.2
56.6
11.4
10.7
0.0
0.0
DINNr
Table 51: Adversarial training with Diff. Ent.: Certainty based on differential entropy under Noise label attacks (AUC-PR).
Alt. Rad.
0.1
0,2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIPAR10
PostNer
64.5
77.4
14.6
0.1
DDNer
99.7
99.4
30.5
5.9
0.6
0.3
86.1
46.5
155
17.4
10.2
10.1
EvNer
99.2
97.0
19.4
0.1
0.0
0.0
67.1
12.7
4.5
12.8
13.2
3.3
Semsorless
Segment
PostNiet
0.6
5.1
12.2
11.7
11.7
36.7
3.6
17.2
20,8
21,3
DDNe
1.5
0.0
0.0
79.6
21.8
0.0
EvNet
1.5
0.0
1.0
0.2
4.9
4.8
75.7
22.0
3.2
0.0
0.0
0.7
Table 52: Adversarial training with Diff. Ent.: Attack -ittcc-eetection based on differential entropy under PGD label attacks (AUC-PR).
MNIST
CIFARIO
PostNer
93.6
47.7
67.1
41.7
36.5
PriceNet
66.9
76.0
95.1
96.4
55.7
74.8
55.7
55.8
53.1
48.9
43.3
37.8
DDNe
53.7
58.5
69.3
85.5
96.1
87.7
56.7
62.5
60.4
412
32.6
31.8
60.4
60.2
39.4
Sensorless.
Segmient
PostNer
PrioeNet
50.4
39.4
31.6
30.8
30.7
30.7
86.1
89.7
50.8
37.5
52.9
303
DDNer
52.2
41.5
35.5
52.3
31.5
35.9
77.9
87.3
43.4
32.4
30.9
303
388
Table 53: Adversarial training with Diff. Ent.: FGSM label attacks (AUC-PR).
based differential entropy under
Aut. Rad.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MNIST
CIFAX1O
PostNer
56.2
61.6
78.0
89.5
92.1
90.7
63.8
68.8
65.8
62.9
67.0
73.6
PrioeNer
67.3
98.0
67.3
67.3
538
72.8
FvNer
53.7
57.2
50.8
65.2
71.5
72.4
56.2
62.4
58.9
90.0
63.2
70.5
Sensrriess
Seymens:
PostNet
98.3
99.8
99.9
99.9
99.9
100.0
96.8
93.3
93.1
97.4
99.8
99.6
PrionNec
67.0
56.0
35.8
30.8
30.7
30.7
85.6
89.3
55.1
45.2
37.6
EvVeet
68.7
69.3
89.9
68.6
08.8
73.2
95.9
86.6
66.2
63.2
74.2
77.2
29

---

Under review as a conference paper at ICLR 2021
Table 54: Adversarial training with Diff. Ent.: Attack-Detection based on differential entropy under Noise label attacks (AUC-PR).
All. Rad.
0.1
1.0
2.0
4.0
0.1
1.0
2.0
PowtNet
53.7
67.7
94.5
94.1
96.1
100.0
85.8
79.1
93.0
90L.
96.3
96.5
ProeNee
33.1
42.4
80.5
95.8
73.3
56.0
52.3
38.2
5188
38.1
87.4
99.9
90,9
100.0
100.0
90.9
98.3
EvNet
97.3
Senselles
PostNet
99,9
100.0
100.0
100.0
99.9
100.0
93.2
99.3
99.9
100.0
100.0
100.0
PriorNet
56.7
12.9
32.0
30.8
31.0
30.7
55.9
63.1
41.7
33.3
20.8
30.8
DDNei
31.2
43.9
33.8
32.4
31.7
35.0
55.9
58.9
42.1
33.4
31.1
32.7
Table 55: Adversarial training with Diff. Ent.: OOD detection based on differential entropy under PGD uncertainty attacks against differential entropy on ID data and OOD data (AUC-PR).
ID-Altick (non-attacc OOD)
OOD-Attekk (non-attisckd
Rad
MINSS KMNIST
PostNict
91.9
PrizdNer
99.6
99.3
97.6
93.7
$1.4
50.8
99.6
99.2
97.4
92.1
66.3
37.6
DDNer
99.1
98.9
97.6
93.6
6UX77
30.7
99.1
99.0
98.3
97.0
91.4
77.5
72.9
33.4
3644
CIFARIO SVIN
PostNee
168
PriceNet
17.3
15.9
17.4
15.5
15.4
15:4
17.1
15.7
16.6
15:4
15.4
15.4
DDNe
77.5
66.0
34,5
16.4
15.4
15.4
79.6
67.4
33.2
16.9
15.4
15.1
EvNet
57.8
35.2
22.0
21.5
17.5
16.0
52.7
30.7
30.3
31.1
20,8
1800
Sens. Sens. class 10,
PostNer
34.8
41.8
41.1
40.6
99.5
26.5
DDNer
26.8
26.5
26.5
26.6
26.6
28.0
26.8
26.6
26.7
26.7
26.6
26.6
EvNet
31.0
29.4
27.22
29.1
32.4
36.5
39.1
35.1
28.9
28.7
30.0
38.3
Seg. SSe. class sky
PostNiet
45.3
44.6
38.8
49.4
98.7
67.3
44.1
47.7
37.5
59.4
DDNe
30.8
30.8
30.8
31.2
30.8
30.8
30.8
30.8
EvNet
$8.0
39.4
31.0
30.8
30.8
31,5
84.1
71.7
36.1
31.1
30.8
30.8
Table 56: Adversarial training with Diff. Ent.: OOD detection based on differential entropy under FGSM uncertainty attacks against differential entropy on ID data and OOD data (AUC-PR).
iDAtttick (non-attuckel OOD)
OOD+Attick (non-sttrccked ID)
Aut. Rad.
0.1
0.2
0.5
1.0
2.0
4.0
0.1
0.2
0.5
1.0
2.0
4.0
MINSS- KMNIST
PostNer
92.8
94.2
93.1
54.7
$1.2
72.9
92.1
92.2
88.7
79.0
78.2
75.9
PrixeNer
99.6
99.1
86.6
99.6
99.3
96.5
99.1
79.2
FvNer
73.7
67.6
74.4
62.2
56.5
59.5
67.7
57.6
67.5
49.5
49.0
55.0
CIFARIO SVHN
PouNNet
71.3
60.8
67.1
68.7
61.8
54.4
65.7
49.9
41.7
37.8
61.0
78.4
PrionNet
17.5
16.3
20.8
17.4
19.8
17.5
16.4
21.7
17.2
21.0
20.1
EvNer
62.9
46.3
37.9
51.4
10,
PostNer
40.1
55.5
33.8
43.0
43.6
47.5
99.7
99.9
99.9
99.9
99.9
100.0
PriodNer
28.4
29.5
39.4
53.3
92.0
98.3
31.2
28.3
27.2
27.8
26.9
26.5
32.2
34.4
93.8
26.7
Sep. sky
PostNet
91.4
54.2
45.4
38.1
40.6
98.8
75,9
87.7
99.1
99.9
108.0
PrionNet
$1.1
30.8
30.8
30.8
30.8
39.2
31.5
30.8
30.8
30.8
30.8
30.8
DDNK
30.9
30.8
30.8
34.9
SL.2
85.A
31.1
30.8
30.8
30.8
30.8
30.8
30

---

Under review as a conference paper at ICLR 2021
Table 57: Adversarial training with Diff. Ent.: OOD detection based differential entropy under Noise uncertainty attacks against differential entropy on ID data and OOD data (AUC-PR).
ID -D-ttccck Loon antacked (OD)
OODaArtack (non attacked IT)
An. Rad.
0.2
0.5
1.0
2.0
40
0.1
0.2
0.5
1.0
2.0
4.0
MINIS-- KMNIST
PowtNet
90.8
$1.0
39.1
38.3
30.8
30.7
86.1
88.9
$1.0
70.8
95.4
100.0
PriorNet
99.8
93.6
42.1
40.0
51.7
100.0
99.3
94.0
44.8
32.1
EvNot
68.6
32.9
53.2
62.2
39.7
43.9
33.7
CIFARIO SVHN
PootNet
46.7
67.3
33.9
30.7
39.4
44.0
88.0
78.2
$4.8
92.1
93.6
95.9
PhonNet
41.9
37.0
33.9
42.5
32.6
50.7
35.4
32.4
38.2
33.3
76.1
99.5
DDNN
58.7
97.5
EvNet
class
PostNet
30.8
47.8
50.0
50.0
50.0
50.0
98.7
99.8
100.0
100.0
99.8
100.0
PriorNet
30.7
30.7
30.9
87.7
100.00
100.0
31.0
30.7
52.4
31.4
34.2
30.7
DDNet
30.7
30.7
47.4
75.0
92.9
70.7
30.7
30.7
50.7
41.3
34.3
39.6
Seg, class sky
50.0
PriorNet
30.8
30.8
30.8
30.8
30.9
100.0
30.9
30.8
50.8
30.9
31.2
32.9
DINNet
30.8
30.8
30.8
31.3
77.8
93.3
50.8
30.8
50.8
30.8
30.8
32.3
FvNett
A.6 VISUALIZATION OF DIFFERENTIAL ENTROPY DISTRIBUTIONS ON ID DATA AND OOD DATA
The following Figures visualize the differential entropy distribution for ID data and OOD data for all models with standard training. We used label attacks and uncertainty attacks for CIFAR 10 and MNIST. Thus, they show how well the DBU models separate on clean and perturbed ID data and OOD data.
Figures and visualizes the differential entropy distribution of ID data and OOD data under label attacks. On CIFARIO, PriorNet and DDNet can barely distinguish between clean ID and OOD data. We observe better ID/OOD distinction for PostNet and EvNet for clean data. However, do observe for any model an increase of the uncertainty estimates on label attacked data. Even worse, PostNet, PriorNet and DDNet seem to assign higher confidence on class label attacks. On MNIST, models show slightly better behavior. They are capable to assign higher uncertainty to label attacks up to some attack radius.
31

---

Under review as a conference paper at ICLR 2021
PoctMet
EvMett
0.4
200
003
004
125
200
002
0.4
100
000
0.39
100
000
0300
004
1300
802
000
6.90
C115
Ca15
877
0:39
7
2
018
030
-1i0s -17
Figure 4: Visualization of the differential entropy distribution of ID data (CIFAR10) and OOD data (SVHN) under label attack. The first row corresponds to no attack. The other rows correspond do increasingly stronger attack strength.
Figures 7 and g visualizes the differential entropy distribution of ID data and OOD data under uncertainty attacks. For both CIFARIO and MNIST data sets, observed that uncertainty estimations all models can be manipulated. That is, OOD uncertainty attacks can shift the OOD uncertainty distribution to more certain predcitions, and ID uncertainty attacks shift the ID uncertainty distribution to less certain predictions.
32

---

Under review as a conference paper at ICLR 2021
PectNet
L-Net
JDhct
600
2.4
150
6.-
300
8.0
23-
13-
06
1.4
100
23-
13-
0.
124
1003
10
7206
1.5
0.
0.5
1.5
80C
06
0.2
153
06
3.2
300
220
12.05 12.03
12.044 12.00
12.002
225
27.5
253
Figure 5: Visualization of the differential entropy distribution of ID data (MNIST) and OOD data (KMNIST) under label attack. The first row corresponds to no attack. The other rows correspond do increasingly stronger attack strength.
33

---

Under review as a conference paper at ICLR 2021
004
110
000
C30
135
ca15
0.-
1300
0.4
0.2
6.00
400
0.5
000
000
0.0
016
144
2.55
2000
L200
00
E.15
114
1200
C320
0.1
0.00
0.
008
23
1300
802
131
1500
004
02
002
01
Figure 6: Visualization of the differential entropy distribution of ID data (CIFARI0) and OOD data (SVHN) under OOD uncertainty attack. The first row corresponds no attack. The other rows correspond do increasingly stronger attack strength.
34

---

Under review as a conference paper at ICLR 2021
PossMct
Drictict
Dyteet
125
0.22
69K
100
032
24
099
692
690
1.5
05
43
035
000
300
30
12.
60
15
130
-12
-00
-30
-3.77
-35.9
113
-3..00
-225
-203
-150
-12 -22 -12.008
Figure 7: Visualization of the differential entropy distribution of ID data (CIFAR10) and OOD data (SVHN) under ID uncertainty attack. The first row corresponds no attack. The other rows correspond do increasingly stronger attack strength.
35

---

Under review as a conference paper at ICLR 2021
L-Net
JDhct
600
2.04
300
1.2
10-
64
300
0.
1.5
0.5
1.5
1.0
221
100
0.
06
0.4
100
1:00
02
2.0
200
20
30
12.00
12.334
32.30
12.000
225
27.5
25.
12.
Figure 8: Visualization of the differential entropy distribution of ID data (MNIST) and OOD data (KMNIST) under OOD uncertainty attack. The first row corresponds to no attack. The other rows correspond do increasingly stronger attack strength.
36

---

Under review as a conference paper at ICLR 2021
Prabliit
15-
1290
500
2.2
C5
IC3
1.2
15-
1.2
509
1.0
2000
15-
08
123
15
08
121
0.5
359
1.
1050
1.3
230
89
-22
Figure 9: Visualization of the differential entropy distribution of ID data (MNIST) and OOD data (KMNIST) under ID uncertainty attack. The first row corresponds to no attack. The other rows correspond do increasingly stronger attack strength.
37


Reviews:
[{"review_id": "gB1Rgl76S9a", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper focuses on quantifying uncertainty for classification problems using Dirichlet based uncertainty (DBU) estimation techniques. The authors study these techniques for their robustness properties under adversarial attacks, proposes a novel attack type targeting uncertainty estimates through differential entropy, and investigates robust training for detecting in and out of distribution data points. Experiments using image datasets showed that uncertainty from DBU models 1) do not provide robust identification of correct predictions under adversarial attacks, 2) are only able to detect weak attacks and do not perform well under strong attacks, 3) robust training does not guarantee generalization to both in and out of distribution datasets.\n\nOverall, I vote for accepting the paper. I like the idea of using uncertainty estimation for adversarial machine learning. My major concern is the limited scope of the paper and the generalization of the results. Hopefully, the authors can address my concerns in the rebuttal period.\n\nPositives:\n-\tThe paper is nicely structured and the organization of the experiments was easy to follow.\n-\tThe experiments were nicely guided by the research questions. Even though the observed results were surprising and do not match the expected behaviors, nonetheless, the observations are important to guide future research in this direction.\n-\tThe idea of attacking the uncertainty estimates is novel and provides an interesting research direction.\n\nNegatives:\n-\tThe scope of the uncertainty estimation to Dirichlet based uncertainty estimation techniques was limited. The authors already cited prominent techniques in the paper, ranging from MC Dropout to ensembles and including the results using these techniques would have made the results more compelling. This will also help generalize the results to the using predictive uncertainty techniques in the context of adversarial robustness.\n-\tThe scope of the adversarial attacks employed in this study were also limited. Even though the authors claim that using uncertainty from DBU models for identifying correct estimates do not produce robust estimates under adversarial attacks, the experiments are performed only using PGD attacks.\n-\tThe uncertainty attacks provide an interesting contribution but I found the section describing the proposed attacks very limited. I would suggest the authors elaborate more on this in the manuscript or in an appendix. Furthermore, I also recommend that the authors elaborate more on the performance of the uncertainty attacks in general and compare success probabilities to the state-of-the-art adversarial attacks.  \n\nMinor comments:\n-\tPlease indicate the best results among the methods used in the tables by highlighting them. \n-\tIn figures 2 and 3, please provide the axes labels.\n-\tI would suggest defining differential entropy in the manuscript for completeness since it is used throughout the paper.\n-\tI would suggest adding a brief summary of the conclusions in the abstract.", "extra_scores": {}, "extra_text": {}}, {"review_id": "TC1JgGaoJtV", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present an analysis of previously proposed Dirichlet based models for adversarial robustness and empirically evaluate exiting methods on two image datasets, MNIST and CIFAR10, as well as 2 tabular datasets.\nWhile in principle adversarial robustness is an interesting topic, the scope of the paper is extremely narrow and I would have liked to see a broader set of Bayesian models being included. \nIn addition, I find the set of experiments very limiting. MNIST is not a representative dataset at all and for a meaningful comparison analysing a large-scale dataset (such as Imagenet) is crucial. Furthermore, I would have liked to see a broader type of data - how about a text dataset (such as 20 newsgroup) or a different sequential dataset with a recurrent architecture? I feel with the limited  experiments is not a very useful resource for practitioners. Also, performance for other attacks (deepfool, black-box attack) would have been interesting.\nEven with the very limited set of experiments, the authors basically report negative results, showing that neither of the approaches could detect adversarial attacks, OOD samples or highly perturbed data\nThe authors also propose a robust training strategy, but concede that this increases performance either for either ID data or OOD data, but not both.\n\nWhile in principle also such negative results can be valuable for the community, I feel in this case the scope of the paper is too narrow and a broader class of Bayesian and ideally non-Bayesian but uncertainty-aware methods should have been analysed.", "extra_scores": {}, "extra_text": {}}, {"review_id": "3LtHJe-a8E", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work, the authors seek to evaluate the robustness and uncertainty of Dirichlet-Based Uncertainty models (DBUs). DBUs are models which rather than outputting a multinoulli distribution (as is the case with DNNs and BNNs) output the parameters of a Dirichet distribution. The authors clearly state and review the history and benefits of such models. While models with calibrated uncertainty are becoming increasingly popular for addressing some of the fundamental shortcomings of standard DNNs, there has been less work in evaluating the relationship between robustness and uncertainty and quantifying the robustness of this uncertainty. This paper make a valuable contribution in that it performs a systematic analysis of the robustness and uncertainty of DBUs. I think the authors evaluation framework is thorough and well-motivated and can serve as a template for how developers of methods which seek to intrinsically capture uncertainty should benchmark their performances wrt adversaries.\n\nThe one minor weak point of the paper is some of its contextualizations with the literature. For example, there are several papers (albeit very recently published ones) which I think it would benefit the authors to reference. In [1] the authors analyze the robustness of a model which outputs the parameters of a Gaussian distribution and find that directly using the parameters of the outputted Gaussian can lead to a strong attack. In [2,3] the authors perform statistical and probabilistic certification of Bayesian neural networks which is strongly related to the claim that the authors are the first to certify methods for uncertainty estimation models. Of course, there is a distinction in the approaches, but it is one the authors should probably make explicit for completeness. \n\nAlso, from a presentation/structural point of view, the authors restate the “assessment metric” paragraph essentially verbatim 3-4 times in the paper and I think perhaps having one global explanation and then discussing the small changes in each subsequent section would make the paper much easier to read. That being said, I do like the format of the presented results, I would just cut down on the redundancy.\n\n[1] - https://arxiv.org/pdf/2003.03778.pdf [ICML]\n[2] - https://www.ijcai.org/Proceedings/2019/789 [IJCAI]\n[3] -http://proceedings.mlr.press/v124/wicker20a.html [UAI]\n\n\nPost Rebuttal Response: \n\nI would like to thank the authors for considering my review and for making some of the suggested edits. I think this paper provides a valuable contribution and point of discussion for those interested in the interplay between robustness and uncertainty in deep learning. I do consider the experimental evaluation in this work to be sufficient given that the authors consider many applications which already exist in the literature, and in my view it is out of the scope of an evaluation/methodology paper to necessarily advance the state-of-the-art in applications of the method they seek to evaluate. \n\nAs I have no major standing criticism of this work, and believe that it provides a useful and interesting contribution to the conference I have increased my score.", "extra_scores": {}, "extra_text": {}}, {"review_id": "-tfd5QuOIKt", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This manuscript addresses an important question of how Drichlet-based uncertainty (DBU) measures can be used to quantify robustness to adversarial label attacks. Robustness to OOD samples of these models were already shown in the papers they were proposed but this work differs from them in using adversarial samples as OOD samples. Via extensive experimental results on various datasets, the authors conclude that the uncertainty estimates are not good indicators for identifying correctly classified samples for adversarially perturbed data.\n\nAlthough I find this contribution valuable, I am still not sure how generalizable the conclusions are. My comments/questions are below:\n\n- What makes adversarial samples so different than other OOD samples? In other words, how come these measures are good indicators in detecting different OOD samples as reported in Malinin & Gates, 2018a but fail in adversarial samples.\n\n- I am surprised that the authors did not mention calibration in the context of uncertainty estimation. Is it possible to relate these findings to calibration? \n\n- I understand that the main focus of this work is Dirichlet-based models but I wonder if it is possible to generalize these findings to other families of models such as deep ensembles (Lakshminarayanan et al., 2017)?\n\n- Figure 1 was never mentioned in text.\n\n- It is not clear what y-axis is in Figure 2. Is it 1-uncertainty?\n\n- I am skeptical about the results on tabular data shown in Table 4 for PostNet. How is possible the detection rate becomes almost perfect for the strongest attack? The authors mention that it is due to density estimation but it is not clear to me how it is possible.\n\n- For the results in section 4.3, I do not understand why the authors decided to report results based on a threshold instead of computing AUCPR as was done in other sections.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers except for AnonReviewer1 were in favour of accept.  AnonReviewer1 was strongly in favour of reject, but AnonReviewer2 argued against some of AnonReviewer1's opinion.  The authors also gave a coherent, well argued statement of their contribution.  Nevertheless, there are some improvements still needed.

Position:   the scope of the uncertainty estimation to Dirichlet based uncertainty estimation techniques was limited.

Sticking to Dirichlet-based uncertainty is limited, although the coverage of methods within the Dirichlet-based family is OK but could be improved. Note (from AnonReviewer1's comments) Joo Chung and Seo, ICML 2020, is one paper that should be included and Chan, Alaa, and van der Schaar, ICML2020 is also relevant.  While its not about adversial attacks it covers a related idea with a good technique.   Finally, these papers cite Ovadia, Fertig Ren etal. NeurIPS 2019, which is an excellent summary of calibration and estimation under shift, not exactly adversarial attacks but surely related.  The big winner is deep ensembles (Lakshminarayanan etal, NeurIPS 2017).  I think using deep ensembles directly would be a good complement to the Dirichlet methods in this paper.
Note, also, the authors already included additional works mentioned by AnonReviewer2.

Critique:   The authors proposed a robust training strategy but this didn't lead to uniform improvement.

Position:  The scope of the adversarial attacks is limited.

The attacks covered are a good though basic range.  But because these show problems, the argument is that more sophisticated attacks do not need to be studied.

Position:  The datasets covered is limited.

Certainly, there are problems with extending experiments to text data.  But the argument is that if things don't work well for the smaller datasets given, then that is still a problem, so why bother extending the evaluation to larger datasets.

Arguably, the latter two positions have been addressed by the authors, but not the first two.  This makes the paper marginal.
So this is a good publishable paper, but comparatively marginal.
Decision: Reject