Avoiding Robust Misclassifications for Improved Robustness without Accuracy Loss
Authors: Yannick Merkli
Pavol Bielik
PETAR TSANKOV
Martin Vechev
Venue: ICLR 2022 Submitted (2022)

Abstract: While current methods for training robust deep learning models optimize robust accuracy, in practice, the resulting models are often both robust and inaccurate on numerous samples, providing a false sense of safety for those. Further, they significantly reduce natural accuracy, which hinders the adoption in practice. In this work, we address both of these challenges by extending prior works in three main directions. First, we propose a new training method that jointly maximizes robust accuracy and minimizes robust inaccuracy. Second, since the resulting models are trained to be robust only if they are accurate, we leverage robustness as a principled abstain mechanism. Finally, this abstain mechanism allows us to combine models in a compositional architecture that significantly boosts overall robustness without sacrificing accuracy. We demonstrate the effectiveness of our approach to both empirical and certified robustness on six recent state-of-the-art models and using several datasets. Our results show that our method effectively reduces robust and inaccurate samples by up to 97.28%. Further, it successfully enhanced the $\epsilon_\infty = 1/255$ robustness of a state-of-the-art model from 26% to 86% while only marginally reducing its natural accuracy from 97.8% to 97.6%.

AVOIDING ROBUST MISCLASSIFICATIONS FOR IMPROVED ROBUSTNESS WITHOUT ACCURACY Loss
Anonymous authors Paper under double-blind review
ABSTRACT
While current methods for training robust deep learning models optimize robust accuracy, in practice, the resulting models are often both robust and inaccurate on numerous samples, providing a false sense of safety for those. Further, they significantly reduce natural accuracy, which hinders the adoption in practice. In this work, we address both of these challenges by extending prior works in three main directions. First, we propose accuracy and minimizes robust inaccuracy. Second, since the resulting models are trained to be robust only if they are accurate, leverage robustness abstain mechanism. Finally, this abstain mechanism allows us to combine models in a compositional architecture that significantly boosts overall robustness without sacrificing accuracy. We demonstrate the effectiveness of our approach to both empirical and certified robustness on six recent state-of-the-an models and using several datasets. For example, on CIFAR-10 with &xc 1/255, it successfully enhanced the robust accuracy of a state-of-the-ar standard trained model from 26% to 86% while only marginally reducing its natural accuracy from 97.8% to 97.6%.
training method that jointly maximizes robust
principled
INTRODUCTION
In recent years, there has been significant amount of work that studies and improves both adver-
Hein 2020 Madry et al. 2018) and certified robustness (Balunovic & Vechev| 2019: Cohen et al. 2019: Salman et al. 2019 Xu et al. 2020 Zhai et al. 2020 Zhang et al. 2019b) of neural networks. However, currently, there two key limitations that hinder the wider adoption of robust models in practice.
sarial (Szegedy et al. 2013 Goodfellow et al. 2014 Carlini & Wagner 2017 Croce
Existing Models are Robustly Inaccurate First, despite substantial progress in training robust models, existing works usually only report robust accuracy, i.c., samples for which the model robustly predicts the correct label. Meanwhile, the issue of robust inaccuracy, i.c., samples that are robustly misclassified with a wrong label, usually not even reported (we formally define robust inaccuracy in Equation 3). This is especially problematic for cafety-critccl models, where the robustness can be mistakenly used as a safety argument. We quantify the severity this issue in Table [ by evaluating recent state-of-the-an robust models. As can be seen, recent models contain up 15% of robust inaccurate samples and the ratio of such samples worsens with smaller perturbation regions.
CIFAR-10
CIFAR-100
MT3D
Krizhevsky et
Krizhevsky al.
Appendix A.1
Appendix A.
Zhang
Gowal
Rehulfi
Zhang et
Zhang
B
4.6%
3.6%
2.9%
15.2%
3.9%
7.0%
3.3%
0.9%
4.3%
2.3%
7.
B5/255
2.6%
0.8%
1.3%
3.9%
1.7%
6.4%
Table 1: Percentage of robust and inaccurate samples for various recent robust models and datasets, which we describe more detail in Section Each model trained for the indicated threat model and evaluated using 40-step APGD (Croce & Hein 2020

---

rubust & accurate
rubust inaccurate
non robust & accurate
[ours]
non-robust inaecurate
Card
CTRADES
CERA
Figure 1: Decision regions for models trained via standard training Lst. adversarial training LTRADES (Zhan al. 2019a), and training LERA (Equation 5). In this case, training achieves the same robust accuracy as LTRADES but avoids all robust inaccurate samples by making them non-robust. Note that all three models predict over all three classes, however, the decision regions for class 2 of the LTRADES and LeRa trained models happen to be too small to be visible. The considered model architecture and the training hyperparameters are provided in Appendix A.2
Robustness Accuracy Tradeoff Second, existing robust training methods improve the model
robustness, but they also typically degrade the standard accuracy on unperturbed inputs. To address this limitation, number of recent works study this issue in detail and propose new methods to
mitigate it (Mueller et al. 2020
Raghunathan et al. 2020 Yang et al. 2020 Stutz et al.) 2019).
Our Work In this work, we advance the line of work that aims boost robustness without sacrific- ing accuracy, but we approach the problem from new perspective by avoiding robust inaccuracy.
Concretely, we propose a new training method that jointly maximizes robust accuracy while mini- mizing robust inaccuracy. We illustrate the effect of our training on synthetic dataset (described in showing the decision boundaries of three models, trained using standard training Catd, adversarial training ATRADES (Zhang et al. 2019a), and our training LERA (Equation 5). First, observe that while the Lid trained model achieves 100% accuracy, only 91.1% 1% of these samples robust (and accurate). When using LTRADES. can observe the robustness accuracy tradeoff the robust accuracy improves to 98.4% at the expense of 1.6% (robust) inaccuracy. In contrast, using LERA. retain the high robust accuracy of 98.4% but avoid all robust inaccurate samples by appropriately shifting the decision boundary, rendering them non-robust.
Appendix AA.2) Figure ]
Second, since models trained be robust only if they are accurate, leverage robustness as a principled abstain mechanism. This abstain mechanism then allows us to combine models in compositional architecture that significantly boosts overall robustness without sacrificing accuracy. Concretely, in Figure I we would define selector model that abstains on all non-robust samples. Then, the abstained (non-robust) samples are evaluated by the standard trained model Letd, while the selected samples are evaluated using the robust model LERA. This allows to achieve the best of both models high robust accuracy (98.4%), high natural accuracy (100%), and no robust inaccuracy.
We show the practical effectiveness of our approach by instantiating over several datasets and existing robust models for both empirical and certified robustness. Our evaluations show that our method effectively reduces robust and inaccurate samples by up to 97. 97.28%. Further, our approach significantly improves robustness for CIFAR-10, CIFAR-100, MTSD and SBB datasets by 60.3%, 38 38.8%, 29.2% and 37.7%, respectively, while simultancously decreasing natural accuracy by only 0.2%.
RELATED WORK
There is a growing body of work that extends models with an abstain option, either by training selector mechanism separately or jointly together with the model. The existing approaches include various selection mechanisms such as entropy selection (Mueller et al. 2020). selection function
(Cortes et al. 2016 Mueller et al. 2020 Geiffan & El- Yaniv 2019), softmax response (Stutz al. 2020 Geifman & El- Yaniv 2017), explicit abstain class (Laidlaw & Feizi 2019 Liu et al. 2019.. In our work, we explore an alternative selector mechanism that uses model robustness. The advantage of this formulation is that the selector provides strong guarantees for each sample and can never produce false-positive selections. The disadvantage is that introduces a significant runtime overhead, compared many other methods that require only single forward pass.
2

---

At the same time. several recent works investigated the robustness and accuracy tradeoff both theoretically (Yang et al. 2020; Dobriban et al. 2020 and practically by proposing new methods to mitigate it. For example. Raghunathan et al. (2020) proposes robust self self-training that leverages unlabeled data to regularize the model. SSttz et al. (2019) considers a new method based on on- manifold adversarial examples, which more aligned with the true data distribution than the f--norm noise models. |Mulllrr et al. (2020) focuses on deterministic certification and also proposes using compositional models control the robustness and accuracy tradeoff.. In our work, also take advantage of compositional models, but we focus on both empirical and probabilistic certified robustness. Further, our selector formulation inaccuracy and can be used to fine-tune any existing robust model. Finally, we provide individual robustness at inference time, rather than distributional robustness considered in prior works.
based on new training that minimizes robust
PRELIMINARIES
Let fo: R2 - IR be a neural network which classifies inputs E X' CRd to outputs RR (e.g., logits or probabilities). The hard classifier induced by the network is given as Fo(ar) = arg max4ey [a()), where (x); is the network output for the i-th class and V/ | k is the finite set of discrete labels.
Natural Accuracy Given a distribution over input-label pairs D and a classifier F'g: X + V, an input-label pair (z, v) is considered accurate iff the classifier Fa predicts the correct label y for a:
Ruat (Fa) B(a.y)>
1
(x) y
(I)
Robust Accuracy Given an input-label pair (z,y), we say that the classifier Fe is robust and accurate iff it predicts the correct label y for all samples from a predefined region (8 (x). such as
f--norm ball centered with radius E, i.e., (x) = {a': || ællp S $). Formally:
Rooe (F)) = ((ay)>> ((Fe(x) = y A 1 [ E (x)) (x = Fa(x))
(2)
Robust Inaccuracy Similarly to robust accuracy, an input-label pair (x, y) is considered robustly
inaccurate iff the classifier F6 predicts an incorrect label Fo(x)) + y and Fg is robust towards that misprediction for all inputs in (x)). Formally, the robust inaccuracy is defined as:
RToce (Fo)
(a,a)D ([F((=) * y) 1(va' (g(()) Fo(æ²) Fo()))
(3)
4 REDUCING ROBUST INACCURACY: ADVERSARIAL & CERTIFIED TRAINING
In this section, we present our training method that extends existing robust training approaches by also considering samples that are robust but inaccurate. We start by describing high-level problem statement which we then instantiate for both empirical robustness as well as certified robustness.
Problem Statement Given a distribution over input-label pairs D, goal is to find model parameters 0 such that the resulting model maximizes robust accuracy, while at the same time minimizing robust inaccuracy. Concretely, this translates the following optimization objective:
arg min E (x.g))D 0
3 Lro v)
+ ([F(())) + f} C (x, y)
(4)
optimize robust accuracy
penalize robust inaccurayy
where B E R+ is a regularization term, (z) # y) is an indicator function denoting samples for which the model is inaccurate, and Lrob (x, y) with C (x, y) are loss functions that optimize robust accuracy and penalize robust inaccuracy, respectively. Here, the first loss function Cros (x. v) is standard and can be directly instantiated using existing approaches (see next). The main challenge comes in defining the second loss term, as well as ensuring that the resulting formulation easy to optimize, e.g., by defining smooth approximation of the non-differentiable indicator function.
4.1 ADVERSARIAL TRAINING
We instantiate the loss function from Equation 4 when training empirically robust models as follows: LERA (fo v)) = B-LAARD (fo v))+ +1 fo())y)) min (ce(fo(æ'), arg max fo(x')) (x)))) a (B(())
(F))(F)(w))
(5)
3

---

Below, we introduce each term in more detail and discuss the motivation behind our formulation.
Frob To instantiate Crob, we can use any existing adversarial training method. For example, considering standard adversarial training (Goodfellow et al. 2014) would result in the following loss:
Lads
max (x)
(CR(Jo(x"),))
where fCE is cross-entropy loss of the worst example in the allowed perturbation region B Similarly, can instantiate the loss using more sophisticated methods, such as TRADES (Zhang al. 2019a):
LTRADES = lCE (cn(Jo((),))) +
max x E (a)
(ot, So(z"))
where DKL is the Hullback-Leiblee divergence (Kullback Leibler 1951).
1[F((x) * y) Next, consider the indicator function, which encourages learning on inaccurate samples. Since the indicator function is computationally intractable, we replace the hard qualifier by soft qualifier 1 fo (aryg- The soft qualifier will be small for accurate and large for inaccurate samples, thus providing a smooth approximation of the original indicator function.
Third, define the loss that penalizes robust but inaccurate samples. This can be formulated similar to the adversarial training objective (Madry et al. 2018), however, instead of optimizing the prediction of the adversarial example fo(x') towards the correct label y, we optimize towards the most likely adversarial label arg max oaX. fo(x").. This leads to the following formulation:
C
min (fo(F) x' (x)
arg argmax max fo(x')))
(F))(F(()))
Note that the purpose of the 4Ta loss is to penalize robustness by making the model non-robust. As result, it is sufficient to consider only single non-robust example, thus the minimization (rather than maximization) in the loss objective
4.2 CERTIFIED TRAINING
instantiate the loss function from Eqqattion4foo probabilistic certified robustness via randomized smoothing (Cohen et al., 2019). Randomized smoothing constructs a smoothed classifier Ge: X from base classifier Fs. where Gg(æ) predicts the class which Fo is most likely to return when æ is perturbed under isotropic Gaussian noise. Our proposed instantiation of Equation forr probabilistic certified robustness as follows:
Similarly to Section 4.1
k fo(x +n)+ CR(fe, (2, (9)
1 k
(z. 9)) = 3- Enoise (fo. (x. y))
Σ j=1
(1-fo(o
where m!..... 7h are k i.i.d. samples from NNO, I² Note that, since the robustness guarantees provided by randomized smoothing hold for the smoothed classifier Gg. the three loss components from Equation 4 need to be formulated with respect to the smoothed classifier Gg.
Crob To instantiate Crob, we can use any existing certified training method for randomized smooth- ing, such as the methods defined by Cohen al. (2019) or Zhai et al. (2020). Concretely, when using Cohen et al. (2019), the loss defined using Gaussian noise augmentation:
Lnoise www(fa((eeee (fo(( n).y), 1 ~~(000001)
(10)
1{Fg(x) # y} We again replace the computationally intractable hard qualifier by a soft qualifier Es-Nio,0"'\1- fo (x + y) which encodes the misprediction probability of the smoothed classifier. In practice, we approximate expectations over Gaussians via Monte Carlo sampling, thus leading to the approximated soft inaccuracy qualifier 1/4 Σ 1 fø(æ + n))y-
Naturally, this assumes that the method used to check robustness can correctly detect the non- roburssssssss even if it is caused by single example. Note that, for fair evaluation, use relatively weak 10-step PGD (Madyy et al. 2018) attack during training and strong 40-step APGD (Croce Hein 2020 for evaluation.
4

---

Finally, instantiate the Cyate loss term, which encourages the model toward non-robust predictions on robust but inaccurate samples. We propose to minimize robustness by directly minimizing the certified radius of the smoothed classifier Gg. The certified radius formulation by
Crob
Cohen et al. (2019) involves a sum of indicator functions, which is not differentiable. However, (Zhai all.0. 2020) have recently proposed the following differentiable certified radius formulation:
(R(fe,(x,))) =
2 +
Σ fo + m;;1),) 4
max
Σ fo(x + m;:l),)] (1I) y 8'
where -- is the inverse of the standard Gaussian CDF, is the inverse softmax temperature multiplied with the logits of fo. and mis are k i.idd. samples from NN0. o² Note that, by setting the loss term Frbb C(foo (xx y)), we directly penalize robustness of the smoothed classifier Gg.
ROBUST ABSTAIN MODELS
In this section, we extend the models trained so far by leveraging robustness as a principled abstain mechanism. Further, we define an additional loss function based on the Deep Gamblers loss (Liu
al. 2019). which is specifically designed for training adversarially robust abstain models.
Abstain Model Consider an input space A' C Rd and a label space V. A model with an abstain option (EI- Yaniv al., 2010) is a pair of functions (Fo, S), where Fo: X -> ) is classifier and S: X + t0, If is a selection mechanism, which acts binary qualifier for Fg. Let S(x) 0 indicate that the model abstains on input æ E X, while ((x) = 1 indicates that the model commits to the classifier Fe for input æ and predicts Fo(x).
Robustness Indicator Selector We now instantiate abstain models with a robustness indicator selector, that abstains all non-robust samples. For adversarial robustness, the selector is defined as:
SERI (x) 1(Va' E B(x): Fe Fx(') = Fo(x)}
(12)
For certified robustness, the selector is defined as:
Sca( 1 1 B(z): Go(z') = Gz(z))
(13)
Robustness Guarantees: Robust Selection Similar to robust accuracy, the robustness of an abstain
model needs be evaluated with respect threat model. In our work, we consider the same threat
model as for the underlying model Fe. namely (x) = {w': : ||æ æ11p S e). a f--norm ball centered with radius Then, we define the robust selection of an abstain model follows:
Rest(s) = E( 1 1[Ve's E (x)) S(z')= =
(14)
That is. say that model is robustly selecting if the selector S would select all valid perturbations x' E (x)). When used together with our definition of SERI, we obtain the following criterion:
Reel (SER))
1 L(vx's E BB((x). SERr =13
B(a,y)/D
1(ve's E (x))). 1 1(va" E (x))..) Folx²) (x(2))} 1(Ya' E BB. Fo(x') Fa(x))
B(a,y)/D
In other words, guarantee that the selector Sear is robust for all ar' E (x)), we in fact need check robustness of the model Fg to double that region x' E B (æ). This is important in order to obtain the correct guarantees and is reflected in our evaluation in Section 7
Note that when evaluating robust selection for certified training. it is sufficient to show that the smoothed model Go can be certified with radius R 2 Then, the smoothed model guarantees that Go(x') = for all x' E (x)), which is equivalent to our condition Yz E BB(x): Gø(z') Go(z).
BOOSTING ROBUSTNESS WITHOUT ACCURACY LOSs
Consider an abstain model (Fo, S) and let D be a dataset to evaluate (Fa, S). The selector S partitions 'D into two disjoint subsets the set of abstained inputs D..s and the set of selected inputs D, for
5

---

which F6 makes prediction. However, depending the application, it may be desirable to make a best-effort prediction on all samples, including D- This insight leads to compositional architectures, already used by number of prior works (Mueller et al. 2020 Wong et al. 2018).
Let H ((Frobust: S), Forre) be a 2-compositional architecture consisting of a selection mech- anism S, robustly trained model Frobust, and a core model Feore. Given input x: X, the selector S decides whether the model is confident on E and commits to the robust model Frobust or whether the model should abstain and fall back to the core model Feore. Formally:
(15) Note that each model of a compositional architecture can be chosen arbitrarily, regardless of the model's network architecture or its training. However, one benefit of using compositional ar- chitecture is combining models that complement each other, resulting in overall improved model performance. In our work, we combine models trained via adversarial or certified training (which typically have lower natural accuracy), with models trained using standard training (which have high natural accuracy but low robustness). The compositional model performance then depends on the quality of the selector S, used to determine which model to evaluate for each sample.
(x)) = S(x) . (r)) + (1 S(x)) Fzore(z)
7 EVALUATION
In this section, we present thorough evaluation of our approach by instantiating it to four different datasets, six recent state-of-the-art models, for both adversarial and certified robustness, and including top-trained models from RobustBench (Croce et al. 2020). We show the following key results:
Fine-tuning existing models with our proposed loss successfully decreases robust inaccuracy and provides a Pareto front of models with different robustness tradeoffs.
Combining our proposed loss and robustness as an abstain mechanism leads to higher robust selection and accuracy compared to softmax response and selection network baselines.
Our 2-compositional models significantly improve robustness by up to +60% while retaining the natural accuracy, causing only minor decrease of up to -0.2% (for B1/255 and 22)250)
We perform all experiments on single GeForce RTX 3090 GPU and use Py PyTorch (Paszke et al. 2019) for our implementation. The hyperparameters used for our experiments are provided in Appendix A.2
Models
Our proposed training method requires neither retraining classifiers from scratch nor modifications to existing classifiers, thus our approach can be applied to fine-tune a wide range of existing m ? To demonstrate this, we use the following robust pre-trained models:
For empirical robustness, we evaluate five existing models from Carmon et al. (219)), Gowal et al. (2020), Rebulfi al. (2021), SShwag al. (2021) and Zhang et (2019a), where all but the last model are top models taken from RobustBench (Croce et al. 2020). The model by [Sehwag et al. trained for E2 = 0.5, while the other models are trained for 6o = 8/255.
our evaluation, we fine-tune each model for 50 epochs for the considered threat model (E E { (1/255, 2/255, +/255 } and 82 E (0,112, 0.255) using both the LTRADES loss (Zhang et al. 2019a) and our proposed LERA loss.
For certified robusiness, we use the 0.12 Gaussian noise augmentation trained model by Cohen jet al. (2019) and the E2 = 0.50 adversarially trained model by /Sehwag et al. (202T). Similarly to empirical robustness, we fine-tune the models for 50 epochs using either Gaussian noise augmentation training (Cohen et al. 2019) or the LCRA loss proposed in our work.
Datasets We evaluate our approach on two academic datasets CIFAR-10 and CIFAR-100 (Krizhessky et al. 2009), and two commercial datasets - Mapillary Traffic Sign Dataset (MTSD) (Ertler et al. 22000) and Rail Defect Dataset kindly provided by Swiss Federal Railways (SBB). We provide full details, including example images and all preprocessings steps, in the Appendix A.
When training the CIFAR-10 and CIFAR-100 datasets,
the AutoAugment (AA) policy by Cubuk et al. (018)) as the image augmentation. For the MTSD and SBB datasets, we use standard image augmentations (SA) consisting of random cropping, color jitter, and random translation and rotation. For completeness, our evaluation also includes models trained without any data augmentations.
Our method can also be used to train from scratch, in which case scheduler for should be introduced.

---

CIFAR-10
CIFAR-100
MTSD
S83
Zhang el
Cammon el
Gowal et ak.
Rebulli el
Zhang et al. B 22/288
Zhang et al.
Robust Accorncy RRaah (96)
B2/258
B2
B 2/256
2//65
B2/268
85-
07
hetter
85
05
84
33
82
10
255
Robust Inaceuracy R [96]
better
Figure 2: Robust accuracy (Roch) and robust inaccuracy (Ryob of existing robust models (4, fine-tuned with our proposed loss (o, Our approach consistently reduces the number of robust
inaccurate samples across various datasets, existing models and different regularization levels B.
Metries We use the natural accuracy, robust accuracy, and robust inaccuracy as our main evaluation metrics, defined in Section but evaluated the corresponding test dataset D {( y/ )
When evaluating the empirical robustness, we use 40-step APGD (Croce & Hein 2020) to evaluate the robustness of the classifier Fe. To evaluate certified robustness, we use the Monte Carlo algorithm for randomized smoothing from (Cohen et al. (2019). We certify 500 test samples and use the same
randomized smoothing hyperparameters as Cohen et al. (2019) (ef. Appendix A.2
7.1 REDUCING ROBUST INACCURACY
Empirical Robustness For empirically robust models, the results in Figure show the robust accuracy (Ros)) and robust inaccuracy (R,a) reee) of different existing models fine-tuned with (4) and without (4) data augmentations. At the same time, Figure also shows the same models fine-tuned with our proposed loss with and without (O) data augmentations. We can see that our approach improves over the existing models across all the datasets. For example, CIFAR-10 and for 2//25 the model from Carmon al. (219) achieves 86.5% robust accuracy, but also robust inaccuracy of 1.34%. In contrast, using our loss LERA. we can obtain a number of models that reduce robust inaccuracy to 0.29%, while still achieving robustness of 83.8%. Similar results are obtained for other models, datasets, and perturbation regions (cf. Appendix A..5p. We generally observe that our approach achieves consistently lower robust inaccuracy compared to adversarial training. Further, by varying the regularization term B, we obtain Pareto front of optimal solutions.
Similarly, we show the robust accuracy (Raa)) and robust inaccuracy (R, race for certifiably robust models, which were fine-tuned using Gaussian noise augmentation Engise and using our proposed loss function LCRA. In Table 2 we show results on CIFAR-10 for BP. and 18.22 perturbation regions. We to existing models. For example, on CIFAR-10 and B8 the Cohhen et al. 2019) model achieves 62% robust accuracy, but also 1% robust inaccuracy. In contrast, our approach reduces the robust inaccuracy to 0.4% while still achieving 53.8% robust accuracy. For the (Sehwag et al. (202T) model, our approach even improves both the robust accuracy and the robust inaccuracy. For B B5.25 our approach improves the robust accuracy by +4.8% and reduces the robust inaccuracy by -0.6%.
Certified Robustness
that our approach achieves lower robust inaccuracy compared
CIFAR- 10010
0 (a 0.06)
25 (a = 0.12)
Pre-trained Model
Finetuning
R R,7o
R R
Cohen al.
2019
Lnoise
74.0
2.8
62.0
1.0
(ResNet- 110)
LCRa (ours)
71.6
2.6
53.8
0.4
Schwag et al. 2021
Lnoise 87.0
1.8
77.4
1.4
(ResNct-18)
LCRA (ours)
90.8
1.6
82.2
0.8
Table 2: Robust accuracy (Ra5)) and robust inaccuracy (R,aaee) (ace) of existing robust models fine- tuned with our proposed loss LCRA. All models are certified via randomized smoothing, using the hyperparameters listed in Appendix A.2
7

---

CIFAR-10
CIFAR-100
XTSD
Carmon al.
2019
Gowal 2020
RRbuffi 2021
Zhang 2019a
WRN-28-10
51/255
WRN-28 28-10, B1/255
WRN 28- 10. B2/250
Re B 2/ 255
Apcuracy 08
67.5
$7.5
40-
26
52.5
02.5
34
botter
22
97-
20
65.0
$5.0
06
80.0
$0.0
Robust Selection R R.ob [96]
Figure 3: Comparison of different abstain approaches including existing robust classifiers TRADES (a, A), classifiers
with our proposed loss ERAgI (O, ). selection network (m, *) and softmax
response (o, abstain models. The higher Ryob and Roob the better (top right corner is optimal).
7.2 USING ROBUSTNESS TO ABSTAIN
Next, we evaluate using robustness as an abstain mechanism (Section 5 and how it benefits from the training proposed in our work. We compare the following abstain mechanisms:
Softmax Response (SR) (Geifman & El- Yaniv 2017), which abstains if the maximum softmax output of the model fo is below threshold for some input x' E (z),, that is:
Ssa (z) 1(vx's E B (x)) max.ey fo(x')) 2 T}
(16)
Similar to the robustness indicator (Section [5). to guarantee robustness of Ssa. we need to check the maximum softmax output of the model
double the region (z)). To evaluate robustness of
SSR, we use modified version of APGD called APGDconf (Appendix A.4). For each model considered
our work (e.g.. Carmon et (2019)). ). we evaluate its corresponding abstain selector:
(o. CARMONga, GOWALsR, ZHANGsa. etc., all of which fine-tuned using TRADES.
Robustness Indicator (RI) (our work), which abstains if the model Fo is non-robust:
(a) 1 Vaa'' (x)): Fo(x') (a:))
(17)
Note that, unlike other selectors, our robustness indicator by design robust against an adversary using the same threat model. For each model, we compare two instantiations of this approach:
(4, A) TRADESRI. (o, ERARI (Equation 5).
Selection Network (SN), which trains separate neural network sp: X' + R and selects if: (s) 2 +}
Sex (x)
(18)
When considering the robustness the abstain model, the robustness of both the classifier and selection network have to be taken into account. We compare against two instantiations of this approach, both trained using certified training:
ACE-COLTsy (Balunovic & Vechev 2019
Mueller al. 2020 and
ACE-IBPet (Gowal et al. 2018 Mueller et al. 2020
Empirical Robustness In Figure 3 we show the comparison of different abstain approaches using two metrics the robust selection (Raef)) and the ratio of non-abstained samples that are robust and accurate (R Ideally, would like both of these to be high possible, but typically there a tradeoff between the two. This can clearly be seen in the results, where both our approach and softmax response can be used to obtain Pareto front of optimal solutions.
Overall, the main results Figure 3 show that as designed, our approach consistently improves robust accuracy. For example, CIFAR-10 Eoc 1/255 and Carmon al. (2019) model,
8

---

CIFAR-10
B6.12 (a = 0.06)
B6.2 (a 0.12)
Pre-trained Model
Finetuning
Ryob
Rrob
RTob
Ryob
Luoise
76,8
96.35
63.0
98,41
Cohen et al. 2019
(ResNet-110)
LCRA (ours)
74.2
96.50
54.2
99.26
Luoise 88.8
97.97
78.8
98.22
Schwag 2021
(ResNet-18)
CCRA (ours)
92.4
98.27
83.0
99.04
Table 3: Comparison of our proposed loss LCRA with the Lnoise used in probabilistic certification. All models are certified via randomized smoothing, using the hyperparameters listed in Table A.2
can successfully improve robust accuracy by +1.18% at the expense of -3.78% decrease in robust selection. This is close optimal since increasing robust accuracy is typically achieved by correctly selecting for which samples to abstain. Interestingly, for some models and datasets, we can strictly improve over the baseline models by increasing both the robust accuracy and the con-abstained samples. On CIFAR-10 at c = 1/255 and Gowal et al. (2020) model, we increase the robust accuracy by +1.06% and robust selection by + +1.61% 1 (when training without data augmentations).
Compared to the other abstain methods, our approach, in general, improves both metrics while also providing much stronger guarantees. Concretely, our approach guarantees that selected samples are robust in the considered threat model. Softmax response only guarantees that all samples in the considered threat model have high confidence and is thus vulnerable to high confidence adversarial examples, and the selection network provides guarantees with regards to the selector's robustness.
Certified Robustness Applying our training for certified robustness CCRA with 1.0 consis- tently improves robust accuracy Ris of robustness indicator abstain models. In Table we show results on CIFAR-10, using & perturbations of radius 0.06 and 0.12. For instance, for the Cohen eet al. (2019) model trained at a = 0.12, we are able to improve the robust accuracy by +0.85% for E2
0.25 perturbations, the expense of -8.8% decrease in robust selection. On the other hand, for the Sehwag et al. (2021) model, our approach improves on both metrics, For e2 0.25 perturbations, we increase robust accuracy by +0.82% and robust selection by +4.2%%
7.3 BOOSTING ROBUSTNESS WITHOUT ACCURACY LOSS
Finally, we present the results of combining the abstain models trained so far with state-of-the-ard models trained achieve high accuracy. Note that, discussed in Section [ when evaluating adversarial robustness for BP, we in fact need to consider B% robustness of the abstain model.
summary of the results shown Figure Similar to the results shown so far, the 2-compositional architectures that use models trained by our method (o, improve over existing methods that optimize only for robust accuracy (4, 4), as well as over models using softmax response (o, + or selection network (m. to abstain. For example, for SIFAR-10 with = 1/255 and the Carmon et al. (2019) model, we improve natural accuracy by +0.58% and +0.62%, while decreasing the robustness only by -2.55% and -2.82%, when training with and without data augmentations respectively.
More importantly, our approach significantly improves robustness of highly accurate non- compositional models, with minimal loss of accuracy. In fact, in half of the cases, the compositional architecture even slightly improves the overall accuracy, as summarized in Table We provide full results, including additional models and perturbation bounds in Appendix A.7 and evaluation of the considered highly accurate non-compositional models in Appendix A.8
CIFAR-10 10
CIFAR-100
MTSD (RasXet-50)
SBB (ReaNat-50), B
Zhao
2020
(WidaRenNat.
182/255
Rome
26.2
60..3%
86.5
3.1
+338 8%
41.9
40.7
+ 29. 29.2%
69.9
44.7
+37 7%
82.4
R.at
97.8
0.22
97.6
80.17
0.01%
80.18
93.8
+0.2%
94.0
91.4
-0.1%
91.3
Table 4: Improvement of applying our approach to models trained to optimize natural accuracy only.

---

CIFAR-10
HTSD
SBB
Carmon al. (011))
Gowal et al. 2020
Zhang el al. 2019a ResNet-50.
Zhang et al 2019a
WRN-28-10
1//255
WRN-28-10, 11/255
22/250
RosNet-50. 2//2bb
Robuat Arcuracy reas
72
62.5
73
75.0
50
92
70.0
40-
92
92
96
92
120
92 52
94
90
00
300
93
92
53
94
95
90.2
50.5
91.0
23.5
02.0
Natural Accuracy R [%6]
belter
Figure 4: Natural (R and robust accuracy (Roa)) for 2-compositional ERARI models (O, and c-comoosiiional TRADESa1 models (4, 4). Further, we also consider 2-compositional ACE-COLTsr: ACE-IBPs: (m, ) and 2-compositional TRADESge (o. models. The core models used in the
compositional architectures are listed Appendix A.8
CONCLUSION
In this work, we address the issue of robust inaccuracy state-of-the-art robust models. Ideally, models should be robust only if accurate, however, show that existing robust models have non- negligible amounts of robust but inaccurate samples. We present a new training method that jointly minimizes robust inaccuracy and maximizes robust accuracy. The key concept was extending existing robust training loss with a term that minimizes robust inaccuracy, making our method widely applicable since it can be instantiated using various existing robust training methods. We show the practical benefits of our approach by both, using robustness as an abstain mechanism, and by leveraging compositional architectures improve robustness without sacrificing accuracy.
However, there are also limitations and interesting extensions to consider in future work. First, while there are some cases where our training improves both robust accuracy and reduces robust inaccuracy (e.g., for certified robust Sehwag et al. (2021) model on CIFAR-10 or empirically robust Gowal al. model on CIFAR-10), , does typically results in trade-off between the two reduced robust inaccuracy also leads to reduced robust accuracy. To address this issue, in practice we compute Pareto front of optimal solutions, An interesting future work would be to explore this trade-off further and develop new techniques to mitigate it. Second, given that we compute a Pareto front of the optimal solutions, a promising future work item is considering model cascades that consist of different models along this Pareto front, and progressively fall back to models with higher robust accuracy but also higher robust inaccuracy. Third, observed that the robust inaccuracy approaches becomes much harder. This is both because these remaining robust inaccurate examples are the hardest to fix, well because there only very few of them. In work, explored using data augmentation to address this issue, but more work is needed to make the training efficient in such a low data regime.
of which can used to instantiate the compositional model.
(i.e., the best case), the training
REFERENCES
Mislav Balunovic and Martin Vechev. Adversarial training and provable defenses: Bridging the gap. In International Conference on Learning Representations, 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Yair Carmon, Aditi Raghumathan, Ludwig Schmidt, Percy Liang, and John Duchi. Unlabeled data improves adversarial robustness. arXiv preprini arXiv: 1905. 13736 2019.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference Machine Learning, pp. 1310-1320. PMLR, 2019.
10

---

Corinna Cortes, Giulia DeSalvo, and Mehryar Mohri. Learning with rejection. In International Conference on Algorithmic Learning Theory, pp. 67-82. Springer, 2016.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fie attacks. In International conference on machine learning. pp. 2206-22 2216. PMLR, 2020.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flam- marion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robusthench: standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670) 2020.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018.
Edgar Dobriban, Hamed Hassani. David Hong, and Alexander Robey, Provable tradeoffss in adversar- ially robust classification. CoRR, abs/2006.05 051.1, 2020. URL https: https::/axxi...rr/ abs/ 2006. 05161
Ran El- Yaniv et al. On the foundations of noise-free selective classification. Journal of Machine Learning Research, 11(5), 2010.
Christian Ertler, Jerneja Mislej, Tobias Ollmann, Lorenzo Porzi, Gerhard Neuhold, and Yubin Kuang. The mapillary traffic sign dataset for detection and classification on a global scale. European Conference Computer Vision, pp. 68-84. Springer, 2020.
Yonatan Geifman and Ran Yaniv.. Selective classification for deep neural networks. In Proceedings of the st International Conference on Neural Information Processing Systems, NeurlPS 17, pp. 4885 4894, 2017.
Yonatan Geifman and Ran El-Yaniv. Selectivenet: A deep neural network with an integrated reject option. In International Conference on Machine Learning, pp. 2151-2159. PMLR, 2019.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 11225:1412.6572, 2014.
Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint 18
2018.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathemati- cal statistics, 22(1):79 86, 1951.
Cassidy Laidlaw and Soheil Feizi. Playing it safe: Adversarial robustness with an abstain option. arXiv preprint urXiv: 11253, 2019.
Ziyin Liu, Zhikang Wang, Paul Pu Liang, Russ R Salakhutdinov, Louis- Morency, and Masahito Ueda. Deep gamblers: Learning to abstain with portfolio theory. Advances in Neural Information Processing Systems, 32: 10623-10633, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant adversarial attacks. In International Conference on Learn- ing Representations, 2018. URL htth:tts::/ /ttp::/://///eeeviie/e/e net.:/oouumiiirra1BBBB....
Mark Niklas Mueller, Mislav Balunovic, and Martin Vechev. Certify predict: Boosting cer- tified robustness with compositional architectures. In International Conference on Learning Representations, 2020.
11

---

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. Proceedings of the 37th International Conference on Machine Learning, volume 1 119 of ICML'200, pp. 7909-7919, PMLR, 13-18 Jul 2020.
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Tim- othy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprini arXiv:2/03.01996, 2021.
Hadi Salman, Greg Yang. Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, and Sebastien Bubeck. Provably robust deep learning via adversarially trained smoothed classifiers. arXiv preprint arXiv: 140,04440 2019.
Vikash Sehwag, Saeed Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, and Prateek Mittal. Improving adversarial robustness using proxy distributions. arXiv preprint arXiv:2104.09425 2021.
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling adversarial robustness and generaliza- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp.
2019.
David Stutz, Matthias Hein, and Bernt Schiele. Confidence-calibrate adversarial training: Generaliz- ing to unseen attacks. In International Conference Machine Learning, pp. 9155-9166. PMLR, 2020.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Eric Wong, Frank R. Schmidt, Jan Hendrik Metzen, and J. Zico Kolter. Scaling provable adversarial defenses. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, NeurlPS' 18, pp. 8410-8419, 2018.
Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang. Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsich. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems, 33, 2020.
Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaud- huri. A closer look accuracy vs. robustness. In Advances in Neural Information Processing Systems, volume 33 of NeurlPS'20, 20, pp. 2588-8601. Curran Associates, Inc., 2020.
Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsich, and Liwei Wang. Macer: Attack-free and scalable robust training via maximizing certified radius. arXiv preprint arXiv:2001.02337, 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472-7482. PMLR, 2019a.
Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsieh. Towards stable and efficient training of verifiably robust neural networks. arXiv preprint arXiv: XIX:11966006166 2019b.
Shuai Zhao, Liguang Zhou, Wenxiao Wang, Deng Cai, Tin Lun Lam, and Yangsheng Xu. Towards better accuracy-efficiency trade-offs: Divide and co-training. arXiv preprint arXiv:2011. 1.220011.4600 14660, 2020.
12

---

A APPENDIX
A.1 DATASETS
We evaluations four different datasets, namely CIFAR-10 and CIFAR-100 (Krizhevsky et al. 2009), the Mapillary Traffic Sign Dataset (MTSD) (Ertler et al.) 2020), and a rail defect dataset provided by Swiss Federal Railways (SBB). Additionally, used synthetic dataset consisting of two-dimensional data points. In the following, we explain the necessary preprocessing steps to create the publicly available MTSD dataset.
Mapillary Traffic Sign Dataset (MTSD) The Mapillary traffic sign dataset (Ertler et al. 2020) is large-scale vision dataset that includes 52' 000 fully annotated street-level images from all around the world. The dataset covers 400 known and other unknown traffic signs, resulting in over 2.5 000 traffic signs in total. Each street-level image manually annotated and includes ground truth bounding boxes that locate each traffic sign in the image, shown in Figure [a Further, each ground truth traffic sign annotation includes additional attributes such as ambiguousness or occlusion. Since the focus of this work is classification,
convert the base NTSD dataset to classification dataset (described below) by cropping to each ground truth bounding box. We show samples from the resulting cropped MTSD dataset in Figure 5b
(a) Base Mapillary Traffic Sign Dataset (MTSD). The ground truth bounding boxes are visualized in green.
information crossing
complementary chevron-right
requistory entry
complementary chevron-loft
yiold g1
Preprocessed Mapillary Traffic Sign Dataset (MTSD).
Figure 5: Illustration of Mapillary Traffic Sign Dataset (MTSD) samples. The base dataset consists of street-level images that include annotated ground truth bounding boxes locating the traffic signs (a). We convert the dataset to a classification task by cropping to the ground truth bounding boxes (b).
We convert the MTSD objection detection dataset into classification dataset as follows:
1. Ignore all bounding boxes that are annotated as occluded (sign partly occluded), out-of- frame (sign cut off by image border), exterior (sign includes other signs), ambiguous (sign is not classifiable at all), included (sign is part of another bigger sign), dummy (looks like sign but is not) (Ertler et al. 2020). Further, we ignore signs of class other-sign, since this is general class that includes any traffic sign with a label not within the MISD taxonomy.
2. Crop to all remaining bounding boxes and produce a labeled image classification dataset. Cropping done with slack, i.c. we crop to randomly upsized version of the original bound- ing box. Given a bounding box BB = ( ([rmin, #mar) [Umin: Mmazl), the corresponding upsized bounding box is given as
UBB ([xmén
Aae (Xman Imin), Imaz + A(1 (2))(mma - Imin))
(19)
[ymmin Aa (ymaz - Umin)) Imar + X(1 (ymox Fmnnn)
where Ckgy ag [ and À is the slack parameter, which set to A 1.0.
3. Resize cropped traffic signs to (64, 64).
MMo is the uniform distribution the interval [a, b).
13

---

Rail Defect Dataset (SBB) The rail defect dataset (SBB) is proprietary vision dataset collected and annotated by Swiss Federal Railways. It includes images of rails, each of which is annotated with ground truth bounding boxes for various types of rail defects. We note that all the models used in our work for this dataset are trained by the authors and not provided by SBB. In fact, for work, even consider a different type of task classification instead of the original object detection. As a consequence, the accuracy and robustness results presented in our work are by no means representative of the actual models used by SBB.
A.2 HYPERPARAMETERS
TRADES We use LTRADES (Zhang et al. (019a) to both train models from scratch and fine-tune existing models. When training models from scratch, we train for 100 epochs using LTRADES, with an initial learning rate le-1, which we reduce to le-2 and le-3, once 75% and 90% of the total epochs are completed. When fine-tuning models, we train for 50 epochs using CTRADES, with an initial learning rate Ie-3, which we reduce to le-4 once 75% of the total epochs are completed. We use batch size 200, use 10-step PGD (Madry et al. 2018) to generate adversarial examples during training. and set the 3 parameter in LTRADES to STRADES 6.0.
Empirical Robustness Abstain Training We fine-tune for 50 epochs using LERA (Equation 5). with an initial learning rate le-3, which we reduce le-4 once 75% of the total epochs are completed. We use batch size 200, use 10-step PGD (Madry et al. 2018) to generate adversarial examples during training, and set STRADES 6.0 for the loss term Lrob
LTRADES.
Certified Robustness Abstain Training We fine-tune for 50 epochs using LCRA (Equation D. with an initial learning rate 1e-3, which we reduce to le-4 once 75% of the total epochs are completed. We use batch size 50, k- = 16 ii.dd samples from NN T²),, and set the inverse softmax temperature to r = 4.0 (cf. Section
Probabilistic Certification via Randomized Smoothing We use the practical Monte Carlo al- gorithm by Cohen et al.] (2019) for randomized smoothing, using the same certification hyperpa- rameters as them. We use No 100 Monte Carlo samples to identify the most probable class CA. N = 100, 000 Monte Carlo samples to estimate a lower bound on the probability PA. and set the failure probability to 0.001.
Synthetic Dataset In Figure 1 we illustrate the effect of our training on a synthetic three-class dataset, where each class follows Gaussian distribution. We then simple four-layer neural network with 64 neurons per layer, and train it on N = 1000 synthetic samples, using Lstd- CTRADES (Zhang et al., 2019a), and CERA (Equation 5). For each loss variant, we train for 20 epochs. use fixed learning rate 1,- and batch size 10. For LIRADES and LERA, we use 10-step PGD (Madry al. 2018 to generate adversarial examples during training, and set STRADES 6.0.
A.3 Loss FUNCTION ABLATION STUDY
Additionally to the LERA loss from Equation training an empirical robustness indicator abstain model. The formulation based on the Deep Gamblers loss (Liu et al. 2019), which considers abstain model (Fa, S) with an explicit abstain class a as a selection mechanism. Since we consider robustness indicator selection, we replace the output probability of the abstain class (a), with the output probability of the most likely adversarial label. This corresponds to the probability of sample being non-robust and thus the probability of abstaining under robustness indicator selector. Similar to LERA, we also add the TRADES loss (Zhang et al. 2019a) to optimize robust accuracy. The resulting loss is then defined as:
consider alternative loss formulation for
LDGA (fo, (x, v)) = B LTRADES (fo, (x. v)) log (fo(#) + max.cy)((oott)) fo(x"). (20)
We conduct ablation study over the two loss functions, LERA and LDGA, for CIFAR-10 and E.c = 8/255 TRADES (Zhang et al. 2019a) trained ResNet-50 model. We fine-tune the model for foo perturbations of radi 1/253 and 2/255, using both LERA and LDGA, training for 50 epochs each and setting the regularization parameter 3 1.0. For each loss variant, we train the base model once without data augmentations and once using the AutoAugment (AA) policy (Cubuk et al. 2018).
14

---

CIFAR-10
D1/265
B
Pre-trained Model
Finetuning
Ryob
R
RTob
LerA
86,31
96.63
78.24
97.33
Zhang et al. 2019a
LDGA
84.98
94.92
75.73
96.22
(Resicet-50)
LERA AA
83,44
97.47
74.63
98.31
LDGA + AA
80.72
96.56
73.59
97.88
Table 5: Robust selection (R and robust accuracy (Roo)) of empirical robustness indicator abstain
models (F, SERI), trained using LERA (Equation 5 and LDGA (Equation 20
We show the robust accuracy and the robust selection of the resulting robustness indicator abstain models in Table [S] Observe that for all experiments, LERA trained models achieve consistently higher robust accuracy and higher robust selection, compared to LDGA trained models. For instance. when training for Eoo = 1/255 perturbations without data augmentations, LERA achieves +1.71% higher robust accuracy and + +1.33% higher robust selection, compared to LDGA- Similarly, when training with Augonug LERA achieves +0.91% % higher robust accuracy and +..72% higher robust selection. Similar results hold for Eoo = 3/255 perturbations.
A.4 COMPARING ADVERSARIES FOR SOFTMAX RESPONSE (SR)
Recall from Section 72 that evaluated the robustness of softmax response (SR) abstain models using APGDconf, which a modified version of APGD (Croce & Hein 2020) using the alternative adversarial attack objective by [Stutz (2020). This modified objective optimizes for adversarial example x' that maximizes the confidence in any label e * (a(x), instead of minimizing the confidence in the predicted label:
= arg araggmmm max max fo(t). EE (a) of
(21)
The resulting adversarial attack finds high confidence adversarial examples, and thus represents an effective attack against softmax response selector Sgg.
In the following, we conduct an ablation study over APGD and APGDconf by evaluating the robust selection R,ee and robust accuracy Roze of an SR abstain model (Fo, Saa) using both APGD and APGDconf. We use the adversarially trained WideResNee-22-110 model by Carmon et al. (0019) (taken from RobustBench (Croce et al. 20200), trained on CIFAR-10 for &oo = =8/255 perturbations. We then evaluate the classifier as an SR abstain model (Fo, Ssa) with varying threshold r E [0. 1), and report the robust selection and robust accuracy for varying &oo perturbations Figure 6 Observe that for small perturbations such as co = 1/255, APGD and APGDconf are mostly equivalent concerning robust selection and robust accuracy. However, for larger perturbations such as exx- = 1/255, the SR abstain model is significantly less robust to APGDconf than to standard APGD, showing the importance of choosing a suitable adversarial attack. High confidence adversarial examples are generally more likely be found for larger perturbations, thus an SR selector is significantly less robust to APGDconf than to APGD for larger perturbations.
Eoc 1/255
6o 2/255
Co 1/255
R 186)
APGD
APGDconf
14
135
04
14
[%)
rob
Figure 6: Robust selection (Rçel) and robust accuracy (Ro0e) for CIFAR-10 softmax response (SR) abstain models (F, SSR for varying threshold e [0, 1) and using the WideResNet-28. classifier
F' by Carmon et al. (2019). Each SR abstain model is evaluated via APGD (Croce Hein 2020) and
APGDconf (Equation 21
15

---

CIFAR-10
CIFAR-100
MTSD
883
Zhang al. B B1/258
Camnon el
Gowal et al.
Rebulli el
Zhang et al.
Zhang et al.
Robust Accorncy Roaah (%6)
Bi
B 2226
B1/255
B 1/26
B B//258
90
16
05
hetter
05
15
2.5
12
Robust Inaceuracy R [%6]
better
Figure 7: Robust accuracy (Roch) and robust inaccuracy (R of existing robust models (4, fine-tuned with our proposed loss (o, Our approach consistently reduces the number of robust inaccurate samples across various datasets, existing models and different regularization levels B.
A.5 ADDITIONAL EXPERIMENTS ON REDUCING ROBUST INACCURACY
In this section, we present additional experiments on reducing robust inaccuracy for empirical robustness.
Similar to the results in Figure 2 we show the robust accuracy (Rao)) and robust inaccuracy (R (R,05 (acc of different existing models fine-tuned with (4) and without (4) data augmentations, in Figure At the same time, Figure 7 |also shows the same models fine-tuned with our proposed loss with and without (O) data augmentations. We again observe that approach achieves consistently lower robust robust inaccuracy, compared to existing robust models. For example, on CIFAR-10 and for Bi/255. the model from Carmon et al. (019)) achieves 91.7% robust accuracy but also 1.8% robust inaccuracy. Using our loss LERA and varying the regularization term /3 we can obtain number of models that reduce robust inaccuracy to 0.14% while still achieving robust accuracy of 75.8%.
A.6 ADDITIONAL EXPERIMENTS USING ROBUSTNESS TO ABSTAIN
In this section, we present additional experiments on comparing different abstain approaches for empirical robustness.
We compare robustness indicator abstain models (F, SRI) using existing robust classifiers TRADESRI and classifiers fine-tuned with our proposed loss ERAgr. Further, we again consider softmax response and selection network abstain models, as described in Section 7.2] Equivalent to Section 77. we use the robust selection (Red). and ratio of non-abstained samples that are robust and accurate (Reot) as our evaluation metrics.
CIFAR-10
NTSD
Zhang et
2019a
Carmon et al. 2019)
Gooaal et 2020
Zhang al 2019a RcsNct-50,
RcaNot-50, 1,905
W WRN-28- 28-4-28-10
2/255
WRN-28-10
57255
Byaas
130.0
100
Accuracy Rease
25.0
60
60
42-
$6.0
57.5
25.0
80
60
96
12.5
40
6.0
30
30
40
50
60
70
80
20
40
20
60 70 50
50
75
20
Robust Selection Re [%)
better
Figure 8: Comparison of different abstain approaches including existing robust classifiers TRADES (a, 4), classifiers fine-tuned with our proposed loss ERARI (O, )), selection network (m, 2) and softmax response (o, abstain models. The higher Resel and Rec0 the better (top right corner is optimal).
16

---

CIFAR-1 -R0A
3BB
Zhang et al. 2019a
Carmon al. 2019
Gowal et al 2020
Zhang el 2019a
RosNet-50.
11225
WRN-28-10, 2/225
WRN-28- B2/25
ResNet-5U, 1/256
Robust Accaracy R2 [X])
72
52
52
50-
93
92
54
96
92
120
90
52
94
90
00
300
90
92
94
56
92
300
92.0
22.5
02.5
92.
93.0
Natural Accuracy R
[96]
betier
Figure 9: Natural (R and robust accuracy (Raah) for 2-compositional ERARI models (O, and 2-compositional TRADESa1 models (4, 4). Further, we also consider 2-compositional ACE-COLTsh: sB sw (m, )) and 2-compositional TRADESge (o. models. The core models used in the
compositional architectures are listed Appendix A.8
We show the comparison of the different abstain models in Figure Similar to the results in Section 7.2 again show that, as designed. approach consistently improves robust accuracy. For instance, consider the CIFAR-10 Zhang et al. (2019a) model at & = 1/255, trained without data augmentations (O). The ERART model with the highest robust selection R improves robust accuracy by +2.39% at the expense of -3.44% decrease in robust selection. This tradeoff is close optimal since our approach increases robust accuracy by correctly abstaining from mispredicted samples, thus an increase in robust accuracy results in corresponding decrease in robust selection. Further, we again observe that by varying the regularization parameter B, we can obtain Pareto front of optimal solutions. Considering the CIFAR-15 AR Zhang et al. (2019a) model at & 1/255, robust accuracy up to 99.75%, an increase
trained with data augmentations (O), we can improve of +4.38% compared to the corresponding TRADESaI model (4). However, this comes the expense of a disproportionally large decrease of -42.27% lower robust selection. We observe similar results for other models, datasets, and perturbations regions, shown in Figure g
Further, we again note that our approach mostly improves both robust selection and robust accuracy when compared to softmax response and selection network abstain models.
A.7 ADDITIONAL EXPERIMENTS ON BOOSTING ROBUSTNESS WITHOUT ACCURACY Loss
In this section, present additional results combining abstain models with state-of-the-art models trained to achieve high natural accuracy.
we put the abstain models trained far in 2-composition (Section 6 with the standard trained core models discussed in Appendix[A.3 [A.8] We show the natural (R.ar) and adversarial accuracy (Reas) the resulting 2-compositional architectures Figure
Equivalent Section 7.3
We again observe that 2-compositional architectures using models trained by our method (o, improve over existing methods that solely optimize for robust accuracy (a. 4). Further, our method mostly improves both the natural and robust accuracy, compared to 2-compositional architectures using softmax response (o, or selection network (m, @) abstain. For example, on and the Zhang et al. (2019a) model at 6 = 1/255, our approach (O) improves natural accuracy by +0.68% while decreasing the robust accuracy by only -1.54%.
Further, we show that 2-compositional architectures using models trained by our method achieve significantly higher robustness and mostly equivalent overall accuracy, compared to ofate-of-the- art non-compositional models trained for high natural accuracy. In Table 6 we show the natural (Rnat) and adversarial accuracy (Rao)) of our 2-compositional models and illustrate the accuracy improvement over the standard trained models discussed in Appendix A.8 For instance, consider CIFAR-10 10 at Exc 2/255 and the 2-compositional architecture using the Gowal et al. (2020) model as robust model F Frobust- Our model improves the robust accuracy by +75.3% 33.3% and the natural accuracy
17

---

by +0.1%, compared to the standard trained model by Zhao et al. 2020). Similar results hold for other models, datasets, and perturbation regions.
CIFAR AR- 10
CIFAR- 100
MTSD
SBB
Feore
Zhao ot al
2020
WideReattet-28-11
(BeeNet-500)
(Reallet-50)
robuut
Carmon et al.
Gowal et al.
Rebufli et al.
Zhang et al.
Zhang et al.
Ryob
86.5 (+00.35%
87.8 (161.96)
44.0 (x24.1%)
84.5 (+9.8%6)
88.4 (+12.75)
B1/25
Rnas
97.6 (902%)
98.0 (+0.2%)
80.5 (+0.2%)
94.1 (+0.3%)
92.3 (+0.9%)
R
73.4 (+700.5%)
78.2 (+75.396)
41.9
(+38899)
69.9 (+29.2%)
82.4 (+37,79)
22/255
Ryat
97.8 (+0.0%)
97.9 (40.15g)
80.18
(+0019))
94.0 (+0.2%)
91.3 6-015)
Table 6: Improvements 2-compositional architectures using models Frobust trained with our method over non-compositional models trained to optimize natural accuracy only (Appendix (.8).
A.8 CORE MODELS
Recall from Section 6 that an abstain model (F, S) can be enhanced by a core model F Foore. which makes a prediction on all abstained samples, resulting in 2-compositional architectures, In Section 7.3 we presented an evaluation of 2-compositional architectures, where we used state-of-the-an standard trained models as core models. In Table 7 we show the natural and adversarial accuracy of core models used in Section 7.3 for varying foo perturbation regions, where we use 40-step APGD (Croce & Hein 2020) to evaluate robustness.
acc [%]
Dataset
Model Foore
Rnat [%]
Bij255
2/2255
B0c
CIFAR-10
Zhao et al. 2020) (MideRestiet-40-11)
97.81
26.18
2.92
0.06
CIFAR-100
(WideReallet22--1)
80.17
19.9
3.06
0.15
MTSD
(Resllet-50)
93.79
74.66
40.71
7.51
SBB
(ResMet-550)
91.37
75.65
44.69
8.76
Table 7: Natural (Reat) and adversarial accuracy (Rag)) of standard trained core models, used in 2-compositional architectures Section 7.3 and Appendix A.7
A.9 ROBUSTNESS/ACCURAAC DATASET SPLITS
Consider robustness indicator abstain model (Fs, SAI) and a labeled dataset D = {<x, yd W on
which we evaluate the classifier Fo: : X' - V. Based on the robustness and accuracy of the classifier Fo. we can partition D into four disjoint subsets D = {D DF D Dー>^~a} }.
where: D ((xx)) v) D: Var E (z)).. Fo() -F((a) Fo(x) # y)
D {(x, v) D: Var E (x)). Fo())) Fo(a)) Fo(x) y)
D
Dr rra {( v) D: Brd E (x)).. Fo(x') * Fo(a)) Fo(x) y)
DFF ((x, D: 3rd e (g).). Fo(x') + Fo(a)) Fo(x)) # y}
We illustrate this dataset partitioning on the CIFAR-10 Krizhevsky
2009) dataset. We consider
a TRADES (Zhang et al.) 2019b) trained 50 and the WideResNet 28-10 models by Carmon al. (2019): Gowal et al. (2020) (taken from Robustbench (Croce et al. 20200), where each model adversarially pretrained for & = 8/255 and then fine-tuned via TRADES to the respective lx threat model illustrated Table 8 Further, we also consider a standard trained ResNet-50. We then evaluate the robustness and accuracy of each model using 40-step APGD (Croce & Hein 2020). Considering Table S, note that standard adversarial training methods do not necessarily eliminate the occurrence of robust inaccurate samples (x, 3) DEA- and that the robust inaccuracy generally increases for smaller perturbation regions. Further, we note that while standard trained models have low robust inaccuracy, they also have low overall robustness, resulting in low overall robust accuracy.
18

---

Threat Model
Relative Split Size [%6]
Data Split
Zhang et al. (ResNet-50)
Carmon et al.
Gowal et al.
Lard (ResNet-50)
(WRN-28-10)
(WRN-28-10)
5.17
3.33
2.85
6.97
Di,
4.64
3.61
2.88
0.0
11/255
6.18
3.32
3.87
74.89
84.01
89.74
90.40
18.14
Dig iTA "I
7.94
7.38
4.86
6.97
DFg
4.13
2.40
2.25
0.0
B3y255
DF,
10.38
3,20
6.74
91.80
77.55
87.02
86.15
1.23
F DTrAm
13.42
8.23
6.64
6.97
D D
3.31
1.05
0.87
0.0
B2/255
Diga
17.19
16.87
15.96
93.03
Di
66.08
73.85
76.53
0.0
Dig na
18.17
9.55
9.21
6.97
D DEAma
2.64
0.76
1.31
0.0
B8/255
DF,
29.79
27.82
23.78
93.03
D
49.40
61.87
65.70
0.0
Table 8: CIFAR-10 robustness- cesssteess-acuraayy dataset partitioning. We consider TRADES (Zhang et al. 2019a) trained ResNet-50, adversarially trained WideResNet-28.. 10 models (Carmon al. 2019 Gowal et al. 2020). and a standard trained ResNet-50. Adversarially trained models are trained for the respective perturbation region. Each model is evaluated for the indicated foo. threat model, using 40-step APGD (Croce & Hein 2020).
Further, we also illustrate the robustness-accuracy dataset partitioning on CIFAR-100 (Krizhessky et al. 2009). We consider standard trained WideResNNet-2--11 and the adversarially trained by |Rebulli et al. (202T). Again, the model by Rebuffi et al. (202 was pretrained for oe 8 /255 perturbations and then TRADES fine-tuned for the respective threat model indicated in
WideResNee-2285
Table 9 We again evaluate the robustness-acet -ccssteessaacuracy dataset partitioning for varying &oo perturbations
using 40-step APGD (Croce & Hein 2020 and list the exact size of each data split in Table
Notably, we observe that on the model by Rebuffi et al. 2021). 15.24% all test samples are robust but inaccurate for 6 1/255 perturbations, which is significantly larger fraction compared to similar models on CIFAR-10.
19

---

Threat Model
Relative Split Size [%]
Data Split
Rebuffi et al.
Lard (WRN-28-10)
(WRN-28-10)
Di
15.20
19.80
D
15.24
0.03
Bi/255
Fe
7.75
60.27
DEAa
61.81
19.9
Die Drrr
32.75
19.82
Dir
8.71
0.01
182/255
D
5.11
77.11
D
53.43
3.06
D
30.57
19.83
D a
4.34
0.0
B1/255
Dr
23.16
80.02
D
41.93
0.15
D ma
33.70
19.83
DrAme
3.91
0.0
B87/20
Darha
26.66
80.17
35.73
0.0
Table 9: CIFAR-100 polustness-accurace dataset partitioning. We consider a standard trained WedeResNet 10 and the adversarially trained WideResNet-228. 28-100 by Rebuffi al. (202)). trained for the respective perturbation region considered each evaluation. Each model is evaluated for the indicated for threat model, using 40-step APGD (Croce & Hein 2020).
20


Reviews:
[{"review_id": "ginqjzskreZ", "reviewer": "Reviewer_wtee", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The abstract assumes the reader is aware that the paper will deal with adversarial DL, it would be better to \"set the stage\" a bit.\nWithout giving the setting of robustness to adversarial examples, the abstract is not clear to me, as it is using a lot of terms and concepts that will only become clear during the paper, such as: \"robust inaccuracy\", \"$\\epsilon_\\infty$ robustness\" and \"certified robustness\".\n\nTable 1 seems to be a justification why the approach presented in the paper is needed. However not much details are given on how to interpret the results shown and how the values were generated.\n\nThe datasets are only explained & cited in section 7, however the abbreviations are used already in the introduction. Either cite them correctly and declare the abbreviations at first occurrence or do not mention them explicitly in the introduction.\n\nInstead of mixing experiments (on synthetic data) and partial method descriptions with the introduction, i would favor a more detailed explanation of the introduced concepts and presented experiments. In the introduction i would suggest to focus on the gap in previous works, while keeping the experimental investigations for the respective chapter and providing more details there.\n\nIn the related work section there are several claims and properties of the proposed approach mentioned, that are not yet explained or proven, i would be better to at least point the reader to where that will be done.\n\nThe important concepts of \"natural accuracy\", \"robust accuracy\" and \"robust inaccuracy\" are only explained in section 3 but used to a large extend prior to that section. At least reference the explanation coming at a later stage or consider rearranging the order by which concepts are introduced. First introduce the concept than use it to explain the novelty or justification of the proposed approach.\n\nMinor point, but similar issue: indicator function introduced in section 4, but already used in section 3.\n\nThe compositional architecture seems like ensembling with weighting based on selector S. It would be helpful to relate to that.\n\nIllustration in figure 2 seems rather cluttered, difficult to grasp the gain by the proposed method.\n\n## Questions to the authors\nHave you compared fine-tuning existing models with training them from scratch with the proposed loss?\nWhy would the method be restricted to fine-tuning?", "summary_of_the_paper": "The paper addresses the issue of trained models being inaccurate but robust for some samples. To address this issue a flexible fine-tuning mechanism is proposed. A robustness-based ensembling method is introduced as well. ", "summary_of_the_review": "In summary a few things are a bit unclear, e.g. to me why the method would be restricted to fine-tuning. Can certainly be improved by re-structuring, some suggestions given above. In the current state for me a bit hard to grasp the contribution. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "X3Xz9hKs60", "reviewer": "Reviewer_sDbQ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. It's interesting to improve robust training methods by avoiding robust inaccuracy, but the proposed method is a little bit straightforward. \n2. I think more discussion and analysis should be provided to explain the reason behind the effectiveness of the proposed method, empirically or theoretically. To be clear, I'm not requesting the authors to provide theoretical justification, and just suggest providing more insights and analysis.\n3. The Figure2, 3, 4 can be improved, and I was confused about what did the curves represent at first.", "summary_of_the_paper": "To tackle the two limitations of the current robust training algorithms: 1). robust inaccuracy 2). the sacrifice of natural accuracy, this paper proposed a new training method that aims to maximize robust accuracy and minimize robust inaccuracy. Moreover, a robustness-based abstain mechanism is adopted to further boost overall robustness without sacrificing accuracy. Experiments show the effectiveness of the algorithm in terms of fewer robust and inaccurate samples and better robustness, with only marginally reduced natural accuracy.", "summary_of_the_review": "From my perspective, the proposed approach is straightforward and not fully explained. Thus I tend to reject this work. But I am open with my score, according to the authors' responses and other reviewers' comments.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "WIaImGDer8f", "reviewer": "Reviewer_xSkN", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I found it difficult to understand this paper. The goal is not well motivated, the proposed terminology is confusing, and I'm not sure what the takeaway is from the results.\n\nFirst of all, the abstract mentions numbers without sufficient context. It is not even stated which benchmark this sentence refers to: \"natural accuracy from 97.8% to 97.6%\". After going through the paper, one can infer that they were referring to CIFAR-10, but even then the model name and training settings are not mentioned (either on table 3 or section 7.3). I see that one could follow the reference or the appendix, but basic information such as this one should be included in the main text. \n\nSimilarly, the phrase \"robust inaccuracy\" is not defined until page 3, even though it is mentioned in the abstract and the introduction. I would suggest authors to define robust inaccuracy, even if informally, earlier in the paper. My personal guess is that a phrase like \"mistake consistency\" or \"prediction consistency\" would be less confusing than the phrase \"robust inaccuracy\". \n\nI would also suggest the authors try to connect the \"robust inaccuracy\" aspect of their work with previous work. The related work section mainly touches on the \"abstain option\" and the \"robustness vs accuracy tradeoff\". Regarding the \"robust inaccuracy\", I'm not sure if this concept as been studied with that name before. But relevant work includes prediction consistency of models in semi-supervised learning and adversarial robustness of models trained on random labels. It is surprising (and relevant to this paper) that models trained on random labels have similar behavior w.r.t. adversarial attacks as models trained on regular data. Thus the adversarial robustness on a sample may not depend too much on whether the sample itself is classified correctly or incorrectly by the model.\n\nMy biggest concern about this submission is that I do not see the motivation of studying the \"robustness of inaccuracy\" on samples that are already classified incorrectly. The adversarial direction is defined as the direction that increases the loss, so it's not surprising that the predictions are still incorrect after the adversarial direction is added to the sample. An interesting study could look at the dynamics of going from one incorrect prediction to another one, but that is not explored in this submission. \n\n", "summary_of_the_paper": "This submission aims to reduce the prevalence of samples for which a neural network might predict the wrong answer, not just for the sample but also its nearby points in the input space. Authors call this metric robust inaccuracy.\n\nNext, after reducing the robust inaccuracy metric, they take advantage of their models to improve conventional robustness, by taking advantage of their more reliable robustness metric to abstain from samples for which model prediction is not robust (perhaps better to call this consistent, since it can refer to both correct and incorrect predictions). ", "summary_of_the_review": "I found that the goal of the paper is not well motivated and the presentation should be improved before it can be accepted. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IipQiZS6tdi", "reviewer": "Reviewer_7Sk1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- The accuracy-robustness trade-off is a highly relevant issue. The emphasis of this work to improve robustness without sacrificing the accuracy is compelling. \n- Very nice and accessible presentation: the goal of the work is well motivated, the technical steps are explained in sufficient depth.\n- The implementation of the robust inaccuracy penalty spans both adversarial and certified training.\n- The experiments are of appropriate scope, contain sufficient detail for reproducibility, and substantiate the claims about the proposed approach well.\n- Although most of the presented technical aspects featured in previous work, their composition seems carefully designed and sufficiently novel.\n\nCons:\n- Although the problem formulation seems novel (Eq. 4), its implementation under both adversarial and certified scenarios of training are closely based on prior work. Similarly for compositional models (Eq. 16). This is appropriately cited, hence does not present a serious issue, but does somewhat limit the novelty.\n\nA more general concern is that the experimental results often do not yet provide as much insight as they potentially could. The showed improvements on one metric often come at a cost of degrading the other. The analysis focuses almost exclusively on the improvement, and I wish the existing trade-off deserved more discussion and analysis in the main text. For example:\n- From Fig. 2, the improvement of robust inaccuracy comes at the expense (in part significant) of robust accuracy. Since robust inaccuracy affects only a fraction of samples (cf. Tab. 1), such cost may outweigh the benefits of the proposed regularisation.  \n- In Tab. 2, $\\mathcal{L}_{\\text{CRA}}$ may also decrease the robust sample selection w.r.t. the baseline for Cohen et al. Why may this be the case and why is it not the case of Sehwag et al.?\n\nMinor:\n* Although this work relates to adversarial robustness, one could relate the issue of robust inaccuracy to model calibration: the model trained with a robust objective may end up overconfident on misclassified examples. The proposed regularisation, in a way, addresses the issue of miscalibration from the perspective of robustness. Nevertheless, I wonder if and how the proposed approach affects the quality of model calibration.\n* While I follow the illustrative goal of Fig. 1,  I wonder why is it reasonable to compare the standard model with 2 (!)  decision boundaries to other models with only one. \n* Fig. 2 could be made more illustrative (e.g. by reducing the scale of the y-axis);\n* Sec. 7.1, certified robustness: It’s at least necessary to summarise the results in the main text, if there is no space to show the results in full. \n* The alternative formulation ($\\mathcal{L}_{\\text{DGA}}$ loss) does not seem to be beneficial. I’d suggest that it is discussed in the appendix, while the allocated space is to address more important concerns.\n* Conclusion is too short. It’d be worthwhile to point out existing limitations, and to offer a perspective on how future works may leverage the obtained results.\n", "summary_of_the_paper": "This well presented work is motivated by the relevant issue that improving model robustness while maintaining high accuracy can result in robust inaccuracy, i.e. a non-negligible amount of samples get classified incorrectly, but the category remains consistent for their perturbations. Supporting this observation empirically, this work proposes an approach to address this issue.\n\nThere are two main technical contributions. The first complements the standard robustness objectives from the literature with a regularisation term penalising robust inaccuracy. The paper provides specific implementations of this term under two setups: empirical (adversarial) and certified robustness. The second contribution is an abstain model: a learned indicator function that decides which samples are to be processed by the robust (but less accurate) classifier, and which ones should default to the non-robust (but accurate) baseline. As a result, this two-stage approach achieves high robustness without compromising the classification accuracy.\nThe experiments on standard benchmarks further confirm, that the robust model consistently decreases the fraction of robust inaccurate samples (albeit at the expense to robust accuracy).", "summary_of_the_review": "This work raises and addresses an interesting artefact of robust training, which was disregarded in previous work. The proposed approach seems technically sound, with a few novel elements. Although the analysis could be stonger, it nevertheless provides sufficient evidence for improved accuracy-robustness trade-off over state-of-the-art methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents the problem of robust inaccuracy (model predictions being robust to perturbations but inaccurate on datapoints), and present methods to maximize robustness while avoiding robust inaccuracy. Furthermore, they develop an abstention mechanism based on robustness to prevent prediction on points where the model is not robust. Results show improvement in adversarial robustness to standard attacks with only small reduction in natural accuracy. 

Reviewers were mixed on the clarity and importance of this submission. A major concern raised was on the importance of robust inaccuracy, motivation for avoiding it, and novelty of the proposed method. Other abstention mechanisms are available and one does not solely need to rely on robustness. Additionally, results are often presented on a pareto front and the method does not strictly dominate prior approaches. Authors addressed many of the clarity concerns in their updated revision, and reviewers commented on the high quality of analysis performed in the experiments. But several reviewers still found the draft and description of the robust inaccuracy problem insufficiently motivated and the methodology not well explained. Given lingering concerns over clarity and motivation (in spite of a revision that exceeds the page limit), I cannot recommend this paper for acceptance.
Decision: Reject