A VOIDING R OBUST M ISCLASSIFICATIONS FOR I MPROVED R OBUSTNESS WITHOUT A CCURACY L OSS
Anonymous authors
Paper under double-blind review
A BSTRACT
While current methods for training robust deep learning models optimize robust accuracy, in practice, the resulting models are often both robust and inaccurate on numerous samples, providing a false sense of safety for those.
Further, they signiﬁcantly reduce natural accuracy, which hinders the adoption in practice.
In this work, we address both of these challenges by extending prior works in three main directions.
First, we propose a new training method that jointly maximizes robust accuracy and minimizes robust inaccuracy.
Second, since the resulting models are trained to be robust only if they are accurate, we leverage robustness as a principled abstain mechanism.
Finally, this abstain mechanism allows us to combine models in a compositional architecture that signiﬁcantly boosts overall robustness without sacriﬁcing accuracy.
We demonstrate the effectiveness of our approach to both empirical and certiﬁed robustness on six recent state-of-the-art models and using several datasets.
For example, on CIFAR - 10 with ε ∞ = 1 / 255 , it successfully enhanced the robust accuracy of a state-of-the-art standard trained model from 26% to 86% while only marginally reducing its natural accuracy from 97.8% to 97.6%.
I NTRODUCTION
In recent years, there has been a signiﬁcant amount of work that studies and improves both adversarial (Szegedy et al., 2013; Goodfellow et al., 2014; Carlini & Wagner, 2017; Croce & Hein, 2020;
Madry et al., 2018) and certiﬁed robustness (Balunovic & Vechev, 2019; Cohen et al., 2019;
Salman et al., 2019; Xu et al., 2020; Zhai et al., 2020; Zhang et al., 2019b) of neural networks.
However, currently, there are two key limitations that hinder the wider adoption of robust models in practice.
Existing Models are Robustly Inaccurate First, despite substantial progress in training robust models, existing works usually only report robust accuracy, i.e., samples for which the model robustly predicts the correct label.
Meanwhile, the issue of robust inaccuracy,
i
[Table]
e., samples that are robustly misclassiﬁed with a wrong label, is usually not even reported (we formally deﬁne robust inaccuracy in Equation 3).
This is especially problematic for safety-critical models, where the robustness can be mistakenly used as a safety argument.
We quantify the severity of this issue in Table 1, by evaluating recent state-of-the-art robust models.
As can be seen, recent models contain up to 15% of robust inaccurate samples and the ratio of such samples worsens with smaller perturbation regions.
[Table]
[Figure]
Note that all three models predict over all three classes, however, the decision regions for class 2 of the L TRADES and L ERA trained models happen to be too small to be visible.
The considered model architecture and the training hyperparameters are provided in Appendix A.2.
Robustness vs Accuracy Tradeoff Second, existing robust training methods improve the model robustness, but they also typically degrade the standard accuracy on unperturbed inputs.
To address this limitation, a number of recent works study this issue in detail and propose new methods to mitigate it (Mueller et al., 2020; Raghunathan et al., 2020;
Yang et al., 2020; Stutz et al., 2019).
Our Work In this work, we advance the line of work that aims to boost robustness without sacriﬁcing accuracy, but we approach the problem from a new perspective – by avoiding robust inaccuracy.
Concretely, we propose a new training method that jointly maximizes robust accuracy while minimizing robust inaccuracy.
We illustrate the effect of our training on a synthetic dataset (described in
Appendix A.2) in Figure 1, showing the decision boundaries of three models, trained using standard training L std , adversarial training L TRADES (Zhang et al., 2019a), and our training L ERA (Equation 5).
First, observe that while the L std trained model achieves 100% accuracy, only 91 .
1% of these samples are robust (and accurate).
When using L TRADES , we can observe the robustness vs accuracy tradeoff – the robust accuracy improves to 98 .
4% at the expense of 1 . 6% (robust) inaccuracy.
In contrast, using our L ERA , we retain the high robust accuracy of 98 .
but avoid all robust inaccurate samples by appropriately shifting the decision boundary, rendering them non-robust.
Second, since our models are trained to be robust only if they are accurate, we leverage robustness as a principled abstain mechanism.
This abstain mechanism then allows us to combine models in a compositional architecture that signiﬁcantly boosts overall robustness without sacriﬁcing accuracy.
Concretely, in Figure 1, we would deﬁne a selector model that abstains on all non-robust samples.
Then, the abstained (non-robust) samples are evaluated by the standard trained model L std , while the selected samples are evaluated using the robust model L ERA .
This allows us to achieve the best of both models – high robust accuracy ( 98 . 4% ), high natural accuracy ( 100% ), and no robust inaccuracy.
We show the practical effectiveness of our approach by instantiating over several datasets and existing robust models for both empirical and certiﬁed robustness.
Our evaluations show that our method effectively reduces robust and inaccurate samples by up to 97.28%.
Further, our approach signiﬁcantly improves robustness for CIFAR - 10 , CIFAR - 100 , MTSD and SBB datasets by 60.3%, 38.8%, 29.2% and 37.7%, respectively, while simultaneously decreasing natural accuracy by only 0.2%.
R ELATED W ORK
There is a growing body of work that extends models with an abstain option, either by training a selector mechanism separately or jointly together with the model.
The existing approaches include various selection mechanisms such as entropy selection (Mueller et al., 2020), selection function (Cortes et al., 2016; Mueller et al., 2020; Geifman & El-Yaniv, 2019), softmax response (Stutz et al., 2020; Geifman & El-Yaniv, 2017), or explicit abstain class (Laidlaw & Feizi, 2019; Liu et al., 2019).
In our work, we explore an alternative selector mechanism that uses model robustness.
The advantage of this formulation is that the selector provides strong guarantees for each sample and can never produce false-positive selections.
The disadvantage is that it introduces a signiﬁcant runtime overhead, compared to many other methods that require only a single forward pass.
At the same time, several recent works investigated the robustness and accuracy tradeoff both theoretically (Yang et al., 2020; Dobriban et al., 2020) and practically by proposing new methods to mitigate it.
For example, Raghunathan et al.
(2020) proposes robust self-training that leverages unlabeled data to regularize the model.
Stutz et al.
(2019) considers a new method based on onmanifold adversarial examples, which are more aligned with the true data distribution than the (cid:96) p -norm noise models.
Mueller et al.
(2020) focuses on deterministic certiﬁcation and also proposes using compositional models to control the robustness and accuracy tradeoff.
In our work, we also take advantage of compositional models, but we focus on both empirical and probabilistic certiﬁed robustness.
Further, our selector formulation is based on a new training that minimizes robust inaccuracy and can be used to ﬁne-tune any existing robust model.
Finally, we provide individual robustness at inference time, rather than distributional robustness considered in prior works.
P RELIMINARIES
Let f θ : R d → R k be a neural network which classiﬁes inputs x ∈X ⊆
R d to outputs R k (e.g., logits or probabilities).
The hard classiﬁer induced by the network is given as F θ ( x ) =
arg max i ∈Y f θ ( x ) i , where f θ ( x ) i is the network output for the i -th class and Y , |Y| =
k is the ﬁnite set of discrete labels.
Natural Accuracy Given a distribution over input-label pairs D and a classiﬁer F θ : X → Y , an input-label pair ( x , y ) is considered accurate iff the classiﬁer F θ predicts the correct label y for x :
[Equation]
Robust Accuracy Given an input-label pair ( x , y ) , we say that the classiﬁer F θ is robust and accurate iff it predicts the correct label y for all samples from a predeﬁned region B pε ( x ) , such as a (cid:96) p -norm ball centered at x with radius ε , i.e., B pε ( x ) ..
= { x (cid:48) :
|| x (cid:48) − x || p ≤ ε } .
Formally:
[Equation]
Robust Inaccuracy Similarly to robust accuracy, an input-label pair ( x , y ) is considered robustly inaccurate iff the classiﬁer F θ predicts an incorrect label F θ ( x ) (cid:54) =
y and F θ is robust towards that misprediction for all inputs in B pε ( x ) .
Formally, the robust inaccuracy is deﬁned as:
[Equation]
R EDUCING R OBUST I NACCURACY : A DVERSARIAL & C ERTIFIED T RAINING
In this section, we present our training method that extends existing robust training approaches by also considering samples that are robust but inaccurate.
We start by describing a high-level problem statement which we then instantiate for both empirical robustness as well as certiﬁed robustness.
Problem Statement Given a distribution over input-label pairs D , our goal is to ﬁnd model parameters θ such that the resulting model maximizes robust accuracy, while at the same time minimizing robust inaccuracy.
Concretely, this translates to the following optimization objective:
[Equation]
where β ∈ R
+ is a regularization term, 1 { F θ ( x ) (cid:54) =
y } is an indicator function denoting samples for which the model is inaccurate, and L rob ( x , y ) with L ¬ acc rob ( x , y ) are loss functions that optimize robust accuracy and penalize robust inaccuracy, respectively.
Here, the ﬁrst loss function L rob ( x , y ) is standard and can be directly instantiated using existing approaches (see next).
The main challenge comes in deﬁning the second loss term, as well as ensuring that the resulting formulation is easy to optimize, e.g., by deﬁning a smooth approximation of the non-differentiable indicator function.
4. A DVERSARIAL T RAINING
We instantiate the loss function from Equation 4 when training empirically robust models as follows:
[Equation]
Below, we introduce each term in more detail and discuss the motivation behind our formulation.
L rob
To instantiate L rob , we can use any existing adversarial training method.
For example, considering standard adversarial training (Goodfellow et al., 2014) would result in the following loss:
[Equation]
where (cid:96) CE is cross-entropy loss of the worst example in the allowed perturbation region B pε .
Similarly, we can instantiate the loss using more sophisticated methods, such as TRADES (Zhang et al., 2019a):
[Equation]
where D KL is the Kullback-Leibler divergence (Kullback & Leibler, 1951).
1 { F θ ( x ) (cid:54) =
y } Next, we consider the indicator function, which encourages learning on inaccurate samples.
Since the indicator function is computationally intractable, we replace the hard qualiﬁer by a soft qualiﬁer 1 − f θ ( x ) y .
The soft qualiﬁer will be small for accurate and large for inaccurate samples, thus providing a smooth approximation of the original indicator function.
L ¬ acc rob
Third, we deﬁne the loss that penalizes robust but inaccurate samples.
This can be formulated similar to the adversarial training objective (Madry et al., 2018), however, instead of optimizing the prediction of the adversarial example f θ ( x (cid:48) ) towards the correct label y , we optimize towards the most likely adversarial label arg max c ∈Y\{ F θ ( x ) } f θ ( x (cid:48) ) c .
This leads to the following formulation:
[Equation]
Note that the purpose of the L ¬ acc rob loss is to penalize robustness by making the model non-robust.
As a result, it is sufﬁcient to consider only a single non-robust example, thus the minimization (rather than maximization) in the loss objective 1 .
4. C ERTIFIED T RAINING
Similarly to Section 4.1, we now instantiate the loss function from Equation 4 for probabilistic certiﬁed robustness via randomized smoothing (Cohen et al., 2019).
Randomized smoothing constructs a smoothed classiﬁer G θ : X → Y from a base classiﬁer F θ , where G θ ( x ) predicts the class which
F θ is most likely to return when x is perturbed under isotropic Gaussian noise.
Our proposed instantiation of Equation 4 for probabilistic certiﬁed robustness is as follows:
[Equation]
where η 1 , ..., η k are k i.i.d.
samples from N (0 , σ 2 I ) .
Note that, since the robustness guarantees provided by randomized smoothing hold for the smoothed classiﬁer G θ , the three loss components from Equation 4 need to be formulated with respect to the smoothed classiﬁer G θ .
L rob
To instantiate L rob , we can use any existing certiﬁed training method for randomized smoothing, such as the methods deﬁned by Cohen et al.
(2019) or Zhai et al.
Concretely, when using
Cohen et al.
, the loss is deﬁned using Gaussian noise augmentation:
[Equation]
) (10) 1 { F θ ( x ) (cid:54) =
y } We again replace the computationally intractable hard qualiﬁer by a soft qualiﬁer E δ ∼N (0 ,σ 2 I ) [1 − f θ ( x + δ ) y ] , which encodes the misprediction probability of the smoothed classiﬁer.
In practice, we approximate expectations over Gaussians via Monte Carlo sampling, thus leading to the approximated soft inaccuracy qualiﬁer 1 / k (cid:80) kj =1 1 − f θ ( x + η j ) y .
[Equation]
Finally, we instantiate the L ¬ acc rob loss term, which encourages the model toward non-robust predictions on robust but inaccurate samples.
We propose to minimize robustness by directly minimizing the certiﬁed radius of the smoothed classiﬁer G θ .
The certiﬁed radius formulation by
Cohen et al.
(2019) involves a sum of indicator functions, which is not differentiable.
However, Zhai et al.
(2020) have recently proposed the following differentiable certiﬁed radius formulation:
[Equation]
where Φ − 1 is the inverse of the standard Gaussian CDF, Γ is the inverse softmax temperature multiplied with the logits of f θ , and η 1: k are k i.i.d.
samples from N (0 , σ 2 I ) .
Note that, by setting the loss term L ¬ acc rob to CR ( f θ , ( x , y )) , we directly penalize robustness of the smoothed classiﬁer G θ .
R OBUST A BSTAIN M ODELS
In this section, we extend the models trained so far by leveraging robustness as a principled abstain mechanism.
Further, we deﬁne an additional loss function based on the Deep Gamblers loss (Liu et al., 2019), which is speciﬁcally designed for training adversarially robust abstain models.
Abstain Model Consider an input space X ⊆ R d and a label space Y .
A model with an abstain option (El-Yaniv et al., 2010) is a pair of functions ( F θ , S ) , where F θ : X → Y is a classiﬁer and
S : X → { 0 , 1 } is a selection mechanism, which acts as a binary qualiﬁer for F θ .
Let S ( x ) =indicate that the model abstains on input x ∈ X , while S ( x ) = 1 indicates that the model commits to the classiﬁer F θ for input x and predicts F θ ( x ) .
Robustness Indicator Selector We now instantiate abstain models with a robustness indicator selector, that abstains on all non-robust samples.
For adversarial robustness, the selector is deﬁned as:
[Equation] [Equation]
For certiﬁed robustness, the selector is deﬁned as:
[Equation]
[Equation]
Robustness Guarantees: Robust Selection Similar to robust accuracy, the robustness of an abstain model needs to be evaluated with respect to a threat model.
In our work, we consider the same threat model as for the underlying model F θ , namely B pε ( x ) ..
= { x (cid:48) :
|| x (cid:48) − x || p ≤ ε } , a (cid:96) p -norm ball centered at x with radius ε .
Then, we deﬁne the robust selection of an abstain model as follows:
[Equation]
That is, we say that a model is robustly selecting x if the selector S would select all valid perturbations x (cid:48)
∈
B p ε ( x ) .
When used together with our deﬁnition of S ERI , we obtain the following criterion:
[Equation]
In other words, to guarantee that the selector S ERI is robust for all x (cid:48) ∈ B pε ( x ) , we in fact need to check robustness of the model F θ to double that region x (cid:48)
∈ B p 2 · ε ( x ) .
This is important in order to obtain the correct guarantees and is reﬂected in our evaluation in Section 7.
Note that when evaluating robust selection for certiﬁed training, it is sufﬁcient to show that the smoothed model G θ can be certiﬁed with a radius R ≥ ε .
Then, the smoothed model guarantees that
G θ ( x (cid:48) )
=
c A for all x (cid:48) ∈ B pε ( x ) , which is equivalent to our condition ∀ x (cid:48) ∈B ( x ): G θ ( x (cid:48) )
=
G θ ( x ) .
B OOSTING R OBUSTNESS WITHOUT A CCURACY L OSS
Consider an abstain model ( F θ , S ) and let D be a dataset to evaluate ( F θ , S ) .
The selector S partitions
D into two disjoint subsets – the set of abstained inputs D ¬ s and the set of selected inputs D s for which F θ makes a prediction.
However, depending on the application, it may be desirable to make a best-effort prediction on all samples, including D ¬ s .
This insight leads to compositional architectures, already used by a number of prior works (Mueller et al., 2020; Wong et al., 2018).
Let H =
(( F robust , S ) , F core ) be a 2-compositional architecture consisting of a selection mechanism S , a robustly trained model F robust , and a core model F core .
Given an input x ∈ X , the selector S decides whether the model is conﬁdent on x and commits to the robust model F robust or whether the model should abstain and fall back to the core model F core .
Formally:
[Equation]
Note that each model of a compositional architecture can be chosen arbitrarily, regardless of the model’s network architecture or its training.
However, one beneﬁt of using a compositional architecture is combining models that complement each other, resulting in overall improved model performance.
In our work, we combine models trained via adversarial or certiﬁed training (which typically have lower natural accuracy), with models trained using standard training (which have high natural accuracy but low robustness).
The compositional model performance then depends on the quality of the selector S , used to determine which model to evaluate for each sample.
E VALUATION
In this section, we present a thorough evaluation of our approach by instantiating it to four different datasets, six recent state-of-the-art models, for both adversarial and certiﬁed robustness, and including top-trained models from RobustBench (Croce et al., 2020).
We show the following key results:
• Fine-tuning existing models with our proposed loss successfully decreases robust inaccuracy and provides a Pareto front of models with different robustness tradeoffs.
• Combining our proposed loss and robustness as an abstain mechanism leads to higher robust selection and accuracy compared to softmax response and selection network baselines.
• Our 2-compositional models signiﬁcantly improve robustness by
up to +60% while retaining the natural accuracy, causing only minor decrease of up to − 0 .
2% (for B ∞ 1 / 255 and B ∞ 2 / 255 ).
We perform all experiments on a single GeForce RTX 3090 GPU and use PyTorch (Paszke et al., 2019) for our implementation.
The hyperparameters used for our experiments are provided in Appendix A.2.
Models Our proposed training method requires neither retraining classiﬁers from scratch nor modiﬁcations to existing classiﬁers, thus our approach can be applied to ﬁne-tune a wide range of existing models 2 .
To demonstrate this, we use the following robust pre-trained models:
For empirical robustness , we evaluate ﬁve existing models from Carmon et al.
(2019), Gowal et al.
(2020), Rebufﬁ et al.
(2021), Sehwag et al.
(2021) and Zhang et al.
(2019a), where all but the last model are top models taken from RobustBench (Croce et al., 2020).
The model by Sehwag et al.
is trained for ε 2 = 0 .
5 , while the other models are trained for ε ∞ = 8 / 255 .
In our evaluation, we ﬁne-tune each model for 50 epochs for the considered threat model ( ε ∞ ∈ { 1 / 255 , 2 / 255 , 4 / 255 } and ε 2 ∈ { 0 .
using both the L TRADES loss (Zhang et al., 2019a) and our proposed L ERA loss.
For certiﬁed robustness , we use the σ = 0 .
12 Gaussian noise augmentation trained model by Cohen et al.
(2019) and the ε 2 = 0 .
50 adversarially trained model by Sehwag et al.
Similarly to empirical robustness, we ﬁne-tune the models for 50 epochs using either Gaussian noise augmentation training (Cohen et al., 2019) or the L CRA loss proposed in our work.
Datasets We evaluate our approach on two academic datasets – CIFAR - 10 and CIFAR -(Krizhevsky et al., 2009), and two commercial datasets – Mapillary Trafﬁc Sign Dataset ( MTSD )
(Ertler et al., 2020) and a Rail Defect Dataset kindly provided by Swiss Federal Railways ( SBB ).
We provide full details, including example images and all preprocessings steps, in the Appendix A.1.
When training on the CIFAR - 10 and CIFAR - 100 datasets, we use the AutoAugment ( AA ) policy by
Cubuk et al.
as the image augmentation.
For the MTSD and SBB datasets, we use standard image augmentations ( SA ) consisting of random cropping, color jitter, and random translation and rotation.
For completeness, our evaluation also includes models trained without any data augmentations.
[Figure]
B ∞ 2 /
Zhang et al.
R obu s t A cc u r ac y
R a cc r o b [ % ]
B ∞ 2 /
Carmon et al.
B ∞ 2 /
Gowal et al.
CIFAR0.0 2.5 5.0 7.
B ∞ 2 /
Rebufﬁ et al.
CIFAR
B ∞ 2 /
Zhang et al.
MTSD
B ∞ 2 /
Zhang et al.
SBB
[Figure]
Metrics We use the natural accuracy, robust accuracy, and robust inaccuracy as our main evaluation metrics, as deﬁned in Section 3, but evaluated on the corresponding test dataset D
= { ( x i , y i )
Ni =1 } .
When evaluating the empirical robustness, we use 40-step APGD (Croce & Hein, 2020) to evaluate the robustness of the classiﬁer F θ .
To evaluate certiﬁed robustness, we use the Monte Carlo algorithm for randomized smoothing from Cohen et al.
We certify 500 test samples and use the same randomized smoothing hyperparameters as Cohen et al.
(cf.
Appendix A.2).
7. R EDUCING R OBUST I NACCURACY
Empirical Robustness For empirically robust models, the results in Figure 2 show the robust accuracy ( R accrob ) and robust inaccuracy ( R ¬ acc rob ) of different existing models ﬁne-tuned with ( ) and without ( ) data augmentations.
At the same time, Figure 2 also shows the same models ﬁne-tuned with our proposed loss with ( ) and without ( ) data augmentations.
We can see that our approach improves over the existing models across all the datasets.
For example, on CIFAR -
B ∞ the model from Carmon et al.
(2019) achieves 86.5% robust accuracy, but also robust inaccuracy of 1.34%.
In contrast, using our loss L ERA , we can obtain a number of models that reduce robust inaccuracy to 0.29%, while still achieving robustness of 83.8%.
Similar results are obtained for other models, datasets, and perturbation regions (cf.
Appendix A.5).
We generally observe that our approach achieves consistently lower robust inaccuracy compared to adversarial training.
Further, by varying the regularization term β , we obtain a Pareto front of optimal solutions.
Certiﬁed Robustness Similarly, we show the robust accuracy ( R accrob ) and robust inaccuracy ( R ¬ acc rob ) for certiﬁably robust models, which were ﬁne-tuned using Gaussian noise augmentation L noise and using our proposed loss function L CRA .
In Table 2, we show results on CIFAR - 10 for B 20 .
12 and
B 20 .
25 perturbation regions.
We can see that our approach achieves lower robust inaccuracy compared to existing models.
For example, on CIFAR - 10 and B 20 . 25 , the Cohen et al.
(2019) model achieves 62% robust accuracy, but also 1% robust inaccuracy.
In contrast, our approach reduces the robust inaccuracy to 0.4% while still achieving 53.8% robust accuracy.
For the Sehwag et al.
(2021) model, our approach even improves both the robust accuracy and the robust inaccuracy.
For B 20 .
25 , our approach improves the robust accuracy by +4 .
8% and reduces the robust inaccuracy by − 0 .
CIFAR -
[Table]
[Figure]
R obu s t A cc u r ac y
R a cc r o b [ % ]
[Figure]
b e tt e r
7. U SING R OBUSTNESS TO A BSTAIN
Next, we evaluate using robustness as an abstain mechanism (Section 5) and how it beneﬁts from the training proposed in our work.
We compare the following abstain mechanisms:
Softmax Response (SR) (Geifman & El-Yaniv, 2017), which abstains if the maximum softmax output of the model f θ is below a threshold τ for some input x (cid:48) ∈ B pε ( x ) , that is:
[Equation]
Similar to the robustness indicator (Section 5), to guarantee robustness of S SR , we need to check the maximum softmax output of the model f θ on double the region B p 2 · ε ( x ) .
To evaluate robustness of
S SR , we use a modiﬁed version of APGD called APGDconf (Appendix A.4).
For each model considered in our work (e.g., Carmon et al.
, we evaluate its corresponding abstain selector:
• ( , ) CARMON SR , GOWAL SR ,
ZHANG SR , etc., all of which are ﬁne-tuned using TRADES .
Robustness Indicator (RI) (our work) , which abstains if the model F θ is non-robust:
[Equation]
Note that, unlike other selectors, our robustness indicator is by design robust against an adversary using the same threat model.
For each model, we compare two instantiations of this approach:
• ( , ) TRADES RI , ( , ) ERA RI (Equation 5).
Selection Network (SN) , which trains a separate neural network s θ : X → R and selects if:
[Equation]
When considering the robustness of the abstain model, the robustness of both the classiﬁer and the selection network have to be taken into account.
We compare against two instantiations of this approach, both trained using certiﬁed training:
• ( ) ACE - COLT SN (Balunovic & Vechev, 2019; Mueller et al., 2020), and • ( ) ACE - IBP SN (Gowal et al., 2018; Mueller et al., 2020).
Empirical Robustness In Figure 3, we show the comparison of different abstain approaches using two metrics – the robust selection ( R sel rob ), and the ratio of non-abstained samples that are robust and accurate ( R accrob ).
Ideally, we would like both of these to be as high as possible, but typically there is a tradeoff between the two.
This can clearly be seen in the results, where both our approach and softmax response can be used to obtain a Pareto front of optimal solutions.
Overall, the main results in Figure 3 show that as designed, our approach consistently improves robust accuracy.
For example, on CIFAR - 10 at ε ∞ = 1 / 255 and Carmon et al.
(2019) model,
we
[Table]
All models are certiﬁed via randomized
, using the hyperparameters listed in Table can successfully improve robust accuracy by +1.18% at the expense of -3.78% decrease in robust selection.
This is close to optimal since increasing robust accuracy is typically achieved by correctly selecting for which samples to abstain.
Interestingly, for some models and datasets, we can strictly improve over the baseline models by increasing both the robust accuracy and the non-abstained samples.
On CIFAR - 10 at ε ∞ = 1 / 255 and Gowal et al.
(2020) model, we increase the robust accuracy by +1.06% and robust selection by +1.61% (when training without data augmentations).
Compared to the other abstain methods, our approach, in general, improves both metrics while also providing much stronger guarantees.
Concretely, our approach guarantees that selected samples are robust in the considered threat model.
Softmax response only guarantees that all samples in the considered threat model have high conﬁdence and is thus vulnerable to high conﬁdence adversarial examples, and the selection network provides no guarantees with regards to the selector’s robustness.
Certiﬁed Robustness Applying our training for certiﬁed robustness L CRA with β = 1 . 0 consistently improves robust accuracy R accrob of robustness indicator abstain models.
In Table 3, we show our results on CIFAR - 10 , using (cid:96) 2 perturbations of radius 0 .
06 and 0 .
For instance, for the Cohen et al.
(2019) model trained at σ = 0 .
12 , we are able to improve the robust accuracy by +0 .
85% for ε 2 = 0 .
25 perturbations, at the expense of − 8 .
8% decrease in robust selection.
On the other hand, for the Sehwag et al.
(2021) model, our approach improves on both metrics.
For ε 2 = 0 .
25 perturbations, we increase robust accuracy by +0 .
82% and robust selection by +4 .
7. B OOSTING R OBUSTNESS WITHOUT A CCURACY L OSS
Finally, we present the results of combining the abstain models trained so far with state-of-the-art models trained to achieve high accuracy.
Note that, as discussed in Section 5, when evaluating adversarial robustness for B pε , we in fact need to consider B p 2 · ε robustness of the abstain model.
A summary of the results is shown in Figure 4.
Similar to the results shown so far, the 2-compositional architectures that use models trained by our method ( , ) improve over existing methods that optimize only for robust accuracy ( , ), as well as over models using softmax response ( , ) or selection network ( , ) to abstain.
For example, for CIFAR - 10 with ε ∞ = 1 / 255 and the Carmon et al.
(2019) model, we improve natural accuracy by +0.58% and +0.62%, while decreasing the robustness only by -2.75% and -2.82%, when training with and without data augmentations respectively.
More importantly, our approach signiﬁcantly improves robustness of highly accurate noncompositional models, with minimal loss of accuracy.
In fact, in half of the cases, the compositional architecture even slightly improves the overall accuracy, as summarized in Table 4.
We provide full results, including additional models and perturbation bounds in Appendix A.7, and an evaluation of the considered highly accurate non-compositional models in Appendix A.8.
CIFAR -
[Table]
[Figure]
Carmon et al.
[Figure]
WRN-
[Figure] [Figure]
R obu s t A cc u r ac y
R a cc r o b [ % ]
[Figure] [Figure]
et al.
[Figure] [Figure]
et al.
[Figure] [Figure]
ResNet
[Figure]
Zhang et al. (2019a)
[Figure] [Figure]
C ONCLUSION
In this work, we address the issue of robust inaccuracy of state-of-the-art robust models.
Ideally, models should be robust only if accurate, however, we show that existing robust models have nonnegligible amounts of robust but inaccurate samples.
We present a new training method that jointly minimizes robust inaccuracy and maximizes robust accuracy.
The key concept was extending an existing robust training loss with a term that minimizes robust inaccuracy, making our method widely applicable since it can be instantiated using various existing robust training methods.
We show the practical beneﬁts of our approach by both, using robustness as an abstain mechanism, and by leveraging compositional architectures to improve robustness without sacriﬁcing accuracy.
However, there are also limitations and interesting extensions to consider in future work.
First, while there are some cases where our training improves both robust accuracy and reduces robust inaccuracy (e.g., for certiﬁed robust Sehwag et al.
(2021) model on CIFAR - 10 or empirically robust Gowal et al
. model on CIFAR - 10 ), it does typically results in a trade-off between the two – reduced robust inaccuracy also leads to reduced robust accuracy.
To address this issue, in practice we compute a
Pareto front of optimal solutions, all of which can be used to instantiate the compositional model.
An interesting future work would be to explore this trade-off further and develop new techniques to mitigate it.
Second, given that we compute a Pareto front of the optimal solutions, a promising future work item is considering model cascades that consist of different models along this Pareto front, and progressively fall back to models with higher robust accuracy but also higher robust inaccuracy.
Third, we observed that as the robust inaccuracy approaches zero (i.e., the best case), the training becomes much harder.
This is both because these remaining robust inaccurate examples are the hardest to ﬁx, as well as because there are only very few of them.
In our work, we explored using data augmentation to address this issue, but more work is needed to make the training efﬁcient in such a low data regime.
A A PPENDIX A. D ATASETS
We ran our evaluations on four different datasets, namely on CIFAR - 10 and CIFAR - 100 (Krizhevsky et al., 2009), the Mapillary Trafﬁc Sign Dataset ( MTSD )
(Ertler et al., 2020), and a rail defect dataset provided by Swiss Federal Railways ( SBB ).
Additionally, we used a synthetic dataset consisting of two-dimensional data points.
In the following, we explain the necessary preprocessing steps to create the publicly available MTSD dataset.
Mapillary Trafﬁc Sign Dataset (MTSD)
The Mapillary trafﬁc sign dataset (Ertler et al., 2020) is a large-scale vision dataset that includes 52’000 fully annotated street-level images from all around the world.
The dataset covers 400 known and other unknown trafﬁc signs, resulting in over 255’000 trafﬁc signs in total.
Each street-level image is manually annotated and includes ground truth bounding boxes that locate each trafﬁc sign in the image, as shown in Figure 5a.
Further, each ground truth trafﬁc sign annotation includes additional attributes such as ambiguousness or occlusion.
Since the focus of this work is on classiﬁcation, we convert the base MTSD dataset to a classiﬁcation dataset (described below) by cropping to each ground truth bounding box.
We show samples from the resulting cropped MTSD dataset in Figure 5b.
[Figure] [Figure]
Sign Dataset ( MTSD ) samples.
The base dataset consists of street-level images that include annotated ground truth bounding boxes locating the trafﬁc signs (a).
We convert the dataset to a classiﬁcation task by cropping to the ground truth bounding boxes (b).
We convert the MTSD objection detection dataset into a classiﬁcation dataset as follows
Ignore all bounding boxes that are annotated as occluded (sign partly occluded), out-offrame (sign cut off by image border), exterior (sign includes other signs), ambiguous (sign is not classiﬁable at all), included (sign is part of another bigger sign), dummy (looks like a sign but is not) (Ertler et al., 2020).
Further, we ignore signs of class other-sign , since this is a general class that includes any trafﬁc sign with a label not within the MTSD taxonomy.
Crop to all remaining bounding boxes and produce a labeled image classiﬁcation dataset.
Cropping is done with slack,
i
e. we crop to a randomly upsized version of the original bounding box.
Given a bounding box BB =
([ x min , x max ] , [ y min , y max ]) , the corresponding upsized bounding box is given as
[Equation]
where α x , α y ∼ U [0 , 1] 3 and λ is the slack parameter, which we set to λ
= 1 . 0 .
Resize cropped trafﬁc signs to (64 , 64) .
is the uniform distribution over the interval
, b
Rail Defect Dataset (SBB)
The rail defect dataset ( SBB ) is a proprietary vision dataset collected and annotated by Swiss Federal Railways.
It includes images of rails, each of which is annotated with ground truth bounding boxes for various types of rail defects.
We note that all the models used in our work for this dataset are trained by the authors and not provided by SBB.
In fact, for our work, we even consider a different type of task – classiﬁcation instead of the original object detection.
As a consequence, the accuracy and robustness results presented in our work are by no means representative of the actual models used by SBB.
A. H YPERPARAMETERS
TRADES We use L TRADES (Zhang et al., 2019a) to both train models from scratch and ﬁne-tune existing models.
When training models from scratch, we train for 100 epochs using L TRADES , with an initial learning rate 1e-1, which we reduce to 1e-2 and 1e-3, once 75% and 90% of the total epochs are completed.
When ﬁne-tuning models, we train for 50 epochs using L TRADES , with an initial learning rate 1e-3, which we reduce to 1e-4 once 75% of the total epochs are completed.
We use batch size 200, use 10-step PGD (Madry et al., 2018) to generate adversarial examples during training, and set the β parameter in L TRADES to β TRADES = 6 . 0 .
Empirical Robustness Abstain Training We ﬁne-tune for 50 epochs using L ERA (Equation 5), with an initial learning rate 1e-3, which we reduce to 1e-4 once 75% of the total epochs are completed.
We use batch size 200, use 10-step PGD (Madry et al., 2018) to generate adversarial examples during training, and set β TRADES = 6 . 0 for the loss term L rob =
L TRADES .
Certiﬁed Robustness Abstain Training We ﬁne-tune for 50 epochs using L CRA (Equation 9), with an initial learning rate 1e-3, which we reduce to 1e-4 once 75% of the total epochs are completed.
We use batch size 50, k = 16 i.i.d.
samples from N (0 , σ 2 I ) , and set the inverse softmax temperature to Γ = 4 .
(cf.
Section 4.2).
Probabilistic Certiﬁcation via Randomized Smoothing We use the practical Monte Carlo algorithm by Cohen et al.
(2019) for randomized smoothing, using the same certiﬁcation hyperparameters as them.
We use N 0 = 100 Monte Carlo samples to identify the most probable class c A ,
N = 100 , 000 Monte Carlo samples to estimate a lower bound on the probability p A , and set the failure probability to α = 0 .
Synthetic Dataset
In Figure 1, we illustrate the effect of our training on a synthetic three-class dataset, where each class follows a Gaussian distribution.
We then use a simple four-layer neural network with 64 neurons per layer, and train it on N = 1000 synthetic samples, using L std , L TRADES (Zhang et al., 2019a), and L ERA (Equation 5).
For each loss variant, we train for 20 epochs, use a ﬁxed learning rate 1e-1, and batch size 10.
For L TRADES and L ERA , we use 10-step PGD (Madry et al., 2018) to generate adversarial examples during training, and set β TRADES = 6 . 0 .
A. L OSS F UNCTION A BLATION S TUDY
Additionally to the L ERA loss from Equation 5, we consider an alternative loss formulation for training an empirical robustness indicator abstain model.
The formulation is based on the Deep
Gamblers loss (Liu et al., 2019), which considers an abstain model ( F θ , S ) with an explicit abstain class a as a selection mechanism.
Since we consider robustness indicator selection, we replace the output probability of the abstain class f θ ( x ) a with the output probability of the most likely adversarial label.
This corresponds to the probability of a sample being non-robust and thus the probability of abstaining under a robustness indicator selector.
Similar to L ERA , we also add the TRADES loss (Zhang et al., 2019a) to optimize robust accuracy.
The resulting loss is then deﬁned as:
L DGA (
[Equation]
We conduct an ablation study over the two loss functions, L ERA and L DGA , for CIFAR - 10 and a ε ∞ = 8 / 255 TRADES (Zhang et al., 2019a) trained ResNet-50 model.
We ﬁne-tune the model for (cid:96) ∞ perturbations of radii 1 / 255 and 2 / 255 , using both L ERA and L DGA , training for 50 epochs each and setting the regularization parameter β = 1 . 0 .
For each loss variant, we train the base model once without data augmentations and once using the AutoAugment (AA) policy (Cubuk et al., 2018).
[Table]
We show the robust accuracy and the robust selection of the resulting robustness indicator abstain models in Table 5.
Observe that for all experiments, L ERA trained models achieve consistently higher robust accuracy and higher robust selection, compared to L DGA trained models.
For instance, when training for ε ∞ = 1 / 255 perturbations without data augmentations, L ERA achieves +1 .
71% higher robust accuracy and +1 .
33% higher robust selection, compared to L DGA .
Similarly, when training with AutoAugment, L ERA achieves +0 .
91% higher robust accuracy and +2 .
72% higher robust selection.
Similar results hold for ε ∞ = 2 / 255 perturbations.
A. C OMPARING A DVERSARIES FOR S OFTMAX R ESPONSE (SR)
Recall from Section 7.2 that we evaluated the robustness of softmax response (SR) abstain models using APGDconf , which is a modiﬁed version of APGD (Croce & Hein, 2020) using the alternative adversarial attack objective by Stutz et al.
This modiﬁed objective optimizes for an adversarial example x (cid:48) that maximizes the conﬁdence in any label c (cid:54)
=
F θ ( x ) , instead of minimizing the conﬁdence in the predicted label:
[Equation] [Equation]
The resulting adversarial attack ﬁnds high conﬁdence adversarial examples, and thus represents an effective attack against a softmax response selector S SR .
In the following, we conduct an ablation study over APGD and APGDconf by evaluating the robust selection R selrob and robust accuracy R accrob of an SR abstain model ( F θ , S SR ) using both APGD and
APGDconf .
We use the adversarially trained WideResNet-28-10 model by Carmon et al.
(2019) (taken from RobustBench (Croce et al., 2020)), trained on CIFAR - 10 for ε ∞ = 8 / 255 perturbations.
We then evaluate the classiﬁer as an SR abstain model ( F θ , S SR ) with varying threshold τ ∈ [0 , 1) , and report the robust selection and robust accuracy for varying (cid:96) ∞ perturbations in Figure 6.
Observe that for small perturbations such as ε ∞ = 1 / 255 , APGD and APGDconf are mostly equivalent concerning robust selection and robust accuracy.
However, for larger perturbations such as ε ∞ = 4 / 255 , the SR abstain model is signiﬁcantly less robust to APGDconf than to standard APGD , showing the importance of choosing a suitable adversarial attack.
High conﬁdence adversarial examples are generally more likely to be found for larger perturbations, thus an SR selector is signiﬁcantly less robust to APGDconf than to APGD for larger perturbations.
[Figure]
[Equation] [Equation]
[Equation]
[Figure]
[Figure] [Figure]
B ∞ 1 /
Zhang et al.
R obu s t A cc u r ac y
R a cc r o b [ % ]
[Figure]
[Figure]
∞ 1 /
Carmon et al.
[Figure] [Figure]
Gowal et al.
CIFAR -
[Figure]
[Figure]
Rebufﬁ et al.
[Figure]
Zhang et al.
[Figure] [Figure]
Zhang et al.
[Figure]
A. A DDITIONAL E XPERIMENTS ON R EDUCING R OBUST I NACCURACY
In this section, we present additional experiments on reducing robust inaccuracy for empirical robustness.
Similar to the results in Figure 2, we show the robust accuracy ( R accrob ) and robust inaccuracy ( R ¬ acc rob ) of different existing models ﬁne-tuned with ( ) and without ( ) data augmentations, in Figure 7.
At the same time, Figure 7 also shows the same models ﬁne-tuned with our proposed loss with ( ) and without ( ) data augmentations.
We again observe that our approach achieves consistently lower robust robust inaccuracy, compared to existing robust models.
For example, on CIFAR - 10 and for B ∞ 1 / 255 , the model from Carmon et al.
(2019) achieves 91 .
7% robust accuracy but also
8% robust inaccuracy.
Using our loss L ERA and varying the regularization term β , we can obtain a number of models that reduce robust inaccuracy to 0 .
14% while still achieving robust accuracy of 75 .
. A. A DDITIONAL E XPERIMENTS ON U SING R OBUSTNESS TO A BSTAIN
In this section [Figure] we present additional experiments on comparing different abstain approaches for empirical robustness.
We compare robustness indicator abstain models ( F, S RI ) using existing robust classiﬁers TRADES RI and classiﬁers ﬁne-tuned with our proposed loss ERA RI .
Further, we again consider softmax response and selection network abstain models, as described in Section 7.2.
Equivalent to Section 7.2, we use the robust selection ( R selrob )
[Figure]
and the ratio of non-abstained samples that are robust and accurate ( R accrob ) as our evaluation metrics
[Figure]
[Figure]
[Figure]
Zhang et al. (2019a)
[Figure]
ResNet
[Figure] [Figure]
R obu s t A cc u r ac y
R a cc r o b [ % ]
[Figure]
[Figure]
[Figure]
Carmon et al.
[Figure]
et al
[Figure]
[Figure] Zhang et al [Figure] [Figure]
[Figure]
b e tt e r
We show the comparison of the different abstain models in Figure 8.
Similar to the results in
Section 7.2, we again show that, as designed, our approach consistently improves robust accuracy.
For instance, consider the CIFAR - 10 Zhang et al.
(2019a) model at ε ∞ = 1 / 255 , trained without data augmentations ( ).
The ERA RI model with the highest robust selection R selrob improves robust accuracy by +2.39% at the expense of -3.44% decrease in robust selection.
This tradeoff is close to optimal since our approach increases robust accuracy by correctly abstaining from mispredicted samples, thus an increase in robust accuracy results in a corresponding decrease in robust selection.
Further, we again observe that by varying the regularization parameter β , we can obtain a Pareto front of optimal solutions.
Considering the CIFAR - 10 Zhang et al.
(2019a) model at ε ∞ = 1 / 255 , trained with data augmentations ( ), we can improve the robust accuracy up to 99.75%, an increase of +4.38% compared to the corresponding TRADES RI model ( ).
However, this comes at the expense of a disproportionally large decrease of -42.27% lower robust selection.
We observe similar results for other models, datasets, and perturbations regions, shown in Figure 8.
Further, we again note that our approach mostly improves both robust selection and robust accuracy when compared to softmax response and selection network abstain models.
A. A DDITIONAL E XPERIMENTS ON B OOSTING R OBUSTNESS WITHOUT A CCURACY L OSS
In this section, we present additional results on combining abstain models with state-of-the-art models trained to achieve high natural accuracy.
Equivalent to Section 7.3, we put the abstain models trained so far in 2-composition (Section 6) with the standard trained core models discussed in Appendix A.8.
We show the natural ( R nat ) and adversarial accuracy ( R accrob ) of the resulting 2-compositional architectures in Figure 9.
We again observe that 2-compositional architectures using models trained by our method ( , ) improve over existing methods that solely optimize for robust accuracy ( , ).
Further, our method mostly improves both the natural and robust accuracy, compared to 2-compositional architectures using softmax response ( , ) or selection network ( , ) to abstain.
For example, on SBB and the
Zhang et al.
(2019a) model at ε ∞ = 1 / 255 , our approach ( ) improves natural accuracy by +0.68%, while decreasing the robust accuracy by only -1.54%.
Further, we show that 2-compositional architectures using models trained by our method achieve signiﬁcantly higher robustness and mostly equivalent overall accuracy, compared to state-of-theart non-compositional models trained for high natural accuracy.
In Table 6, we show the natural ( R nat ) and adversarial accuracy ( R accrob ) of our 2-compositional models and illustrate the accuracy improvement over the standard trained models discussed in Appendix A.8.
For instance, consider
CIFAR - 10 at ε ∞ = 2 / 255 and the 2-compositional architecture using the Gowal et al.
(2020) model as robust model F robust .
Our model improves the robust accuracy by +75.3% and the natural accuracy by +0.1%, compared to the standard trained model by Zhao et al.
Similar results hold for other models, datasets, and perturbation regions.
[Table]
A. C ORE M ODELS
Recall from Section 6 that an abstain model ( F, S ) can be enhanced by a core model F core , which makes a prediction on all abstained samples, resulting in 2-compositional architectures.
In Section 7.3, we presented an evaluation of 2-compositional architectures, where we used state-of-the-art standard trained models as core models.
In Table 7, we show the natural and adversarial accuracy of core models used in Section 7.3, for varying (cid:96) ∞ perturbation regions, where we use 40-step APGD (Croce & Hein, 2020) to evaluate robustness.
[Table]
A. R OBUSTNESS /A CCURACY D ATASET S PLITS
Consider a robustness indicator abstain model ( F θ , S RI ) and a labeled dataset D = { ( x i , y i )
Ni =1 } on which we evaluate the classiﬁer F θ : X → Y .
Based on the robustness and accuracy of the classiﬁer
F θ , we can partition D into four disjoint subsets D
= { D r ∧ a F θ , D
¬ r ∧ a F θ , D r ∧¬ a F θ , D ¬ r ∧¬ a F θ } , where:
[Equation]
F θ ( x ) ∧ F θ ( x ) (cid:54) = y } D [Equation]
θ =
{ ( x , y ) ∈ D : ∃ x (cid:48)
∈ B pε ( x ) .
F θ ( x (cid:48) )
(cid:54) =
F θ ( x ) ∧ F θ ( x ) = y }
D ¬ r ∧¬ a F θ =
[Equation]
x
[Equation]
D
[Equation]
x
[Equation]
pε
[Equation]
x [Equation] F θ [Equation]
x
[Equation]
x
[Equation] (cid:54) [Equation] We illustrate this dataset partitioning on the CIFAR - 10 (Krizhevsky et al., 2009) dataset.
We consider a TRADES (Zhang et al., 2019b) trained ResNet-50 and the WideResNet-28-10 models by Carmon et al.
; Gowal et al.
(2020) (taken from Robustbench (Croce et al., 2020)), where each model is adversarially pretrained for ε ∞ = 8 / 255 and then ﬁne-tuned via TRADES to the respective (cid:96) ∞ threat model illustrated Table 8.
Further, we also consider a standard trained ResNet-50.
We then evaluate the robustness and accuracy of each model using 40-step APGD (Croce & Hein, 2020).
Considering
Table 8, note that standard adversarial training methods do not necessarily eliminate the occurrence of robust inaccurate samples (
x , y ) ∈
D r ∧¬ a F θ , and that the robust inaccuracy generally increases for smaller perturbation regions.
Further, we note that while standard trained models have low robust inaccuracy, they also have low overall robustness, resulting in low overall robust accuracy.
[Table] Further, we also illustrate the robustness-accuracy dataset partitioning on CIFAR - 100 (Krizhevsky et al., 2009).
We consider a standard trained WideResNet-28-10 and the adversarially trained
WideResNet-28-10 by Rebufﬁ et al.
Again, the model by Rebufﬁ et al.
(2021) was pretrained for ε ∞ = 8 / 255 perturbations and then TRADES ﬁne-tuned for the respective threat model indicated in
Table 9.
We again evaluate the robustness-accuracy dataset partitioning for varying (cid:96) ∞ perturbations using 40-step APGD (Croce & Hein, 2020), and list the exact size of each data split in Table 9.
Notably, we observe that on the model by Rebufﬁ et al.
24% of all test samples are robust but inaccurate for ε ∞ = 1 / 255 perturbations, which is a signiﬁcantly larger fraction compared to similar models on CIFAR - 10 .
[Table]