R EDUCING E XCESSIVE M ARGIN TO A CHIEVE A B ETTER A CCURACY VS . R OBUSTNESS T RADE - OFF
A BSTRACT
While adversarial training has become the de facto approach for training robust classiﬁers, it leads to a drop in accuracy.
This has led to prior works postulating that accuracy is inherently at odds with robustness.
Yet, the phenomenon remains inexplicable.
In this paper, we closely examine the changes induced in the decision boundary of a deep network during adversarial training.
We ﬁnd that adversarial training leads to unwarranted increase in the margin along certain adversarial directions, thereby hurting accuracy.
Motivated by this observation, we present a novel algorithm, called Helper-based Adversarial Training (HAT) , to reduce this effect by incorporating additional wrongly labelled examples during training.
Our proposed method provides a notable improvement in accuracy without compromising robustness.
It achieves a better trade-off between accuracy and robustness in comparison to existing defenses.
Code is available at https://github.com/imrahulr/hat .
I NTRODUCTION
[Figure]
PreAct
[Figure]
Clean
[Figure]
92 91              (2021) (2018) +HAT +HAT +HAT by prominent defenses on CIFAR-10 against perturbations.
We compile top models which use additional data and are available in Croce et al.
From 2018 to 2021, the accuracy for robust WideResNet-28/34-10 models has only increased by around 2 .
While our proposed
HAT, combined with recent approaches, advances clean accuracy further by about 2 - 4% .
Despite the remarkable success of modern deep networks on computer vision tasks, they are notoriously fragile to small, imperceptible changes in the input (Szegedy et al., 2014).
Ever since the introduction of (cid:96) p -norm based adversaries (Goodfellow et al., 2015; MoosaviDezfooli et al., 2016), a growing body of researchers has focused on developing solutions for improving the robustness of DNNs.
At present, adversarial training (Madry et al., 2018) remains the most effective approach for robustiﬁcation.
Nevertheless, it is known to cause an undesirable reduction in accuracy, thus leading to the much-debated trade-off between accuracy and robustness (Tsipras et al., 2019;
Yang et al., 2020).
Subsequently, in an endeavour to alleviate the accuracy-robustness tradeoff, several reﬁnements have been proposed to the vanilla adversarial training, e.g., Zhang et al.
; Wang et al.
Rice et al.
; Wu et al.
; Zhang et al.
However, as shown in Fig.
1, the progress has been rather incremental over the last few years.
Notwithstanding these recent advances, closing the large gap between accuracy and robustness still remains an open challenge and every modest improvement in this regard remains signiﬁcant.
In this paper, we take a step towards demystifying and improving the aforementioned trade-off.
To this end, we take a closer look at the effect of training with adversarial examples on the geometry of decision boundaries learnt by deep networks.
Speciﬁcally, relying on the perspective proposed by Ortiz-Jimenez et al.
(2020) which relates the directional margin with the discriminative features used by a network, we uncover an unwanted consequence of adversarial training.
That is, we identify that adversarial training leads to a superﬂuous increase in the margin along the adversarial directions of the input space computed for a regularly trained network.
We refer to these directions as initial
[Figure]
adversarial directions 1 .
In other words, the network becomes excessively robust (compared to the attack radius ε ) along the directions which are otherwise pivotal for regular networks, and hence the accuracy drops.
Further, we establish an empirical connection between the margin along initial adversarial directions and the classiﬁcation performance of the network.
In particular, a smaller margin along these directions corresponds to a higher accuracy.
Thus, while the excessive margin is not essential for robustness, it severely derails the performance on clean samples.
This naturally raises the following question:
Can we foresee an improvement in accuracy by reducing the margin along initial adversarial directions while preserving the same level of robustness?
To address this question, we propose a heuristic adversarial training scheme to reduce the directional margin and thus, achieve a better accuracy without sacriﬁcing robustness.
We call this algorithm, Helper-based Adversarial Training (HAT) , the name derived from the fact that we incorporate additional training examples comprising of overly perturbed adversarial images (possibly) wrongly labelled using a standard trained network to help impede excessive directional robustness and hence, improve clean accuracy (see
Fig. 2).
We also provide an extensive analysis of HAT and compare it with state-of-the-art AT methods.
The main contributions of this paper include
: • We demonstrate that adversarial training leads to a large margin along initial adversarial directions.
In addition, we identify a direct connection between the excessive margin and the reduction in clean accuracy caused by adversarial training.
• We propose Helper-based Adversarial Training (HAT) to explicitly reduce the excessive margin by augmenting the training data with properly-labelled helper examples.
• Experimentally, we show that HAT consistently improves accuracy without sacriﬁcing robustness.
Notably, in combination with recent improvements from Rebufﬁ et al. (2021), HAT sets a new state-of-the-art and improves clean accuracy by 3 .
33% on CIFAR-10 and 4 .
63% on CIFAR-100 using PreAct ResNet-18 against (cid:96) ∞ perturbations of size 8 / 255 .
1. R ELATED WORK
Improving adversarial robustness.
To mitigate the adversarial susceptibility of DNNs, a myriad of defense methodologies (Papernot et al., 2016; Tram`er et al., 2018; Kannan et al., 2018; Wang et al., 2019; Xiao et al., 2020) have been proposed.
Yet, many of them have been shown to provide superﬁcial robustness by introducing gradient obfuscation (Athalye et al., 2018), hence calling for a cautious evaluation of robustness (Tramer et al., 2020; Croce et al., 2020).
Nonetheless, adversarial training (AT) (Madry et al., 2018) or training with worst-case inputs has been the most popular approach for improving the robustness of DNNs albeit at the cost of a sizeable drop in accuracy.
Notably, AT has been able to withstand newer sophisticated adversaries (Gowal et al., 2019; Croce & Hein, 2020).
Another prominent defense is TRADES (Zhang et al., 2019), which provides a systematic way to control the trade-off between accuracy and robustness by solving:
[Equation]
where CE is the cross-entropy loss, KL is the Kullback-Leibler divergence and β controls the trade-off between accuracy and robustness.
More recent attempts to improve the accuracy vs. robustness trade-off have predominantly dealt with reﬁning adversarial training by (i) modifying the objective function (Wang et al., 2020;
Zhang et al., 2020; 2021), (ii) early-stopping (Rice et al., 2020), (iii) heuristics and hyperparameter selection (Pang et al., 2021; Gowal et al., 2021), (iv) extra data (Hendrycks et al., 2019;
Alayrac et al., 2019; Carmon et al., 2019; Rebufﬁ et al., 2021; Sehwag et al., 2021), and (v) weight initialization for training with large perturbations (Shaeiri et al., 2020).
Despite these advancements, the problem is far from being solved with a huge gap between accuracy and robustness on practical image recognition benchmarks.
We revisit this issue in our paper and build upon TRADES to further ameliorate the trade-off.
Understanding adversarial vulnerability.
A concurrent stream of prior works has provided plausible reasons for the existence of adversarial examples.
Adversarial vulnerability of DNNs has been accredited to the existence of highly generalizable, but non-robust features in the data which are exploited by DNNs to achieve state-of-the-art performance (Jetley et al., 2018; Ilyas et al., 2019).
Besides, Ortiz-Jimenez et al.
(2020) devised a novel experimental framework which relates the distance of data points to the decision boundary of a DNN with the discriminative features used by the
DNN in order to understand the mechanism behind AT.
P RELIMINARIES
Consider the input space X ⊆ R d .
Let f θ : X → R C represent a deep neural network classiﬁer parameterized by θ , where C is the number of output classes.
Let F θ ( x ) =
arg max k f θ ( x ) k be the class label predicted by f θ for any x ∈ X , where f θ ( x ) k is the k th component of f θ ( x ) .
Margin.
Given a classiﬁer F θ , input x and an unit vector ˆ r ∈ S d − 1 , we deﬁne margin µ at x along the direction ˆ r as:
[Equation]
Additionally, note that we refer to a deep network trained only on clean samples as standard network and a network trained via adversarial training as robust network .
Besides, we reuse the deﬁnitions of clean (natural) and robust (adversarial) accuracy as stated by Zhang et al.
Initial adversarial directions.
Given a standard network f θ , input dataset { ( x i , y i ) } ni =1 , we deﬁne the set of initial adversarial directions as R init =
{ r i / || r
i || 2 } ni =1 where r i is obtained by solving:
[Equation]
where, (cid:96) ( · , · ) is an arbitrary loss function e.g. cross-entropy (CE).
This optimization is usually solved via a multi-step procedure called projected gradient descent (PGD) (Madry et al., 2018).
A DVERSARIAL TRAINING INTRODUCES EXCESSIVE INVARIANCE
We begin our analysis by examining the effect of adversarial training (AT) on the decision boundary of DNNs.
Via novel analysis on a synthetic dataset and CIFAR-10 (Krizhevsky, 2009), we show that
AT triggers a superﬂuous increase in the margin along the initial adversarial directions as compared to the nominal increase required to attain robustness.
In addition, we provide empirical evidence which signiﬁes a connection between the increase in margin and reduction in clean accuracy.
Toy problem.
First, we study a toy setting to shed some light on the phenomenon of excessive directional margin caused by AT.
We construct a 3-d binary classiﬁcation dataset drawn from two distributions which live on two noisy concentric circles of different radii in the x 1 - x 2 plane and being linearly separable along the third dimension x 3 .
In particular, x 1 = ρ i cos ( z ) + (cid:15) 1 , x 2 =
ρ i sin ( z ) + (cid:15) 2 and x 3 ∼ U
( α i , β i ) where z ∼ U (0 , 2 π ) and (cid:15) 1 , (cid:15) 2 ∼ N (0 , σ 2 ) where i = 1 ,for class 1 and 2 respectively.
We train a single hidden-layer MLP via both standard training and adversarial training.
Fig.
3 visualizes the decision regions with both the training procedures.
It is
[Figure]
evident that the network primarily uses x 3 to achieve zero classiﬁcation error when trained using standard training, but the resulting model performs poorly in terms of robustness.
In contrast, when we use AT, the learned decision boundary is completely different from that in the standard case.
Here, the network becomes reasonably invariant along x 3 (Fig. 3b), thus causing the directional margin along x 3 to tend to ∞ .
This enables the network to attain robustness at the cost of a small increase in classiﬁcation error.
Evidence on CIFAR-10.
Next, we illustrate that a similar phenomenon occurs in the case of stateof-the-art deep networks trained on CIFAR-10.
To this end, we measure directional margin (as deﬁned by Eq. 2) to the classiﬁcation boundary which allows us to partly capture the geometry of decision boundary.
We restrict ourselves to the following setting.
We take a ResNet-
(He et al., 2016) trained until convergence on CIFAR-10 (achieving 94 .
6% accuracy and 0% robustness to (cid:96) ∞ -
PGD perturbations on the test set) and then ﬁne-tune it using AT with adversarial examples.
This framework allows us to study the evolution of the decision boundary caused by AT in comparison to that learnt by a standard network.
We use (cid:96) ∞ -PGD with norm ε = 8 / 255 for training.
The network attains 83 .
3% accuracy and 51 .
6% robustness on the test set after adversarial ﬁne-tuning.
Please refer to App.
B.1 for more details of the experimental setup.
Meanwhile, during adversarial ﬁne-tuning, we track the margins along the adversarial directions found by PGD to shed some light on the learning dynamics.
Suppose R k
= { r ki / ||
r
ki || 2 } , where r ki denotes adversarial perturbation found by PGD at k th epoch of adversarial ﬁne-tuning for the input sample x i .
Thus, R init =
R 0 represents the set of initial adversarial directions.
We hypothesize that during adversarial training, the network becomes excessively robust to these initial adversarial directions R 0 while slightly shifting the decision boundaries along other adversarial directions
R k ≥ 1 .
Fig.
4 illustrates the margins along R 0 , R 1 and R 5 before and after adversarial ﬁne-tuning respectively.
The dashed red line indicates the value of ε used for training, i.e., any allowable pertur- bation r has || r || ∞ ≤ 8 / 255 or equivalently
||
r || 2 ≤ 1 . 74 .
Intuitively, one might expect adversarial ﬁne-tuning to cause small shifts in the decision boundary so that the margin becomes greater than 1.74, and attain robustness.
However, this is not the case in practice.
Intriguingly, the classiﬁer instead resorts to becoming largely insensitive along R 0 for some data points as evident in Fig.while undergoing small shifts along other directions R k ≥ 1 .
We also observe a decrease of 11 .
3% in clean accuracy after ﬁne-tuning.
[Table]
Connection between margin along R init and clean accuracy.
We now provide a two-fold argument which justiﬁes the following hypothesis: ”The drastic rise in the margin along R init is directly correlated to the observed reduction in accuracy”.
In fact, a larger margin contributes to a larger drop in accuracy.
(i)
Firstly, we complement our hypothesis with the following observation by Ortiz-Jimenez et al.
The directions of input space with small margins and in turn, the initial adversarial directions in the case of standard network, are associated with discriminative features learnt by the network.
We believe that these directions are crucial for the performance of the network.
Thus, a drastic directional margin along these directions contributes to the drop in classiﬁcation accuracy.
(ii) We train a robust network on CIFAR10 using TRADES with different values for the trade-off parameter β .
(Please refer to Sec.
5.1 for the exact experimental setup.)
As β increases, we observe an increase in the margin along R init (computed on a subset of 1024 examples from the CIFAR-10 test set) and a corresponding reduction in clean accuracy (see Table 1).
This further corroborates our hypothesis.
H ELPER - BASED ADVERSARIAL TRAINING
As demonstrated in Sec. 3, AT triggers an unwarranted increase in the margin along initial adversarial directions, thus hindering the network from using highly discriminative features in those directions.
Note that a large margin is not a sufﬁcient condition for (cid:96) p -based ε -robustness.
A trivial example of this is a constant classiﬁer which has an inﬁnite margin, but has both poor robustness and poor accuracy.
Instead, the excessive rise in margin translates to large changes in the geometry of decision boundary compared to a standard network and causes a corresponding drop in clean accuracy.
We here base our algorithm on the intuition that slightly pushing the decision boundary away from data should sufﬁce for attaining robustness as opposed to the large shifts observed.
Our results in Sec.
5 suggest that HAT, by reducing margin, can indeed lead to improved performance.
Now, we introduce our proposed algorithm, helper-based adversarial training (HAT) , to reduce the excessive directional margin observed in practice.
We aim to preserve certain geometric properties of a standard trained network viz.
the predictive power along adversarial directions while learning a robust model.
To this end, we add additional training examples, called helper examples, which are generated on-the-ﬂy during the adversarial training procedure.
In particular, a helper example is constructed by extrapolating the adversarial perturbation found during training and is (possibly wrongly) labelled by a standard network (see Fig. 2).
Formally, we deﬁne a helper example as:
Helper examples.
Given an input sample ( x i , y i ) , a standard network f θ std , a robust network iterate f θ k rob at k th training iteration and adversarial example x (cid:48)
i computed by the adversary ϕ for f θ k rob , the corresponding helper example is given by ( ˜ x i , ˜ y i )
where
[Equation]
The motivation behind this deﬁnition is illustrated in Fig.
2 where it is evident that we can stimulate a slight push to the decision boundary along any adversarial direction by making the network predict the correct label
y i at adversarial example x (cid:48)
i and have it predict ˜
y
i (often ˜ y
i (cid:54) =
y i ) at helper example ˜ x
i to preserve the discriminative characteristics as modelled by a standard network.
Deﬁning the helper examples as x +2 r
i is a heuristic choice that provides a good compromise between the helper example being sufﬁciently dissimilar (from x ) for the model to assign a different label (see Fig.for a visual example), and not too dissimilar to affect the performance on other clean samples.
In
Input
Training dataset D
= { x i y i ) }
ni = repeat (cid:46) do 6: x i j ← x i j + 0 .
001 · N
I ) (cid:46) example 7:
K do 8: x (cid:48)
i j ← (cid:81) B ( x ij ,ε ) ( x (cid:48) i j + α · sign ( ∇ x (cid:48) ij KL ( f θ HAT ( x i j ) , f θ HAT ( x (cid:48) i j )))) 9: end for 10:
Compute helper example: ˜ x i j ← x i j + 2 ( x (cid:48)
i j − x i j ) 11:
Set helper label: ˜ y
i j ← arg max k f θ std ( x (cid:48) i j ) k 12: for 13: θ HAT ← θ HAT − η m · (cid:80) m j =1 ∇ θ HAT (cid:16) CE (cid:0)
y i , f θ HAT ( x i j ) (cid:1) + β · KL ( f θ HAT ( x i j ) , f θ HAT ( x (cid:48) i j ))
+ γ · CE (˜ y
i
j , f θ HAT (˜ x
[Equation]
[Table]
contrast to AT, this allows preventing the undesirable excessive rise in the margin to some extent, thus making it possible to achieve signiﬁcantly better performance on clean samples.
Put differently,
HAT can also be framed as performing geometric self-distillation (Hinton et al., 2015) to mimic certain geometric properties of a standard trained network.
HAT algorithm.
We choose to instantiate HAT by extending TRADES which alike TRADES allows us to balance the accuracy-robustness trade-off.
Algorithm 1 summarizes the pseudo-code for HAT.
The training objective for HAT comprises of three terms: standard loss, robust loss, and an additional helper loss.
Thus, in comparison to TRADES, we have an additional parameter γ which controls the weight of helper loss and thus, the extent of resistance to excessive directional margin.
Finally, note that our extension can be easily plugged into other recent techniques e.g. Wu et al.
; Gowal et al.
; Rebufﬁ et al.
In fact, in our experiments, we indeed combine HAT with the approach in Rebufﬁ et al.
(2021) to advance the prior art by a signiﬁcant margin (Sec. 5.3).
E XPERIMENTS
This section constitutes an extensive evaluation of HAT.
Initially, to test the universality of our approach, we study the performance of HAT with ResNets on different datasets and attack conﬁgurations.
Next, we leverage extra data and wider networks to obtain state-of-the-art performance on conventional robustness benchmarks.
Towards the end, we conduct experiments to analyze HAT.
5. P ERFORMANCE EVALUATION WITH R ES N ETS
We here empirically validate the performance of HAT.
We report results using ResNet-18 (He et al., 2016) on three datasets: CIFAR-10, CIFAR-100 (Krizhevsky, 2009) and SVHN (Netzer et al., 2011).
We compare HAT with three prominent adversarial defenses: (i) AT (Madry et al., 2018), (ii)
TRADES (Zhang et al., 2019) and (iii) MART (Wang et al., 2020).
Training setup.
We borrow the set of training hyperparameters from DAWNBench (Coleman et al., 2017).
Precisely, we use SGD optimizer with Nesterov momentum (Nesterov, 1983); cyclic learning rates (Smith & Topin, 2018) with cosine annealing and a maximum learning rate of 0 .
21 for CIFAR10, CIFAR-100, and 0 .
05 for SVHN.
We train each model for 50 epochs on CIFAR-10 and CIFAR100 whereas we apply 15 epochs on SVHN.
For (cid:96) ∞ training, we use PGD attack with maximum perturbation ε = 8 / 255 and run the attack for K = 10 iterations for all datasets.
The PGD step size is set to α = ε/ 4 = 2 / 255 for CIFAR-10, CIFAR-100; α = 1 / 255 for SVHN.
For HAT, we ﬁx γ to
5 and use β = 2 . 5 for CIFAR-10 and SVHN; β = 3 . 5 for CIFAR-100.
Whereas the regularization parameter β for TRADES is set to 5 . 0 for CIFAR-10, SVHN and 6 . 0 for CIFAR-100.
For MART, we choose β = 5 . 0 .
More details can be found in App.
C.1.
Evaluating robustness.
During training, we perform early stopping (Rice et al., 2020) and select the model that has the highest robustness against PGD ( K = 20 ) for further evaluation.
Further, to evaluate the robust accuracy of our models, we use AutoAttack (Croce & Hein, 2020) which comprises an ensemble of four diverse attacks (including a black-box attack).
AutoAttack has been consistently shown to provide a reliable estimation of robustness.
Nevertheless, we also conduct additional evaluations and checks with HAT to eliminate the possibility of gradient obfuscation in
App. C.4.
[Table]
Results.
Table 2 reports the performance of HAT and other prominent defenses in the literature.
It is evident that HAT can notably increase the clean accuracy of the models with little to no degradation of robustness.
In other words, HAT consistently shrinks the gap between accuracy and robustness by around 2% compared to prior adversarial training schemes.
For example, in the case of CIFAR-10,
HAT boosts clean accuracy by 2 . 17% whilst achieving similar robustness as TRADES.
In addition, we ﬁnd that the 2 .
17% gain in clean accuracy on CIFAR-10 equates to a 2 .
08% jump on common corruptions (Hendrycks & Dietterich, 2019)
(see App. C.3) which is highly desirable given the practical relevance of common corruptions.
Finally, we note that our models do not show any signs of gradient obfuscation as evident from the analysis in App.
C.4.
5. O THER THREAT MODELS AND LARGER DATASETS
To verify the generality of our results, we now experiment with different attack conﬁgurations and larger benchmarks.
[Table]
Threat models.
We investigate two different threat conﬁgurations on CIFAR-10: (i) (cid:96) ∞ with norm 12 / 255 and (ii) (cid:96) 2 with norm 128 / 255 and follow the same setup as
Sec. 5.1.
Here, we compare HAT only with
TRADES since TRADES achieves superior results than other techniques on CIFAR10.
As shown in Table 3, HAT surpasses
TRADES by a large margin.
Speciﬁcally, with (cid:96) ∞ perturbations of size 12 / 255 , HAT clearly surpasses TRADES, effecting 5 .
95% improvement in clean accuracy while simultaneously bettering robustness by 0 .
Moreover,
HAT achieves a higher clean accuracy ( ↑ 1 . 46% ) against (cid:96) 2 perturbations as well.
However, the improvement here is rather limited
owing to the fact that when training with smaller ε ’s, the phenomenon of excessive directional margin might not be very severe.
Larger datasets.
The efﬁcacy of HAT indeed holds for large-scale datasets such as TinyImageNet200 and ImageNet-100 (Deng et al., 2009), where AT is known to cause a huge degradation in accuracy.
We observe that HAT improves the accuracy of AT from 47 .
76% to 52 .
60% and robustness from 17 .
92% to 18 .
14% on TinyImageNet-200 against (cid:96) ∞ distortions of size 8 / 255 .
With
[Table]
ImageNet-100 against (cid:96) ∞ perturbations of size 4 / 255 , HAT outperforms TRADES by 4 .
20% on clean accuracy and 1 .
92% on robustness; surpasses AT by 1 .
36% on clean accuracy and 0 .
98% on robustness.
More details on the experiment can be found in App.
C.3.
5. E FFECT OF ADDITIONAL DATA AND WIDER NETWORKS
It has been observed that robust generalization beneﬁts remarkably from the use of extra training data (Schmidt et al., 2018) and increased model capacity (Madry et al., 2018).
We investigate the impact of these two factors on HAT training.
Speciﬁcally, we build upon the recent approaches from Gowal et al.
(2021) and Rebufﬁ et al.
(2021) which provide a carefully designed experimental suite composed of model weight averaging, SiLU activation function, larger models and additional training data to considerably progress the state-of-the-art performance on multiple robustness benchmarks.
While Gowal et al.
(2021) uses an additional subset of 500k natural images extracted from
Million Tiny Images (80M TI) dataset (Torralba et al., 2008), Rebufﬁ et al.
(2021) leverages 1M synthetic images generated by DDPM
(Ho et al., 2020).
Following the exact experimental setup from these papers 2 , we train several models on CIFAR-10 and CIFAR-100 using three different model architectures namely, PreAct ResNet-18 (PRN-18) (He et al., 2016), WideResNet-28-10 (WRN28-10) and WideResNet-34-10 (WRN-34-10) (Zagoruyko & Komodakis, 2016).
However, since the additional data from Gowal et al.
(2021) is not publicly available 3 , we resort to using a similar dataset provided by Carmon et al.
Note that we always pick β = 3 . 5 and γ = 0 .
5 for HAT.
In Table 4, we benchmark the performance of HAT along with that of state-of-the-art approaches.
Using PRN-18 and synthetic DDPM-generated images, HAT improves clean and robust accuracy by (i) 3 .
33% and 0 .
43% respectively on CIFAR-10 and (ii) 4 .
63% and 0 .
38% respectively on CIFAR100 against (cid:96) ∞ perturbations with norm 8 / 255 .
Notably, HAT sets a new state-of-the-art on CIFAR10 obtaining 89 .
02% clean accuracy and 57 .
67% robust accuracy.
More importantly, these sizeable improvements with PRN-18 suggest that smaller networks have not yet exhausted their limits and we must invent techniques which better exploit the capacity of these networks.
Moreover, with
WRN-28-10, HAT also surpasses the state-of-the-art accuracy in the respective categories by 2 - 3% .
5. A NALYSIS
Accuracy vs. robustness trade-off.
We investigate the sensitivity of HAT to its hyperparameters namely, the weight of robust loss β and the weight of helper loss γ .
Fig.
5 includes a plot which
82.5 83.0 83.5 84.0 84.5 85.0 85.5 86.0 86.5 Clean Acc.
R o b u s t
A cc .
AT
TRADES (.)
HAT (·, =0.25)
[Figure]
P G D 40 H
A T
P G
[Figure]
40 T R A D E S
Algorithm
MedianMargin
AT 9.
TRADES 9.
HAT 9.
demonstrates the performance of HAT at different values of β ∈ [1 . 5 , 4 . 0] and γ ﬁxed to 0 .
We also show the curve obtained by varying the regularization parameter β of TRADES in Fig.
We do not include MART here since it achieves a relatively poor trade-off.
We can clearly see that for HAT, β controls the trade-off between accuracy and robustness where increasing β improves robustness.
Next, we study the impact of γ .
First, as expected, HAT with γ = 0 performs identically to TRADES.
Second, γ ∈ { 0 . 25 , 0 . 5 } achieves the best trade-off and within this range, the choice of γ has negligible inﬂuence on the resulting trade-off exhibited by HAT.
Third, increasing γ beyond 0 .
75 causes the robust accuracy to deteriorate due to the dominance of helper loss over robust loss.
The hyperparameter sweeps for γ are presented in App.
C.5.
How does HAT improve clean accuracy.
The improvement in the performance on clean samples provided by HAT can be accredited to the following two observations on CIFAR-10:
(i)
In contrast to TRADES, HAT marginally compromises robustness to (cid:96) ∞ perturbations with a larger norm.
We take a robust network and evaluate it using PGD attack ( K = 40 ) for different values of (cid:96) ∞ -norm ε ∈ [0 , 20 / 255] .
Fig.
6 plots the difference between robustness of HAT and TRADES vs. ε .
We observe that while HAT outperforms TRADES at smaller ε ’s, it performs slightly worse after the ε exceeds the value used during training, i.e., ε > 8 / 255 .
This implies that we have traded robustness to high ε ’s for an improvement in accuracy.
(ii) HAT exhibits a slightly lower directional margin along initial adversarial directions R init in comparison to AT and TRADES.
The margin along R init for different algorithms (as evaluated on a random subset of 1024 samples from the CIFAR-10 test set) is illustrated in Table 5.
Note that in Table 5, we list the models that achieve the same robust accuracy ( ∼ 47 .
9 % to AutoAttack).
Yet, the median margin for HAT is slightly lower than that for
AT and TRADES.
Additionally, in comparison to TRADES, HAT reduces the number of samples with margin greater than 10 ε from 40% to 27% .
C ONCLUSION
We presented experimental evidence to highlight that state-of-the-art adversarial defenses foster a superﬂuous increase in the margin along certain adversarial directions of the input space.
This largely destroys the discriminative characteristics along these directions and partly contributes to the much-debated accuracy vs. robustness trade-off.
Further, inspired by our analysis, we introduced a novel algorithm, Helper-based Adversarial Training (HAT) , to alleviate the problem of excessive directional margin.
HAT attempts to mimic the discriminative features learnt by standard trained networks to improve the accuracy on clean samples, hence achieving a superior accuracy vs. robustness trade-off compared to existing defenses.
Moreover, HAT surpasses the known state-of-the-art clean accuracy by 3 - 5% without sacriﬁcing robustness.
Finally, we verify that HAT slightly reduces the directional margin, thus directly beneﬁting the accuracy.
We believe that our experiments and the proposed HAT algorithm can open the door for further research on relieving the accuracy vs. robustness trade-off under limited model capacity.
While HAT is only one possible candidate, future works can come up with more effective techniques to alleviate the problem of excessive margin. A CKNOWLEDGMENTS
The authors would like to thank Alhussein Fawzi and Guillermo Ortiz-Jimenez for their feedback on an earlier version of this paper
T
[Figure]
Fig.
7 illustrates the toy dataset used for our experiment from two different viewpoints.
To be precise, we draw the 3 features x 1 , x 2 and x 3 as follows.
x 1 =
ρ
i cos ( z ) + (cid:15) 1 , x
2 =
ρ i sin ( z ) + (cid:15)and x 3 ∼ U
( α i , β i ) where z ∼ U (0 , 2 π ) and (cid:15) 1 , (cid:15) 2 ∼ N (0 , σ 2 ) where i = 1 , 2 for class 1 andrespectively.
We set σ = 0 .
2 , ρ 1 = 0 .
35 , ρ 2 = 1 , α 1 = 0 .
65 , β 1 = 0 .
70 , α 2 = 0 .
80 and β 2 = 0 .
We use a single hidden layer MLP with 25 hidden units and ReLU activation.
For training, we use
SGD with momentum 0 . 9 , weight decay 0 . 0005 and set the learning rate to 0 .
We train the model via standard training and adversarial training respectively for 100 epochs.
We use (cid:96) ∞ PGD with step size α = 0 .
05 , maximum perturbation radius ε = 0 .
1 and run K = 5 iterations for AT.
[Figure]
A. HAT ALLEVIATES EXCESSIVE MARGIN
In this part, we check whether HAT can correct the failure mode of adversarial training for the toy problem studied in Sec. 3.
Fig.
8 shows the classiﬁcation boundary learnt with HAT.
Clearly, the decision boundary in the x 1 - x 3 plane is tilted and not completely parallel to the x 3 axis, indicating some dependence on the feature x 3 .
Thus, HAT indeed attenuates the problem of excessive margin which is also reﬂected in a 4% improvement in accuracy over adversarial training.
Additionally, it improves robustness from 73 .
8% to 74 .
B S EC . 3 CONTINUED : EVIDENCE ON CIFARB. E XPERIMENTAL SETUP
For the margin experiment, we train ResNet-18 (He et al., 2016) on CIFAR-10 (Krizhevsky, 2009) training set.
We simply adopt the set of hyperparameters and some improvements from DAWNBench (Coleman et al., 2017) submissions.
We use SGD optimizer with Nesterov momentum 0 .
9 (Nesterov, 1983) and weight decay 0 . 0005 .
We further use cyclic learning rates (Smith & Topin, 2018) with cosine annealing and a maximum learning rate of 0 . 21 .
We train the model forepochs via standard training.
Then, we perform adversarial ﬁne-tuning for 25 epochs with the same scheduler and learning rate settings.
Further, we evaluate margins on a set of 512 samples drawn uniformly at random from CIFAR10 test set for the visualizations in Fig.
4 (and Fig. 9).
B. A DDITIONAL MARGIN PLOTS
The margins along other adversarial directions R 10 , R 15 , R 20 before and after adversarial ﬁnetuning are displayed in Fig.
Here, R k corresponds to the adversarial directions computed at the k th iteration of adversarial ﬁne-tuning.
We see an increase in the margin along other adversarial directions as expected.
Nevertheless, the relative increase along these directions is not as large as compared to that along initial ones R
(see Fig. 4).
C F URTHER PERFORMANCE EVALUATION C. E XPERIMENTAL SETUP FOR S EC . 5.
In this section, we list all the details of our training and evaluation setup.
We run all our experiments thrice and report the average scores obtained unless stated otherwise.
Training setup.
We use ResNet-18 (He et al., 2016) for CIFAR-10 and CIFAR-100 (Krizhevsky, 2009); and PreAct ResNet-18 (He et al., 2016) for SVHN (Netzer et al., 2011).
For all our experiments, we use SGD optimizer with Nesterov momentum (Nesterov, 1983), momentum factor 0 .and
weight decay 0 .
We further employ cyclic learning rates (Smith & Topin, 2018) with cosine annealing and a maximum learning rate of 0 .
21 for CIFAR-10 and CIFAR-100; 0 .
05 for SVHN.
For CIFAR-10 and CIFAR-100, we train the models for 50 epochs with a batch size of 128 .
In the case of SVHN, we only train for 15 epochs.
For computing adversarial examples during training, we apply (cid:96) ∞ -PGD with the following hyperparameters: (cid:96) ∞ norm ε = 8 / 255 , step size α = 2 / 255 and run the attack for K = 10 iterations.
Note that we re-implement AT, TRADES and MART and train existing methods and HAT according to the aforementioned settings.
The hyperparameters of TRADES, MART and HAT are as follows.
For HAT, we ﬁx γ to 0 .
5 and use β = 2 . 5 for CIFAR-10 and SVHN; β = 3 . 5 for CIFAR-100.
The regularization parameter β for TRADES is set to 5 .
for CIFAR-10, SVHN and 6 . 0 for CIFAR-100.
For MART, we choose β = 5 . 0 .
We also use the same setup to train a standard model for computing helper labels during HAT training.
[Table]
Evaluation protocol.
During training, we perform early stopping (Rice et al., 2020) i.
[Table]
., we track the robustness of the model to PGD ( K = 20 ) on the test set and pick the model that performs the best for further evaluation.
In order to benchmark the (cid:96) ∞ robustness, we always test our models against AutoAttack (AA) (Croce & Hein, 2020) using the default code available at https:// github.com/fra31/auto-attack .
Setup for other attack conﬁgurations.
We reuse the exact aforementioned setup when investigating different threat models on CIFAR-10.
In the case of (cid:96) 2 perturbations, we use the following training adversary: ε = 128 / 255 , α = 15 / 255 and K = 10 .
We choose β = 2 .
5 and γ = 0 .
5 for
HAT and β = 5 . 0 for TRADES.
Whereas we set α = 4 / 255 , K = 10 for (cid:96) ∞ perturbations with norm ε = 12 / 255 .
Additionally, we pick β = 3 . 5 and γ = 0 .
5 for HAT and β = 6 . 0 for TRADES.
C. E XPERIMENTAL SETUP FOR S EC . 5.
For training the models enumerated in Table 4 of Sec.
5.3, we follow the setting designed by Gowal et al.
(2021) and Rebufﬁ et al.
(2021) respectively.
For the sake of completeness, we brieﬂy recap the training setup here.
We use SiLU activation function (Hendrycks & Gimpel, 2016) with PreAct
ResNet (He et al., 2016)/WideResNet (Zagoruyko & Komodakis, 2016) backbone.
The optimizer used is SGD with Nesterov momentum (Nesterov, 1983), momentum factor 0 .
9 and weight decay 0 . 0005 .
We further use cyclic learning rates (Smith & Topin, 2018) with cosine annealing, a maximum learning rate of 0 . 4 and a warmup of 10 epochs.
The training batch size is set to 1024 with 70% of the batch comprising of extra/synthetic data.
We also use model weight averaging (Izmailov et al., 2018) with decay τ = 0 .
995 and train the models for 400 CIFAR-10-equivalent epochs with the extra data from Carmon et al.
In the setting with synthetic DDPM-generated data, we apply 800 epochs.
The training attack used is PGD ( K = 10 ) with step size α = 2 / 255 and norm ε = 8 / 255 .
Finally, we perform early-stopping by tracking the performance on a disjoint validation set using PGD (K=40) with margin loss (Carlini & Wagner, 2017).
We separate ﬁrst 1024 samples from the training set for validation.
Note that we do not employ CutMix (Yun et al., 2019).
C. A DDITIONAL RESULTS FOR S EC . 5.
Detailed version of Table 2.
The complete performance evaluation with ResNet-18 across different datasets appears in Table 6.
Here, we also include the standard deviation over 3 runs for completeness.
It is clear that HAT outperforms other existing methods, hence narrowing the gap between accuracy and robustness.
Evaluation with other adversaries.
We additionally scrutinize the performance of robust models trained on CIFAR-10 against conventional adversaries.
In order to benchmark the (cid:96) ∞ robustness, we apply PGD with ε = 8 / 255 , α = 0 .
01 , K = 40 and r = 5 restarts following the evaluation protocol used in Carmon et al.
(2019), we denote this adversary as PGD+.
We also evaluate the vulnerability of our models to PGD with CW loss (Carlini & Wagner, 2017), ε = 8 / 255 , α = 0 .
01 and K = 40 steps.
Besides, we also test the vulnerability against naturally occurring benign corruptions.
Speciﬁcally, we use the CIFAR-10-C dataset from Hendrycks & Dietterich (2019).
The results presented in Table 7 show that HAT does not break down against other adversaries.
Crucially, HAT leads to 2 .
08% improvement against common corruptions.
This suggests that one needs to improve both accuracy as well as robustness in order to advance the performance under real-world distortions.
[Table]
combined
[Table]
Combination with other adversarial defenses.
To study the utility of HAT in concert with recent adversarial defenses
[Table]
we examine the performance of HAT when integrated with two schemes that improve over TRADES namely, (i) FAT (Zhang et al., 2020) which utilizes earlystopping when constructing adversarial data during training and (ii)
AWP (Wu et al., 2020) which encourages ﬂatness in weight loss landscape.
Note that although HAT is based on TRADES, it can be easily integrated into these defenses.
In our experiments, we follow the same training and evaluation setup as employed by Zhang et al.
(2020) and Wu et al.
(2020) respectively.
The resulting performance on CIFAR-10 is summarized in Table 8.
As shown in Table 8, incorporating HAT pushes the clean accuracy of FAT by 2% and also beneﬁts robust accuracy by 0 .
In combination with AWP, HAT leads to a 2 .
6% gain in clean accuracy relative to AWP.
These results underline that newer methods also suffer from the phenomenon of excessive margin and HAT indeed provides a general ﬁx to alleviate this problem.
[Table]
Larger datasets.
We also examine the performance of HAT on large-scale datasets such as
TinyImageNet-200 and ImageNet-100 (Deng et al., 2009).
ImageNet-100 (Laidlaw et al., 2021) is a 100-class subset of ImageNet.
We use a similar setup as Sec. 5.1 with PreAct ResNet-18 except the following changes.
We train for 30 epochs on TinyImageNet-200 against (cid:96) ∞ perturbations of size 8 / 255 .
Whereas on ImageNet-100, we train for 50 epochs with batch size 256 , weight decay 0 . 0001 , use input normalization and (cid:96)
∞ perturbations of size 4 / 255 .
On these datasets, we observed that
TRADES performed signiﬁcantly worse than AT.
This might be due to the use of KL-divergence loss instead of cross-entropy (CE) loss for crafting adversarial examples.
So, we use AT-based formulation of HAT which utilizes CE loss as robust loss as well as for computing adversarial perturbations.
For TRADES, we use β = 8 . 0 for TinyImageNet-200; β = 6 .
for ImageNet-100 while we pick β = 1 . 75 and γ = 1 . 0 for HAT on both the datasets.
The results are presented in Table 9.
Note that we measure robust accuracy using AutoAttack.
C. C HECKING FOR GRADIENT OBFUSCATION
We evaluate our models with AutoAttack (Croce & Hein, 2020) which has been consistently shown to provide a reliable evaluation of robustness.
Nevertheless, we also include additional sanity checks
[Table] [Table]
[Table] [Table]
to eliminate the possibility of gradient masking.
First
[Table]
following the guidelines in Athalye et al.
[Table]
we examine the impact of following changes on the robustness of a model trained via HAT on CIFAR-10 against (cid:96) ∞ perturbations of size 8 / 255 :
• Attack radius ε : Fig.
10 plots the robustness of HAT vs. the radius ε of PGD ( K = 40 ) attack.
As expected, increasing the attack budget causes the robust accuracy to monotonically drop to 0% .
In particular, unbounded PGD adversary reduces the robustness to 0% .
• Attack iterations K and random restarts r :
Increasing the number of attack iterations K or number of random restarts r only marginally lowers the robust accuracy (see Tableand Table 11).
In order words, the attack has converged and does not suffer from gradient obfuscation.
• As shown in Tables 12 and 13, transfer and non-transfer black box evaluations do not show any signs of gradient obfuscation.
We use square attack (Andriushchenko et al., 2020) for non-transfer black box evaluation.
Importantly, black box attacks have a lower success rate than white box attacks.
[Figure]
[Table]
Moreover, as a sanity check, we also visualize the loss landscapes of our trained models in Fig.
The loss landscapes are smooth and exhibit low curvature which are the typical characteristics of a robust network (Moosavi-Dezfooli et al., 2019).
In summary, these observations indicate that HAT does not lead to gradient obfuscation.
Evaluation with GAMA-PGD.
To further conﬁrm that our method does not suffer from gradient masking, we investigate the robustness of models reported in Table 4 against GAMA-PGD (Sriramanan et al., 2020), a stronger adversary than vanilla PGD attack.
The evaluation results are provided in Table 14.
As evident from Table 14, the performance of HAT does not break down even against GAMA attack which signiﬁes that HAT does not cause obfuscated gradients.
C. HAT: ANALYSIS ( CONTINUED )
Accuracy vs. robustness trade-off.
Fig.
12 compares the trade-off obtained by HAT with that of
AT and MART (Wang et al., 2020).
For HAT, we ﬁx γ = 0 .
and vary β ∈ [1 . 5 , 4 . 0] ; for MART, we vary β ∈ [1 . 0 , 5 . 0] .
HAT surpasses AT and MART by a large margin.
With MART, we observe notable improvements in robustness to weaker adversaries such as PGD (see Table 7), but the gains diminish when evaluated with a stronger adversary such as AutoAttack (Croce & Hein, 2020).
[Figure] [Figure]
Sensitivity of HAT to γ .
We here examine the impact of the weight of helper loss γ on the accuracy vs. robustness trade-off exhibited by HAT and conduct additional hyper-parameter sweeps.
We show the corresponding trade-off curves for HAT with γ ∈ { 0 . 0 , 0 .
75 } in Fig 13.
As expected, the trade-off curve for γ = 0 almost coincides with that of TRADES (note that there is a minor difference due to randomness).
Further, γ ∈ { 0 . 25 , 0 . 5 } achieve the best performance and identical trade-off curves.
With γ = 0 .
75 , the robust accuracy starts to deteriorate due to the dominance of helper loss over robust loss with the effect being more severe for lower values of β .
We believe that increasing γ beyond 0 .
75 would further negatively affect the performance.
Next, we conduct ablation studies on CIFAR-10 ( (cid:96) ∞ , 8 / 255 ) to better understand HAT algorithm.
While we elaborate on the HAT design choices below for the sake of completeness, note that we do not conduct any hyperparameter tuning concerning these choices and naively resort to the setting mentioned in Algorithm 1 throughout this work.
In the following analysis, we use the same experimental setup as in Sec. 5.1.
Impact of helper example deﬁnition.
We analyze the inﬂuence of different choices for deﬁning helper examples for HAT (refer Algorithm 1).
We consider the following choices: x + α r where r is the adversarial perturbation computed at x and α ∈ { 1 . 5 , 2 . 0 , 2 . 5 , 3 . 0 , 3 . 5 } .
Note that the helper label is always queried at x + r as in Algorihtm 1.
Selecting α = 2 . 0 means that the helper examples are given by x + 2 r which represents the setup used in this paper.
Fig.
14 illustrates the performance obtained by the resulting HAT-trained models.
A lower value for α e.g., α = 1 . 5 , hinders the model from increasing the margin and hence, the resulting model achieves high clean accuracy but a slightly poor robust accuracy.
α = 2 .
and α
= 2 .
5 achieve identical results, and the model does not compromise robustness yet achieves a gain in clean accuracy.
Further increasing α beyond 3 . 0 leads to a drop in both clean as well as robust accuracy.
This is because the resulting training samples might not be well separated and their target labels may conﬂict with each other.
[Figure]
deﬁnitions.
We deﬁne a helper example as
[Equation]
Choosing α = 2 . 0 corresponds to the setting used throughout this paper and is indicated by the red dashed line in the plot.
Note that the helper label is still queried at x + r .
We pick β = 2 . 5 for HAT.
3.5 : Helper label queried at x + r
C l e a n
A cc .
R o b u s t
A cc .
Clean Acc.
Robust Acc.
Ablation study for obtaining helper labels.
We perform an ablation study with different choices for obtaining the labels for helper examples during HAT training (refer Algorithm 1).
Speciﬁcally, we query the helper label ˜
y , via a regularly trained model f θ std , at x + α r where r is the adversarial perturbation computed for input x
and vary α ∈ { 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 , 2 . 5 , 3 . 0 , 3 . 5 } .
In other words, the helper label is deﬁned as the prediction of the standard model f θ std at x + α r ; whereas the helper example is always deﬁned to be x + 2 r .
Setting α = 1 .
recovers the setup used in this work.
The results are shown in Fig.
15 (solid lines).
For α ∈ { 0 . 5 , 1 . 0 , 1 . 5 } , we observe a similar performance relative to that reported in Table 2.
With α ≥ 2 . 0 , the clean accuracy of the robust model slightly increases whilst its robust accuracy starts to degrade.
This is because helper labels queried at farther points with α ≥ 2 .
might be much more pessimistic (worst-case) and hence, present more opposition to the robust loss.
Thus, we observe a drop in robust accuracy which can be in fact addressed by simply increasing the weight of robust loss.
This is indicated by the dashed lines in Fig.
15 where we increase the weight of robust loss β in proportion to the rise in α .
Taking this adjustment into account leads to the same performance as that with α = 1 . 0 , indicating that the underlying trade-off still remains the same.
C. C OMPARISON WITH C HEN ET AL . (2021)
The work by Chen et al.
(2021) is closest our approach in the sense that it also uses knowledge distillation or self-training.
However, the standpoint and the way of using it is signiﬁcantly different from our work.
We use distillation with a motivation to prevent excessive margin and beneﬁt clean accuracy, whilst Chen et al.
(2021) uses it from the usual perspective of using self-learning to improve generalization and thereby alleviate robust overﬁtting.
Moreover, it applies self-training at the adversarial inputs while we apply it for overly perturbed helper data.
From a computational perspective, the approach from Chen et al.
(2021) uses two self-teachers, one of which is adversarially trained, which presents a signiﬁcant overhead compared to HAT which requires only a regularly trained one.
In addition, Chen et al.
(2021) use model weight averaging which begs a larger number of training iterations to obtain improvements.
Empirically, the distillation setup from Chen et al.
(2021) which uses two self-teachers attains 83 .
67% accuracy and 48 .
03% robustness against AutoAttack.
Clearly,
HAT outperforms these results by achieving 84 .
90% accuracy and 49 .
08% robustness.
Moreover, we also replicate their setup with model weight averaging and perform HAT to further improve performance.
In contrast to their scores of 84 .
65% accuracy and 49 .
35% robustness, our proposed HAT obtains 85 .
69% accuracy and 49 .
34% robustness.