Towards Achieving Adversarial Robustness Beyond Perceptual Limits
Abstract
The vulnerability of Deep Neural Networks to
Adversarial Attacks has fuelled research towards building robust models.
While most existing Adversarial Training algorithms aim towards defending against imperceptible attacks, real-world adversaries are not limited by such constraints.
In this work, we aim to achieve adversarial robustness at larger epsilon bounds.
We ﬁrst discuss the ideal goals of an adversarial defense algorithm beyond perceptual limits, and further highlight the shortcomings of naively extending existing training algorithms to higher perturbation bounds.
In order to overcome these shortcomings, we propose a novel defense, Oracle-Aligned Adversarial
Training (OA-AT), that attempts to align the predictions of the network with that of an Oracle during adversarial training.
The proposed approach achieves state-of-the-art performance at large epsilon bounds ( (cid:96) ∞ bound of 16 / 255 ) while outperforming adversarial training algorithms such as AWP, TRADES and PGD-AT at standard perturbation bounds ( (cid:96) ∞ bound of 8 / 255 ) as well.
1. Introduction
Deep Neural Networks are known to be vulnerable to Adversarial Attacks, which are perturbations crafted with an intention to fool the network [24].
In a classiﬁcation setting, adversarially perturbed images cause the network prediction to ﬂip to unrelated classes, while causing no change in a human’s prediction (Oracle label).
The deﬁnition of adversarial attacks involves the presence of an Oracle and this makes it challenging to formalize threat models for the training and veriﬁcation of adversarial defenses.
The widely accepted convention used in practice is the (cid:96) p norm based threat model [4] with low-magnitude bounds to ensure im-
Lab, Department of Computational and Data Sciences, Indian Institute of Science, Banga- lore, India 2 Department of Computer Science and Engineering, Indian Institute of Technology (BHU), Varanasi, India. Correspon- dence to: Sravanti Addepalli < sravantia@iisc.ac.in > .
Accepted by the ICML 2021 workshop on A Blessing in Disguise: The Prospects and Perils of Adversarial Machine Learning. Copyright 2021 by the author(s).
cat dog dog dog frog bird ship auto auto truck truck aero frog
[Figure]
bird cat bird bird frog
[Figure]
perceptibility [10].
For example, attacks constrained within an (cid:96) ∞ norm of 8 / 255 on the CIFAR- 10 dataset are imperceptible to the human eye as shown in Fig.1(b), ensuring that the Oracle label is unchanged.
While low-magnitude (cid:96) p norm based threat models form a crucial subset of the widely accepted deﬁnition of adversarial attacks [9], they are not sufﬁcient, as there exist valid attacks at higher ε -bounds as well, as shown in Fig.1(g).
However, the challenge at large perturbation bounds is the existence of attacks that can ﬂip Oracle labels as well [27], as shown in Fig.1(c-e).
This makes it difﬁcult to naively scale existing Adversarial Training algorithms to large ε bounds.
In this work, we aim to improve robustness at larger epsilon bounds, such as an (cid:96) ∞ norm bound of 16 / 255 .
We deﬁne this as a moderate-magnitude bound, and discuss the ideal goals for achieving robustness under this threat model in Sec.3.
We further propose a novel defense OracleAligned Adversarial Training (OA-AT), which attempts to align the predictions of the network with that of an Oracle, rather than enforcing all samples within the constraint set to have the same label as the unperturbed image.
Our contributions have been summarized below:
• We deﬁne the ideal goals for a moderate- ε threat model ( (cid:96) ∞ radius of 16 / 255 ) and construct our goals as a feasible subset of the same.
• We propose methods for generating Oracle-Aligned adversaries, which can be used for adversarial training.
• We propose Oracle-Aligned Adversarial Training (OAAT) to improve robustness within the deﬁned moderateε threat model.
• We demonstrate superior performance when compared to state-of-the-art methods such as AWP [29], TRADES [30] and PGD-AT [17, 18] at ε = 16 /while also performing better at ε = 8 / 255 .
• We achieve improvements over the baselines even at larger model capacities such as ResNet-34 and WideResNet-34-10.
2. Related Works
Robustness against imperceptible attacks: Adversarial
Training has emerged as the most successful defense strategy against (cid:96) p norm bound imperceptible attacks.
PGD Adversarial Training (PGD-AT) [17] constructs multi-step adversarial attacks by maximizing Cross-Entropy loss within the considered threat model and subsequently minimizes the same for training.
This was followed by several adversarial training methods [30, 31, 19, 29, 22, 18] that improved accuracy against such imperceptible threat models further.
Zhang et al.
[30] proposed the TRADES defense, which maximizes the Kullback-Leibler (KL) divergence between the softmax outputs of adversarial and clean samples for attack generation, and minimizes the same in addition to the Cross-Entropy loss on clean samples for training.
Improving Robustness of base defenses: Wu et al.
[29] proposed an additional step of Adversarial Weight Perturbation (AWP) to perturb the weights of the model in order to maximize the training loss, and further train the perturbed model to minimize the same.
This generates a ﬂatter loss surface [23], thereby improving robust generalization.
While this can be integrated with any defense, AWP-TRADES is the state-of-the-art adversarial defense today.
On similar lines, the use of stochastic weight averaging of model weights [13] is also seen to improve the ﬂatness of loss surface, resulting in a boost in robustness [11, 5].
Recent works [18, 19, 11] attempt to ﬁnd the best training techniques such as early stopping, use of optimal weight decay and weight averaging to achieve enhanced robust performance on base defenses such as PGD-AT [17] and TRADES [30].
Robustness against large perturbation attacks: Shaeiri et al.
[20] demonstrate that the standard formulation of adversarial training is not well-suited for achieving robustness at large perturbations, as the loss saturates very early.
The authors propose Extended Adversarial Training (ExAT), where a model trained on low-magnitude perturbations ( ε = 8 / 255 ) is ﬁne-tuned with large magnitude perturbations ( ε = 16 / 255 ) for merely 5 training epochs, to achieve improved robustness at large perturbations.
The authors also discuss the use of a varying epsilon schedule to improve training convergence.
Friendly Adversarial Training (FAT) [31] performs early-stopping of an adversarial attack by thresholding the number of times the model misclassiﬁes the image during attack generation.
The threshold is increased over training epochs to increase the strength of the attack over training.
On similar lines, Sitawarin et al.
[21] propose Adversarial Training with Early Stopping (ATES), which performs early stopping of a PGD attack based on the margin of the perturbed image being greater than a threshold that is increased over epochs.
We improve upon these methods signiﬁcantly using our proposed approach (Sec.4).
3. Preliminaries and Threat Model 3.1. Notation
We consider an N -class image classiﬁcation problem with access to a labelled training dataset D .
The input images are denoted by x ∈ X and their corresponding labels are denoted as y ∈ { 1 , ..., N } .
The function represented by the Deep Neural Network is denoted by f θ where θ ∈ Θ denotes the set of trained network parameters.
The Ndimensional softmax output of the input image x is denoted as f θ ( x ) .
Adversarial examples are deﬁned to be images that are crafted speciﬁcally to fool a model into making an incorrect prediction [9].
An adversarial image corresponding to a clean image x would be denoted as (cid:101) x .
The set of all images within an (cid:96) p norm ball of radius ε , S ( x ) is deﬁned as, S ( x ) = { ˆ x : ||
ˆ x − x || p < ε } .
The set of all (cid:96) p norm bound adversarial examples, A ( x ) is deﬁned as,
A ( x ) = { (cid:101) x : f θ ( (cid:101) x ) (cid:54) =
y, (cid:101) x ∈ S ( x ) } .
In this work, we speciﬁcally consider robustness to (cid:96) ∞ norm bound adversarial examples.
We deﬁne the Oracle prediction of a sample x as the label that a human is likely to assign to the image, and denote it as O ( x ) .
For a clean image, O ( x ) would correspond to the true label y , while for a perturbed image it could differ from the original label.
3.2. Nomenclature of Adversarial Attacks
Tram ` er et al.
[27] discuss the existence of two types of adversarial examples: Sensitivity-based examples, where the model prediction changes, but the Oracle prediction remains the same as the unperturbed image, and Invariance-based examples, where the Oracle prediction changes, while the model prediction remains unchanged.
Models trained using standard empirical risk minimization are susceptible to sensitivity-based adversarial examples, while models which are overly robust to large perturbation bounds could be susceptible to invariance-based examples.
Since these definitions are dependent on the model being considered, we deﬁne a different nomenclature which only depends on the input image and the threat model considered, as below:
• Oracle-Invariant set OI ( x ) , is deﬁned as the set of all images within the bound S ( x ) , which preserve the Oracle label.
The Oracle is invariant to such perturbations:
[Equation]
• Oracle-Sensitive set OS ( x ) , is deﬁned as the set of all images within the bound S ( x ) , which ﬂip the Oracle label.
The Oracle is sensitive to such perturbations:
[Equation]
Objectives of the Proposed Defense
Defenses based on the conventional (cid:96) p norm based threat model deﬁned in Sec.3.1 attempt to train models which are invariant to all samples within S ( x ) .
This is an ideal requirement for low ε -bound perturbations, where the added noise is imperceptible, and hence all samples within the threat model are Oracle-Invariant.
An example of a low ε threat model is the constraint set deﬁned by ε = 8 / 255 for the CIFAR- 10 dataset, which produces adversarial examples that are perceptually similar to the corresponding clean images, as shown in Fig.1(b).
As we move to larger ε bounds, Oracle-labels begin to change, as shown in Fig.1(c, d, e).
For a very high perturbation bound such as 32 / 255 , the changes produced by an attack are clearly perceptible and cause a change in the Oracle label in many cases.
Hence, robustness at such large bounds may not be of much practical relevance.
The focus of this work is to achieve robustness within a moderatemagnitude (cid:96) p norm bound threat model, where some perturbations look partially modiﬁed (Fig.1(c)), while others look unchanged (Fig.1(g)), as is the case with ε = 16 /for CIFAR-10.
The existence of attacks that do not signiﬁcantly change the perception of the image necessitates the requirement of robustness within such bounds, while the existence of partially Oracle-Sensitive samples makes it difﬁcult to use standard adversarial training methods on the same.
The ideal goals for training defenses under this moderate-magnitude threat model are described below:
• Robustness against samples which belong to OI ( x ) • Sensitivity towards samples which belong to OS ( x ) , with model’s prediction matching the Oracle label • No speciﬁcation on Out-of-Distribution (OOD) images
We incorporate these goals as the training objective in our proposed defense, which is discussed in Sec.4.
Given the practical difﬁculty in assigning Oracle labels, we consider the following criteria for our defense evaluations:
• Robustness-Accuracy trade-off, measured using accuracy on clean samples and robustness against valid attacks within the threat model (discussed below) • Robustness against all attacks within ε = 8 / 255 , measured using strong white-box attacks [7, 22] • Robustness to Oracle-Invariant samples within ε = 16 / 255 , measured using gradient-free attacks [1]
4. Proposed Method
In order to achieve the goals discussed in Sec.3.3, we require to generate Oracle-Sensitive and Oracle-Invariant samples and impose speciﬁc training losses on each of them individually.
Since the labeling of adversarial samples as Oracle-Invariant or Oracle-Sensitive is expensive and cannot be done while training networks, we propose to use attacks which ensure a given type of perturbation (OI or OS) by construction, and hence do not require explicit annotation.
Generation of Oracle-Sensitive examples: Tsipras et al.
[28] show that robust models have perceptually aligned gradients.
Adversarial examples generated using a robust model tend to start looking like the target (other) class images at large perturbation bounds, as seen in Fig.1(c, d, e).
We therefore use large ε white-box adversarial examples generated from the model being trained as Oracle-Sensitive samples, and use the model’s prediction as a proxy to the prediction of the Oracle.
Generation of Oracle-Invariant examples:
While the strongest Oracle-Invariant examples are generated using the gradient-free Square attack [1], it uses 5000 queries, which is computationally expensive for use in adversarial training. Reducing the number of queries weakens the attack signiﬁcantly.
The most efﬁcient attack that is widely used for adversarial training is the PGD 10 -step attack.
However, it cannot be used for the generation of Oracle-Invariant samples as gradient-based attacks generated from adversarially trained models produce Oracle-Sensitive samples.
We propose to use the Learned Perceptual Image Patch Similarity (LPIPS) measure for the generation of Oracle-Invariant attacks, as it is known to match well with perceptual similarity [32, 16].
As shown in Fig.
A1, while the standard AlexNet model that is used in prior work [16] fails to distinguish between Oracle-Invariant and Oracle-Sensitive samples, an adversarially trained model is able to distinguish between the two types of attacks effectively.
We therefore propose to minimize the LPIPS distance between the natural and perturbed images, in addition to the maximization of Cross-Entropy loss for attack generation: L CE ( x, y ) − λ · LPIPS ( x, ˆ x ) .
We choose λ as the minimum value that transforms the attack from Oracle-Sensitive to Oracle-Invariant (OI), to generate strong OI attacks.
This is further ﬁne-tuned during training to achieve the optimal robustness-accuracy trade-off.
Oracle-Aligned Adversarial Training (OA-AT):
The training algorithm for the proposed defense, Oracle-Aligned
Adversarial Training (OA-AT) is presented in AlgorithmA1.
We use the Trades-AWP formulation [30, 29] as the base implementation in the proposed training algorithm.
We use the Cross-Entropy loss instead of KL-divergence loss for attack generation, as it generates stronger attacks [11].
Different from Wu et al.
[29], we maximize loss on x i +2 · (cid:101)
δ i (where (cid:101) δ
i is the attack) in the additional weight perturbation step, as it results in improved robust generalization.
We use cosine learning rate schedule.
We start with an initial ε value of 4 / 255 upto one-fourth
[Table]
the training epochs, and ramp up this value linearly to a value of 16 / 255 at the last epoch.
We use 5 attack steps when ε = 4 / 255 and 10 attack steps later.
We perform standard adversarial training upto ε = 12 / 255 as the attacks in this range are imperceptible.
Beyond this, we start incorporating separate training losses for Oracle-Invariant and Oracle-Sensitive samples in alternate training iterations.
Oracle-Sensitive samples are generated by maximizing Cross-Entropy loss in a PGD attack formulation.
Rather than enforcing the predictions of such attacks to be similar to the original image, we allow the network to be partially sensitive to such attacks by training them to be similar to a convex combinations of predictions on the clean image and perturbed samples at larger ( 1 . 5 · ε max ) bounds as shown:
[Equation]
Here (cid:101)
δ i is the perturbation at the varying epsilon value (cid:101) ε , and (cid:98)
δ i is the perturbation at 24 / 255 .
This results in better robustness-accuracy trade-off as shown in Table-A1.
In the other alternate iteration, we use the LPIPS metric to generate strong and efﬁcient Oracle-Invariant attacks during training.
We perform exponential weight-averaging of the network being trained and use this for computing the LPIPS metric for improved and stable results (Table-A1).
We increase the values of α and λ over training, as the nature of attacks changes with varying ε .
The use of both Oracle-Invariant and Oracle-Sensitive samples ensures that the proposed algorithm is robust to Oracle-Invariant samples and also agrees with the Oracle on the partially Oracle-Sensitive samples.
5. Experiments and Results
We compare performance of the proposed approach with the existing defenses discussed in Sec.2 on the CIFAR-
[15] dataset in Table-1.
We train all models on ResNetarchitecture for 110 epochs.
For each baseline, we ﬁnd the best set of hyperparameters to achieve clean accuracy of around 80% to ensure a fair comparison across all meth-
[Table]
ods. We also perform baseline training across various ε values and report the best baselines in Table-1.
We observe that baseline defenses do not perform well when trained using large ε bounds such as 16 / 255 (Table-A2).
We report adversarial robustness against the strongest known attacks, AutoAttack (AA) [7] and GAMA PGD-100 (GAMA)
[22] for ε = 8 / 255 in order to obtain the worst-case robust accuracy.
For larger bounds such as 12 / 255 and 16 / 255 , we primarily aim for robustness against the Square attack [1], as it is the strongest known Oracle-Invariant attack.
We compare the proposed approach against the strongest baseline AWP-TRADES [29] on CIFAR- 100 in Table-1 (ref.
Table-A3 for detailed results), and on CIFAR- 10 with larger capacity models in Table-2.
We observe signiﬁcant gains with the use of AutoAugment [8, 23] on CIFAR- 100 , and additionally with Model Weight Averaging (WA) [13, 11, 5] at larger model capacities.
To ensure a fair comparison, we consider these for the AWP baseline as well.
Results:
The proposed defense achieves consistent gains across all metrics considered in Sec.3.3 (AutoAttack [7] at ε = 8 / 255 and Square attack [1] at larger ε bounds).
Although we train the model for achieving robustness at larger ε bounds, we achieve an improvement in the robustness at ε = 8 / 255 as well, which is not observed in any of the existing methods (Table-A2).
We evaluate the proposed defense against diverse attacks (Table-A4) and sanity checks (Sec.A5) to ensure the absence of gradient masking.
6. Conclusions
In this work, we propose Oracle-Aligned Adversarial Training (OA-AT) for achieving robustness beyond perceptual limits.
We ﬁrst discuss the ideal goals of an adversarial defense at larger perturbation bounds, and further explore methods of effectively generating Oracle-Aligned adversaries during training.
The key aspects of the defense include the use of LPIPS metric for generating OracleInvariant attacks during training, and the use of a convex combination of clean and adversarial image predictions as targets for Oracle-Sensitive samples.
We achieve signiﬁcant gains in robustness at low and moderate perturbation bounds, and better robustness-accuracy trade-off as well. 7. Acknowledgements
This work was supported by Uchhatar Avishkar Yojana (UAY) project (IISC 10), MHRD, Govt. of India.
A1. Oracle-Invariant Attacks
Square Attack: The strongest Oracle-Invariant examples are generated using the Square attack [1].
These images are Oracle-Invariant since this is a query-based attack and does not use gradients from any model for attack generation.
However this attack uses 5000 queries, and is a computationally expensive attack.
Hence this attack cannot be used for adversarial training, although it is one of the best attacks for evaluations.
We note that reducing the number of queries makes it computationally efﬁcient, however it also reduces the effectiveness of the attack signiﬁcantly.
PGD based Attacks: While the most efﬁcient attack that is widely used for adversarial training is the PGD 10 -step attack, it cannot be used for the generation of Oracle-Invariant samples as adversarially trained models have perceptually aligned gradients, and tend to produce Oracle-Sensitive samples.
Therefore, we explore some variants of the PGD attack to make the generated perturbations Oracle-Invariant.
We denote the Cross-Entropy loss on a data sample x with ground truth label y using L CE ( x, y ) .
We explore the addition of regularizers to the Cross-Entropy loss weighted by a factor of λ X in each case.
The value of λ X is chosen as the minimum value which transforms the PGD attacks from Oracle-Sensitive to Oracle-Invariant.
This results in the strongest possible Oracle-Invariant attacks.
Discriminator based PGD Attack: We train a discriminator to distinguish between Oracle-Invariant and OracleSensitive adversarial examples, and further maximize the below loss for the generation of Oracle-Invariant attacks:
[Equation]
Here L BCE (ˆ x, OI ) is the Binary Cross-Entropy loss of the adversarial example ˆ x w.r.t. the label corresponding to an Oracle-Invariant (OI) attack.
We train the discriminator to distinguish between two input distributions; the ﬁrst corresponding to images concatenated channel-wise with their respective Oracle-Sensitive perturbations, and a second distribution where perturbations are shufﬂed across images in the batch.
This ensures that the discriminator relies on the spatial correlation between the image and its corresponding perturbation for the classiﬁcation task, rather than the properties of the perturbation itself.
The attack in Eq. A therefore attempts to break the most salient property of Oracle-Sensitive attacks, which is the spatial correlation between an image and its perturbation. [Figure]
LPIPS based PGD Attack:
We propose to use the
Learned Perceptual Image Patch Similarity (LPIPS) measure for the generation of Oracle-Sensitive attacks, as it is known to match well with perceptual similarity [32, 16].
As shown in Fig.
A1, while the standard AlexNet model that is used in prior work [16] fails to distinguish between
Oracle-Invariant and Oracle-Sensitive samples, an adversarially trained model is able to distinguish between the two types of attacks effectively.
In this plot, we consider attacks generated from a PGD-AT [17, 18] model (Fig.1(c-e)) as
Oracle-Sensitive attacks, and attacks generated from a Normally Trained model (Fig.1(h))
as Oracle-Invariant attacks.
We therefore propose to minimize the LPIPS distance between the natural and perturbed images, in addition to the maximization of Cross-Entropy loss for attack generation as shown below:
[Equation]
We choose λ LPIPS as the minimum value that transforms the PGD attack from Oracle-Sensitive to Oracle-Invariant (OI), to generate strong OI attacks.
This is further ﬁnetuned during training to achieve the optimal robustnessaccuracy trade-off. As shown in Fig.
A2, setting λ LPIPS to 1 changes adversarial examples from Oracle-Sensitive to Oracle-Invariant, as they look similar to the corresponding original images shown in Fig. A2(a). This can be observed more distinctly at perturbation bounds of 24 / 255 and 32 / 255 .
The perturbations in Fig.
A2(c) are smooth, while those in (e) and (g) are not.
This shows that the addition of cat dog dog dog dog cat cat dog dog cat horse dog dog ship auto auto auto auto ship ship auto auto ship truck truck ship frog cat cat frog frog frog frog cat deer frog cat deer frog frog frog frog frog frog frog frog auto frog frog auto frog frog cat bird bird dog dog cat cat bird dog cat bird bird dog (a)
{+Clean (b) 16/255( = 0) (c) 16/255( = 0) (noise) (d) 16/255( = 1) (e) 16/255( = 1) (noise) (f) 16/255( = 2) (g) 16/255( = 2) (noise) (h) 24/
+}
( = 0)
{+(i) 24/
 ( = 1) (j) 24/255( = 2) (k) 32/255( = 0) (l) 32/255( = 1) (m) 32/255( = 2) [Figure]+}
the LPIPS term helps in making the perturbations OracleInvariant. Very large coefﬁcients of the LPIPS term make the attack weak as can be seen in Fig. A2(f, j, m) where the model prediction is same as the true label. We therefore set the value of λ LPIPS to 1 to obtain strong Oracle-Invariant attacks.
As shown in Table-A1, while we obtain the best results using the LPIPS based PGD attack for training (E1), the use of discriminator based PGD attack (E6) also results in a better robustness-accuracy trade-off when compared to E2, where there is no explicit regularizer to ensure the generation of
Oracle-Invariant attacks.
Evaluation of the proposed defense against OracleInvariant Attacks: We compare the performance of the proposed defense OA-AT with the strongest baseline AWP [29] against the two proposed Oracle-Invariant attacks,
LPIPS based attack and Discriminator based attack in
Fig.
A3 (a) and (b) respectively.
We vary the coefﬁcient of the regularizers used in the generation of attacks, λ Disc (Eq.A1) and λ LPIPS (Eq.A2) in each of the plots.
As we increase the coefﬁcient, the attack transforms from OracleSensitive to Oracle-Invariant.
We achieve better performance across all values of the coefﬁcients when compared to the baseline.
A2. Details on the Datasets used
We evaluate the proposed approach on the CIFAR-10 and
CIFAR-100 [15] datasets.
The two datasets consist of RGB images of spatial dimension 32 × 32, and contain 10 anddistinct classes respectively.
CIFAR-10 is the most widely used benchmark dataset to perform a comparative analysis across different adversarial defense and attack methods.
CIFAR-100 is a challenging dataset to achieve adversarial robustness given the large number of diverse classes that are interrelated.
Each of these datasets consists of 50, training images and 10,000 test images.
We split the original training set to create a validation set of 1,000 images in CIFAR-10 and 2,500 images in CIFAR-100.
We ensure that the validation split is balanced equally across all classes, and use the remaining images for training.
To ensure a fair
1: Input: Deep Neural Network f θ with parameters θ ,
Training Data { x i , y i } Mi =1 , Epochs T , Learning Rate η , Perturbation budget ε max , Adversarial Perturbation function A ( x, y, (cid:96), ε ) which maximises loss (cid:96) 2: for epoch
= 1 to T do 3: (cid:101)
ε
= max { ε max / 4 , ε max · epoch /T } 4: for i = 1 to M do 5: δ i ∼ U ( − min( (cid:101) ε, ε max / 4) , min( (cid:101) ε, ε max / 4))
6: if (cid:101) ε < 3 / 4 · ε max then 7: (cid:96) =
(cid:96) CE
( f θ ( x i + δ i ) , y i ) 8: (cid:101) δ i
=
A ( x i , y i , (cid:96), (cid:101) ε ) 9:
L adv = KL (cid:0) f
θ ( x i + (cid:101) δ i ) ||
f θ ( x i ) (cid:1) 10: else if i % 2 = 0 then 11: (cid:96) =
(cid:96) CE
( f θ ( x i + δ i ) , y i ) 12: (cid:98) δ i
=
A ( x i , y i , (cid:96), 1 . 5 · ε max ) 13: (cid:101) δ i =
Π ∞ ( (cid:98) δ i , (cid:101) ε ) 14:
L adv = KL (cid:0) f
θ ( x i + (cid:101) δ i ) ||
α · f θ ( x i ) + (1 − α ) · f θ ( x i + (cid:98) δ i ) (cid:1) 15: else 16: δ i ∼ U ( − (cid:101) ε, (cid:101) ε ) 17: (cid:96) =
(cid:96) CE
( f θ ( x i + δ i ) , y i ) − LPIPS( x i , x i + δ i ) 18: (cid:101) δ i
=
A ( x i , y i , (cid:96), (cid:101) ε ) 19:
L adv = KL (cid:0) f
θ ( x i + (cid:101) δ i ) ||
f θ ( x i ) (cid:1) 20: end if 21:
L = (cid:96) CE
( f θ ( x i ) , y i )
+ L adv 22: θ = θ − η · ∇ θ L 23: end for 24: end for comparison, we use the same split for training the proposed defense as well as other baseline approaches. For both datasets, we consider the (cid:96) ∞ threat model of radius 8 /to be representative of imperceptible perturbations, that is, the Oracle label does not change within this set. Further, we consider the (cid:96) ∞ threat model of radius 16 / 255 to investigate robustness within moderate magnitude perturbation bounds.
A3. Details on Training
The algorithm for the proposed method as explained in Sec. is presented in Algorithm-A1.
We use a varying ε schedule and start training on perturbations of magnitude ε = 4 / 255 .
This results in marginally better performance when compared to ramping up the value of ε from 0 (E8 of Table-A1).
For CIFAR-10 training on ResNet-18, we set the weight of the adversarial loss L adv in L21 of Alg. A1 ( β parameter of TRADES [30]) to 1.5 for the ﬁrst three-quarters of training, and then linearly increase it from 1.5 to 3 in the moderate perturbation regime, where ε is linearly increased from 12 / 255 to 16 / 255 .
In this moderate perturbation regime, we also linearly increase the coefﬁcient of the LPIPS distance ( ) from 0 to 1, and linearly decrease the α parameter used in the convex combination of softmax prediction (Alg.A1, L14) from 1 to 0.8.
This results in a smooth transition from adversarial training on imperceptible attacks to attacks with larger perturbation bounds.
We set the weight decay to 5e-4.
For all our experiments, we use the cosine learning rate schedule with 0.2 as the maximum learning rate.
We use
SGD optimizer with momentum of 0.9, and train forepochs.
We compute the LPIPS distance using an exponential weight averaged model with τ = 0 .
[-[Table] .+1 .+0 .+
 .+2 .+
 .+1 .+
-]We note from
[-.+3 .+
 .nential weight averaging with τ = 0 . 995 for generating-]Table-A1 that the use of weight-averaged model results in better performance when compared to using the model being trained for the same (E5).
This also leads to more stable results across reruns.
We utilise AutoAugment [8] for training on CIFAR-100, and for CIFAR-10 training on large model capacities.
We apply AutoAugment with a probability of 0 .
5 for CIFAR-100, and for the CIFAR-10 model trained on ResNet-34.
Since the extent of overﬁtting is higher for large model capacities, we use AutoAugment with p = 1 on WideResNet-34-10.
While the use of AutoAugment helps in overcoming overﬁtting, it could also negatively impact robust accuracy due to the drift between the training and test distributions.
We observe a drop in robust accuracy on the CIFAR-10 dataset with the use of AutoAugment (E11, E12 in Table-A1), while there is a boost in the clean accuracy.
On similar lines, we observe a drop in robust accuracy on the CIFAR-100 dataset as well, when we increase the probability of applying AutoAugment from 0 . 5 (E11 in Table-A1) to 1 (E12 in Table-A1).
To investigate the stability of the proposed approach, we train a ResNet-18 network multiple times by using different random initialization of network parameters.
We observe that the proposed approach is indeed stable, with standard deviation of 0 . 180 and 0 .
143 for clean accuracy, GAMA PGD-100 accuracies with ε = 8 /and 16 / 255 , and accuracy against the Square attack with ε = 16 / 255 respectively over three independent training runs on CIFAR-10.
We also observe that the last epoch is consistently the best performing model for the ResNetarchitecture.
Nonetheless, we still utilise early stopping on the validation set using PGD 7-step accuracy for all the baselines to enable a fair comparison overall.
A3.1. Ablation Study
In order to study the impact of different components of the proposed defense, we present a detailed ablative study using
ResNet-18 models in Table-A1.
We present results on the CIFAR-10 and CIFAR-100 datasets, with E1 representing the proposed approach.
First, we study the efﬁcacy of the LPIPS metric in generating Oracle-Invariant attacks.
In experiment E2, we train a model without LPIPS by setting [Table] its coefﬁcient to zero.
While the resulting model achieves a slight boost in robust accuracy at ε = 16 / 255 due to the use of stronger attacks for training, there is a considerable drop in clean accuracy, and a corresponding drop in robust accuracy at ε = 8 / 255 as well.
We observe a similar trend by setting the value of α to 1 as shown in E3, and by combining E2 and E3 as shown in E4.
We note that E4 is similar to standard adversarial training, where the model attempts to learn consistent predictions in the ε ball around every data sample.
While this works well for large ε attacks ( ε = 16 / 255 ), it leads to poor clean accuracy as shown in the ﬁrst partition of Table-A2.
As discussed in Sec.4, we maximize loss on x i + 2 · (cid:101)
δ i (where (cid:101) δ
i is the attack) in the additional weight perturbation step.
We present results by using the standard ε limit for the weight perturbation step as well, in E7.
This leads to a drop across all metrics, indicating the importance of using large magnitude perturbations in the weight perturbation step for producing a ﬂatter loss surface that leads to better generalization to the test set.
Different from the standard
TRADES formulation, we maximize Cross-Entropy loss for attack generation in the proposed method.
From E9 we note that the use of KL divergence leads to a drop in robust accuracy since the KL divergence based attack is weaker.
This is consistent with the observation by Gowal et al. A4. Detailed Results
In Tables-A2 and A3, we present results of different defense methods such as AWP-TRADES [29], TRADES [30], PGDAT [17], ExAT [20], ATES [21] and FAT [31], evaluated across a wide range of adversarial attacks.
We present evaluations on the Black-Box FGSM attack [10] and a suite of
White-Box attacks, on (cid:96) ∞ constraint sets of different radii: 8 / 255 , 12 / 255 and 16 / 255 .
The white-box evaluations consist of the single-step Randomized-FGSM (R-FGSM) attack [26], the GAMA PGD-100 attack [22] and AutoAttack [7], with the latter two being amongst the strongest of attacks known to date.
Lastly, we also present evaluations on the Square attack [1] for ε = 12 / 255 and 16 / 255 in order to evaluate performance on Oracle-Invariant samples at large perturbation bounds.
CIFAR-10:
To enable a fair comparison of the proposed approach with existing methods, we present comprehensive results of various defenses trained with different attack strengths in Table-A2.
In the ﬁrst partition of the table, we present baselines trained using attacks constrained within an (cid:96) ∞ bound of 16 / 255 .
While these models do achieve competitive robustness on adversaries of attack strength ε = 8 / 255 , 12 / 255 and 16 / 255 , they achieve signiﬁcantly lower accuracy on clean samples which limits their use in practical scenarios.
Thus, for better comparative analysis that accounts for the robustness-accuracy trade-off, we present results of the existing methods with hyperparameters and attack strengths tuned to achieve the best robust performance, while maintaining clean accuracy close to 80% as commonly observed on the CIFAR-10 dataset on ResNet-18 architecture, in the second partition of Table-A2.
We observe that the proposed method OA-AT consistently outperforms other approaches on all three metrics described in Sec.3.3, by achieving enhanced performance on both Oracle-Sensitive and Oracle-Invariant attacks, while striking a favourable robustness-accuracy trade-off.
Furthermore, the proposed defense is seen to have better robust performance even on the standard (cid:96) ∞ constraint set of 8 / 255 as compared to existing approaches, despite being trained on larger perturbations sets.
CIFAR-100: In Table-A3, we present results on models trained on the highly-challenging CIFAR-100 dataset.
Since this dataset contains relatively fewer training images per class, we seek to enhance performance further by incorporating the augmentation technique, AutoAugment [8, 23]. [Table] [Table]
AWP [29] is the strongest baseline. The baselines are sorted by AutoAttack accuracy [7] (AA 8 / 255 ).
The proposed defense achieves signiﬁcant gains in accuracy against the strongest attacks across all ε bounds. Since the proposed defense uses AutoAugment [8] as the augmentation strategy, we present result on the strongest baseline AWP [29] with AutoAugment as well
[Table]
To enable fair comparison, we incorporate AutoAugment for the strongest baseline, AWP [29] as well. We observe that the proposed method consistently performs better than existing approaches by signiﬁcant margins, both in terms of clean accuracy, as well as robustness against adversarial attacks conforming to the three distinct constraint sets.
Further, this also conﬁrms that the proposed method scales well to large, complex datasets, while maintaining a consistent advantage in performance compared to other approaches.
A5. Gradient Masking Checks
As discussed by Athalye et al.
[2], we present various checks to ensure the absence of Gradient Masking in the proposed defense.
In Fig.
A4(a,c), we observe that the accuracy of the proposed defense on the CIFAR-10 and CIFAR-100 datasets monotonically decreases to zero against 7-step PGD whitebox attacks as the perturbation budget is increased.
This shows that gradient based attacks indeed serve as a good indicator of robust performance, as strong adversaries of large perturbation sizes achieve zero accuracy, indicating the absence of gradient masking.
In Fig.
A4(b,d), we plot the Cross-Entropy loss against FGSM attacks with varying perturbation budget.
We observe that the loss increases linearly, thereby suggesting that the ﬁrst-order Taylor approximation to the loss surface indeed remains effective in the local neighbourhood of sample images, again indicating the absence of gradient masking.
We verify that the model achieves higher robust accuracy against weaker Black-box attacks, as compared to strong gradient based attacks such as GAMA or AutoAttack in Tables-A2,A3. We also observe that adversaries that con- [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
form to larger constraint sets are stronger than their counterparts that are restricted to smaller epsilon bounds, as is expected.
In Table-A4, we perform exhaustive evaluations using various attack techniques to further verify the absence of gradient masking.
In addition to AutoAttack [7] which in itself consists of an ensemble of four attacks- AutoPGD with Cross-Entropy and Difference-of-Logits loss, the FAB attack [6] and Square Attack [1], we present evaluations against strong multi-targeted attacks such as GAMA-MT [22] and the MDMT attack [14] which speciﬁcally target other classes during optimization.
We also consider the untargeted versions of the latter two attacks, the GAMA-PGD and MD attack respectively.
We also present robustness against the ODS attack [25] with 100 restarts, which diversiﬁes the input random noise based on the output predictions in order to obtain results which are less dependent on the sampled random noise used for attack initialization.
Next, the Logit-Scaling attack [3, 12] helps yield robust evaluations that are less dependent on the exact scale of output logits predicted by the network, and is seen to be effective on some defenses which exhibit gradient masking.
However, we observe that the proposed method is robust against all such attacks, with the lowest accuracy being attained on the AutoAttack ensemble.
Furthermore, we evaluate the model on PGD 50-step attack run with 1000 restarts.
The robust accuracy saturates with increasing restarts, with the ﬁnal accuracy still being higher than that achieved on AutoAttack.
Lastly, we observe that the PGD-1000 attack is not very strong, conﬁrming that the accuracy does not continually decrease as the number of steps used in the attack increases.
Thus, we observe that the proposed approach is robust against a diverse set of attack methods, thereby conﬁrming the absence of gradient masking and verifying that the model is truly robust.