Is RobustBench/AutoAttack a suitable Benchmark for Adversarial Robustness?
Abstract
Recently, RobustBench (Croce et al. 2020) has become a widely recognized benchmark for the adversarial robustness of image classification networks.
In its most commonly re- ported sub-task, RobustBench evaluates and ranks the adversarial robustness of trained neural networks on CIFAR10 un- der AutoAttack (Croce and Hein 2020b) with l ∞ perturbations limited to ϵ = 8 / 255 .
With leading scores of the currently best performing models of around 60% of the baseline, it is fair to characterize this benchmark to be challenging.
Despite its general acceptance in recent literature, we aim to foster discussion about the suitability of RobustBench as a key indicator for robustness which could be generalized to practical applications.
Our line of argumentation against this is two-fold and supported by excessive experiments presented in this paper:
We argue that I) the alternation of data by AutoAttack with l ∞ , ϵ = 8 / 255 is unrealistically strong,
result- ing in close to perfect detection rates of adversarial samples even by simple detection algorithms while other attack meth- ods are much harder to detect and achieve similar success rates, II) results on low resolution data sets like CIFAR10 do not generalize well to higher resolution images as gradient based attacks appear to become even more detectable with increasing resolutions.
Source code: github.com/adverML/SpectralDef Framework
Introduction
Increasing the robustness of neural network architectures against adversarial examples in general and more specifically against coordinated adversarial attacks, has recently received increasing attention.
In this work, we focus on the benchmarking of robustness in the context of CNN based computer vision models.
RobustBench.
In 2020, (Croce et al. 2020) launched a benchmark website 1 with the goal to provide a standardized benchmark for adversarial robustness on image classification models.
Until then, single related libraries such as FoolBox (Rauber, Brendel, and Bethge 2018), Cleverhans (Papernot et al. 2016) and AdverTorch (Ding, Wang, and Jin Copyright ©
Association for the Advancement of Artificial
2019) were already available but did not include all state-ofthe-art (SOTA) methods in one evaluation.
The current rankings in RobustBench as well as the majority of evaluations of adversarial robustness in recent literature are dominated by RobustBench’s own attack scheme AutoAttack (Croce and Hein 2020b).
AutoAttack is an ensemble of 4 attacks two variations of the Projected Gradient Descent (PGD) (Madry et al.
2018) attack with cross-entropy loss (APGD-CE) and difference of logits ratio loss (APGDt), the targeted version of the FAB attack (Croce and Hein 2020a), and the black-box Squares attack (Andriushchenko et al. 2020).
Contributions
The aim of this paper is to raise the awareness that RobustBench’s AutoAttack in its default evaluation scheme l ∞ , ϵ = 8 / 255 is unrealistically strong, resulting in close to perfect detection rates of adversarial samples even by simple detection algorithms.
Also we find that benchmarks on low resolution datasets like CIFAR10 tend to underestimate the strength of adversarial attacks and can not directly generalized to applications with higher resolutions.
In detail, we show that:
• adversarial samples generated by AutoAttack l ∞ , ϵ = 8 / 255 are modifying test images to the extent that these manipulations can easily be detected, almost entirely preventing successful attacks in practice.
• given a simple defense, AutoAttack is outperformed by other existing attacks even for optimized ϵ parameters.
• in contrast to other methods, the effectiveness of AutoAttack is dropping with increasing image resolutions.
Methods 2. Attack Methods
For our analysis, we generate test data using AutoAttack and a baseline of five other commonly used attack methods from the foolbox (Rauber, Brendel, and Bethge 2018).
We employ the untargetet version of all attacks, if available.
AutoAttack (AA): RobustBench is based on the evaluation of AA (Croce and Hein 2020b), which is an ensemble of 4 parameter-free attacks: two variations of the PGD attack (Madry et al. 2018)
(see Section 2.1) with cross-entropy loss (APGD-CE) and difference of logits ratio loss (APGD-t):
[Equation]
where π is ordering of the components of z in decreasing order.
The APGD-t can handle models with a minimum of 4 classes.
The targeted version of the FAB attack (Croce and
Hein 2020a), and the Black-Box (BB) Squares attack (Andriushchenko et al. 2020).
The AA framework provides two modes.
RobustBench uses the “standard” mode, executing the 4 attack methods consecutively.
The failed attacked samples are handed over to the next attack method, to ensure an higher attack rate.
Fast Gradient Method (FGSM): The FGSM (Goodfellow, Shlens, and Szegedy 2015) uses the gradients of the
Deep Neural Network (DNN) to create adversarial examples.
For an input image, the method uses the gradients of the loss w.r.t.
the input image to create a new image that maximises the loss.
This output is called the adversarial image.
This following expression summarizes this:
[Equation]
where X adv is the adversarial image, X is the original input image, y is the original input label, ε is the multiplier to ensure the perturbations are small and J is the loss.
There is no guarantee that the generated adversarial examples by this method are similar to its real counterpart.
Basic Iterative Method (BIM): The method BIM (Kurakin, Goodfellow, and Bengio 2017) is the iterative version of FGSM.
After each iteration the pixel values need to be clipped to ensure the generated adversarial examples is still within the range of both the ε ball (i.e. [ x − ε, x + ε ] ) and the input space (i.e. [0 , 255] for the pixel values).
The formulation is expressed as follows:
[Equation]
where N denotes the number of iterations.
Projected Gradient Descent (PGD): The PGD (Madry et al. 2018) is a variant of BIM and one of the most popular white-box (allowing full access to model gradients and weights) attacks.
It introduces random initialization of the perturbations for each iteration.
This algorithm strives to find the perturbation that maximizes a model’s loss on a particular input.
The size of the perturbation is kept smaller than an amount by ϵ .
This constraint is expressed ether as l 2 or l ∞ norm.
DeepFool (DF): The DF is a non-targed method that is able to find the minimal amount of perturbations possible which mislead the model using an iterative linearization approach (Moosavi-Dezfooli, Fawzi, and Frossard 2016).
The main idea is to find the closest distance from the input sample to the model decision boundary.
Carlini&Wagner (C&W): The attack method Carlini&Wagner (C&W) (Carlini and Wagner 2017) is based on the L-BFGS and has three versions: l 0 , l 2 and l ∞ .
We employ the l 2 variant which is most commonly used.
This attack method generates for an given input X an adversarial example X adv by formulating following optimization problem:
[Equation]
where Z ( x ) is the softmax classification result vector.
The initial value for c is c = 10 − 3 , a binary search is performed to then find the smallest c , s.t. f
( X adv ) ≤ 0 .
2. Measuring the Success of Adversarial Attacks
RobustBench , like most of the benchmarks in literature regarding adversarial robustness, uses a Robust Accuracy (Croce et al. 2020) measure to compare different methods.
However, this approach does not fit our evaluation scheme, since we are aiming to measure the success of adversarial samples under defence in order to obtain a more realistic view on the practical impact of the applied attacks.
Therefore, we reformulate the robustness measures and report two different indicators:
Attack Success Rate (ASR)
The Adversarial Succes Rate (ASR) in eq.
(1) is calculated as
[Equation]
the fraction of successfully perturbed test images and it provides a baseline of an attacker’s ability to fool unprotected target networks.
Hence, ASR is providing the same information as Robust Accuracy from an attackers perspective.
Attack Success Rate under Defense (ASRD)
We extend
ASR by the practical assumption that too strong perturbations can be detected at inference time.
To measure the performance of attacks under defense, we introduce the Adversarial Success Rate under Detection (ASRD) in eq.
(2), computing the ratio of successful attacks
ASRD
[Equation]
undetected perturbations # all samples
[Equation]
where FNR is the false negative rate of the applied detection algorithm.
2. A Simple Adversarial Detector
In order to measure the magnitude of perturbations imposed by RobustBench , we apply a simple and easy to implement adversarial detector introduced in (Harder et al. 2021;
Lorenz et al. 2021).
This method is based on a feature extraction in the Fourier domain, followed by a Logistic Regression or Random Forest classifier.
It can be applied in a black-box fashion, using only the (adversarial) input images, or as white-box detector accessing the feature maps of attacked neural networks.
In both cases, the detector is based on a Fourier transformation (Cooley and Tukey 1965):
For a discrete 2D signal, like color image channels or single CNN feature maps – X ∈ [0 , 1] N × N – the 2D discrete Fourier transform is given as
[Equation]
for l, k = 0 , . . .
N − 1 , with complex valued Fourier coefficients F ( X )( l, k ) .
The detector then only utilizes the magnitudes of Fourier coefficients
[Equation]
to detect adversarial attacks with high accuracy.
Black-Box Detection: Fourier Features of Input Images
While different attacks show distinct but randomly located change patterns in the spatial domain (which makes them hard to detect), (Harder et al. 2021) showed that adversarial samples have strong, well localized signals in the frequency domain.
Hence, the detector extracts and concatenates the 2D power spectrum of each color channel as feature representations of input images and uses simple classifiers like Random Forests and Logistic Regression to learn to detect perturbed input images.
White-Box Detection: Fourier Features of Feature-Maps
In the White-Boxcase, the detector applies the same method as in the Black-Boxapproach, but extends the inputs to the feature map responses of the target network to test samples.
Since this extension will drastically increase the feature space for larger target networks, only a subsets of the available feature maps are selected.
In original paper (Harder et al. 2021) and in the follow-up paper (Lorenz et al. 2021), it is stated that a combination of several layers delivers better detection results.
Experiments
Since most of the successful methods ranked on Robustbench are based on a WideResNet28-10 (Zagoruyko and
Komodakis 2017) architecture, we also conduct our evaluation on a baseline WideResNet28-10 using the following datasets without applying adversarial examples or other methods to increase the robustness during training.
CIFAR10.
We train on the plain CIFAR10 training set to a test-accuracy of 87% and apply the different attacks on the test set.
Then, we extract the spectral features and use a random subset of 1500 samples of this data for each attack method to evaluate ASR and ASRD .
CIFAR100.
The procedure is similar to CIFAR10 dataset.
We train on the CIFAR100 training set to a test-accuracy of 79% and apply the attacks on the test set.
ImageNet-32.
(64 and 128.)
This dataset (Chrabaszcz,
Loshchilov, and Hutter 2017) (and its variants 64 × 64 and 128 × 128 pixels) has the exact same number of classes (1000) and images as the original ImageNet with the only difference that the images are downsampled.
Moreover, a lower resolution of the images makes the classification task more difficult and the baseline test accuracy is 66% and 77% respectively.
CelebaHQ-32.
(64 and 128.)
This dataset (Liu et al. 2015) provides images of celebrities faces in HQ quality ( 1024 × 1024 px ) whereas we downsampled it to 32 , 64 and 128 pixels width and height.
We only selected the attributes “Brown
Hair”, “Blonde Hair”, “Black Hair” and “Gray Hair” to train the Wide Residual Networks (WRN) to an test-accuracy of 91%.
The data is unbalanced, where the class “Gray Hair” has least samples.
3. Detecting Attacks
Figures 1 and 2 show a subset of white-box and black-box
ASRD results for all attack methods on datasets with a resolution of 32 × 32 2 .
In both cases, AutoAttack has very low
ASRD rates, not only compared to other methods but also in absolute values.
In most cases, the probability of successful
AA attacks is marginally low.
3. AutoAttack for different choices of ϵ
One might argue that the low ASRD rates of AA might be caused by too high choice of ϵ .
Hence, we repeat the full set of AutoAttack experiments for a full range of different ϵvalues.
Figures 3 and 4 show a subset of these evaluation for
ImageNet and CelebHQ on different ϵ , image resolutions as well as WB and BB detectors with Random Forests 3 .
3. Success Rates depending on Image Resolution
As shown in Figure 5 and 6, we compare the ASRD over the three image size ( s = { 32 , 64 , 128 } ) on the datasets CelebaHQ and ImageNet.
The attacks FGSM, BIM, PGD and
AA are sensitive to the image size.
The used detector has better results as the image size is increased.
In contrast, DF and C&W keep their attack strength over all image sizes s .
Again, AA does not show sufficient results for using adversarial detection robustness.
Discussion
The results of our empirical evaluations show strong evidence that the widely used AutoAttack scheme for benchmarking the adversarial robustness of image classifier models on low resolution data might not be a suitable setup in order to generalize the obtained results to estimate the robustness in practical vision applications.
Even for lower choices of the ε -parameter, AutoAttack still appears to modify target images beyond reasonable class boundaries.
Additionally, the resolution of the benchmark images should not be neglected.
In terms of resolution as well as in the number of classes and training images, CIFAR10 is a conveniently sized dataset for the very expensive state of the art adversarial training approaches.
However, our experiments suggest that these results might not generalize to more complex problems.
In light of our results, we argue that too strong adversarial benchmarks like the current setting of RobustBench might hamper the development of otherwise practically relevant methods towards more model robustness.