Detecting AutoAttack Perturbations in the Frequency Domain
Abstract
Recently, adversarial attacks on image classiﬁcation networks by the AutoAttack (Croce & Hein, 2020b) framework have drawn a lot of attention.
While AutoAttack has shown a very high attack success rate, most defense approaches are focusing on network hardening and robustness enhancements, like adversarial training.
This way, the currently best-reported method can withstand ∼ 66% of adversarial examples on CIFAR10.
In this paper, we investigate the spatial and frequency domain properties of AutoAttack and propose an alternative defense.
Instead of hardening a network, we detect adversarial attacks during inference, rejecting manipulated inputs.
Based on a rather simple and fast analysis in the frequency domain, we introduce two different detection algorithms.
First, a black box detector which only operates on the input images and achieves a detection accuracy of 100% on the AutoAttack
CIFAR10 benchmark and 99 .
3% on ImageNet, for (cid:15) = 8 / 255 in both cases.
Second, a whitebox detector using an analysis of CNN featuremaps, leading to a detection rate of also 100% and 98 .
7% on the same benchmarks.
1. Introduction
The vulnerability of neural networks towards adversarial attacks is one of the key obstacles which are currently limiting the applicability of deep learning models for a wide range of practical use cases.
While the latest attack methods, like AutoAttack (Croce & Hein, 2020a), are achieving very high success rates perturbing input data on Convolutional
Neural Networks (CNN) based image classiﬁers, the avail- Institute for Machine Learning and Analytics - IMLA,
Offenburg University, Germany. Correspondence to: Peter Lorenz < peter.lorenz@itwm.fhg.de > .
Accepted by the ICML 2021 workshop on A Blessing in Disguise: The Prospects and Perils of Adversarial Machine Learning. Copyright 2021 by the author(s).
[Table]
ASR= Attack Success Rate , ASRD=
Attack Success Rate under Detection .
White-Box results on ImageNet are obtained by a Logistic Regression classiﬁer, Random Forests were used in all other cases.
Table 5 in the appendix is showing the full results for both classiﬁers.
F1 and the False Negative
Rate (FNR) are used to report the detection performance.
See section 3 for details of the experimental setup.
[Table]
ImageNet on WideResNet
[Table]
[Table]
[Table] able defense methods appear to be “always one step behind”. In general, adversarial defense strategies can be grouped into roughly two different approaches: First, the hardening of networks, which is mostly done via adversarial training, and second, the detection of adversarial samples during inference.
In this work, we propose a simple feature-driven approach to detect adversarial examples, where features are extracted in the frequency domain, based on prior work by (Harder et al., 2021), which shows almost perfect detection results on state-of-the-art (SOTA) adversarial benchmarks.
1.1. Related Work
The AutoAttack Benchmark.
In 2020, (Croce et al., 2020) launched a benchmark website 1 with the goal to provide a standardized benchmark for adversarial robustness.
Until then, single related libraries such as FoolBox (Rauber et al.,
[Figure]
2018), Cleverhans (Papernot et al., 2016) and AdverTorch (Ding et al., 2019) were already available but did not include all SOTA methods in one evaluation.
The currently dominating adversarial attack method is AutoAttack (Croce & Hein, 2020a) which is an ensemble of 4 attacks: two variations of the PGD (Madry et al., 2018) attack with cross-entropy loss (APGD-CE) and difference of logits ratio loss (APGD-t), the targeted version of the FAB attack (Croce & Hein, 2020c), and the black-box Squares attack (Andriushchenko et al., 2020).
The AutoAttack benchmark provides several modes.
The “standard” mode executes the 4 attack methods consecutively.
Only if one attack fails, the failed samples are handed over to the next attack method.
The “individual” mode provides results for each attack on all input samples.
Adversarial Training.
Adversarial Training (AT) can be backtracked to (Goodfellow et al., 2015), in which models were hardened by generating adversarial examples and adding them into training data.
An adversarial example is a subtly modiﬁed image causing a machine learning model to misclassify it.
The achieved robustness by AT depends on the strength of the adversarial examples used.
E.g., training on Goodfellow’s FGSM, which is fast and non-iterative, only provides robustness against non-iterative attacks, but e.g. not against PGD (Madry et al., 2018) attacks.
Consequently, (Tram`er et al., 2020) propose training on multi-step PGD adversaries, achieving SOTA robustness levels against (cid:96) ∞ attacks on MNIST and CIFAR10 datasets.
Unfortunately, the high computational complexity of adversarial training makes it impractical for large-scale problems such as ImageNet.
Adversarial Detection.
Many recent works have focused on adversarial detection, trying to distinguish adversarial from natural images.
The authors in (Hendrycks & Gimpel, 2017) showed that adversarial examples have higher weights for larger principal components of the images’ decomposition and use this ﬁnding to train a detector.
Both (Li & Li, 2017) and (Bhagoji et al., 2017) leverage Principal
Component Analysis (PCA) as well.
Based on the responses of the neural networks’ ﬁnal layer (Feinman et al., 2017) deﬁne two metrics, the kernel density estimation and the
Bayesian neural network uncertainty to identify adversarial perturbations.
(Liu et al., 2019) proposed a method to detect adversarial examples by leveraging steganalysis and estimating the probability of modiﬁcations caused by adversarial attacks.
(Grosse et al., 2017) used the statistical test of maximum mean discrepancy to detect adversarial samples.
Using the correlation between images, based on inﬂuence functions and the k-nearest neighbors in the embedding space of the Deep Neural Network (DNN), (Cohen et al., 2020)
[Table]
proposed an adversarial detector.
Besides the statistical analysis of the input images, adding a second neural network to decide whether an image is an adversarial example is another possibility.
(Metzen et al., 2017) proposed a model that is trained on outputs of multiple intermediate layers
[Table]
Two strong and popular detectors are the Local Intrinsic Dimensionality (LID)
(Ma et al., 2018) and the Mahalanobis
Distance (M-D) (Lee et al., 2018) detectors.
Ma et al.
used the LID as a characteristic of adversarial subspaces and identiﬁed attacks using this measure.
Lee et al. computed the empirical mean and covariance for each training sample and then calculated the M-D distance between a test sample and its nearest class-conditional Gaussians.
Fourier Analysis of Adversarial Attacks.
(Tsuzuku &
Sato, 2019) showed that CNN are sensitive in the direction of Fourier basis functions, and proposed a Fourier-based attack method.
Investigating trade-offs between Gaussian data augmentation and adversarial training (Yin et al., 2019) take a Fourier perspective and observe that adversarial examples are not only a high-frequency phenomenon.
In (Ma et al., 2020) it is assumed that internal responses of DNN follow the generalized Gaussian distribution, both for benign and adversarial examples, but with different parameters.
They extract the feature maps at each layer in the classiﬁcation network and calculate the Benford-Fourier coefﬁcients for all of these representations.
This approach is similar to our white-box detector, but as our experiments show, it is more than sufﬁcient to use our simpliﬁed features built on a standard 2D Discrete Fourier Transformation (DFT).
2. Methods: DFT based Detection
Our proposed detection method is based on the frequencydomain features originally introduced in (Harder et al., 2021), which we revise in the next subsections.
In contrast to them, we explicitly propose two types of detectors.
First, a White-Box detector which has access to the feature maps of the target network, allowing it to observe the network response to input images, and second, a more general Black-Box detector which has no knowledge about the target network.
We found that the Fourier power spectrum provides sufﬁcient information to detect perturbations in both cases.
Hence, we neglect the phase-based features which are also suggested in (Harder et al., 2021).
Fourier Analysis.
The Fourier transformation decomposes a function into its constituent frequencies.
A signal sampled at equidistant points is thereby known as discrete Fourier transform.
The discrete Fourier transform of a signal with length N can be computed efﬁciently with the Fast Fourier
Transformation (FFT) in O ( N log N ) (Cooley & Tukey, 1965).
For a discrete 2D signal, like color image channels or single CNN feature maps – X ∈ [0 , 1] N × N – the 2D discrete Fourier transform is given as F ( [Equation] [Equation]
[Equation]
N (cid:88) n
[Equation]
[Equation]
for l, k = 0 , . . .
N − 1 , with complex valued Fourier coefﬁcients F ( X )( l, k ) .
In the following, we will only utilize the magnitudes of Fourier coefﬁcients
[Equation]
and show that this information is sufﬁcient to detect adversarial attacks with high accuracy.
2.1. Black-Box Detection: Fourier Features of Input Images
Figure 1 gives a brief visualization of the analysis of the changes in successfully perturbed images from AutoAttack :
While different attacks show distinct but randomly located change patterns in the spatial domain (which makes them hard to detect), adversarial samples show strong, well localized signals in the frequency domain.
Hence, we extract and concatenate the 2D power spectrum of each color channel (see eq.
(2) and the right column of ﬁg. 1) as feature representations of input images and use simple classiﬁers like Random Forests and Logistic Regression to learn to detect perturbed input images.
2.2. White-Box Detection: Fourier Features of Feature-Maps
In the white-box case, we apply the same method as in the black-box approach, but extend the inputs to the feature map responses of the target network to test samples.
Since this extension will drastically increase the feature space for larger target networks, we select only a subset of the available feature maps.
Note that the optimal selection of feature maps heavily depends on the topology of the target network.
See table 4 for details on our selection for
CIFAR10.
2.3. Measuring Adversarial Detection
The AutoAttack benchmark (Croce & Hein, 2020a), like most of the literature regarding adversarial robustness, uses a ”Robust Accuracy” measure to compare different methods (see table 2 for details).
However, our approach does not ﬁt this evaluation scheme, since we are aiming to reject adversarial test samples instead of hardening the networks.
Therefore, we report two different indicators: The Adversarial Succes Rate (ASR) in eq.
(3) is calculated as
[Equation]
samples #
[Equation]
samples
[Equation]
the fraction of successfully perturbed test images and provides a baseline of AutoAttack ’s ability to fool unprotected target networks.
We measure the performance of our defense by the Adversarial Success Rate under Detection (ASRD) in eq.
Here, we compute the ratio of successful attacks under defense
ASRD =
# undetected perturbations
[Equation]
where FNR is the false negative rate of the applied detection algorithm.
ASR is only a scaling factor for the FNR.
The lower the ASRD rate, the more pertubated examples are defeated.
3. Experiments
CIFAR10.
We trained a WideResNet28-10 (Zagoruyko &
Komodakis, 2017) on the CIFAR10 training set to a testaccuracy of 94% and applied AutoAttack on the test set.
Then we extracted the spectral features and used a random subset of 1500 samples of this data for each attack method to train and evaluate our base classiﬁers (train:test split 80:20).
Table 1 shows results using AutoAttack in “standard” mode for various ε on (cid:96) ∞ -perturbations.
Table 3 shows the results for the “individual” mode for ε = 8 / 255 and (cid:96) ∞ - perturbations as well as a comparison to other detection methods.
Here we used 1000 samples from each dataset,
CIFAR10 and ImageNet, for our evaluation.
ImageNet.
For the benchmarks on ImageNet we used the pre-trained WideResNet51-2 (Zagoruyko & Komodakis, 2017) from the PyTorch library.
As test set, we apply the ofﬁcial validation set from ImageNet.
The accuracy of this pre-trained model is about 78% .
As for CIFAR10,
AutoAttack also shows strong adversarial performance on
ImageNet.
Table 1 shows very high ASR even for low ε values.
Note that ε < 0 .
5 would not represent realistic attack scenarios since saving the perturbed images would round the adversarial changes to the next of 256 available
[Table]
[Table] [Table]
bins in commonly used 8-bit per channel image encodings.
Table 3 shows the results for individual attacks.
4. Discussion and Conclusion
In this paper, we are able to show a ﬁrst proof of concept that simple frequency features can be used to detect current SOTA attacks with a very high accuracy on the standard CIFAR10 benchmark and on a the more complex ImageNet dataset.
Especially the black-box approach could provide a practical counter-measure for the defense of realworld applications.
However, there are still many open questions: I) How well will the detectors generalize to other datasets, network architectures, and new attacks?
II) Why is
AutoAttack so successful for very small ε in ImageNet?
III)
Can the detection be combined with Adversarial Training methods like the ones shown in table 2?
In light of these open questions, we expect our approach can build a solid basis for future research.
Appendix
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] [Table]