Sensitivity of Deep Convolutional Networks to Gabor Noise
Abstract
Deep Convolutional Networks (DCNs) have been shown to be sensitive to Universal Adversarial Perturbations (UAPs): input-agnostic perturbations that fool a model on large portions of a dataset.
These UAPs exhibit interesting visual patterns, but this phenomena is, as yet, poorly understood.
Our work shows that visually similar procedural noise patterns also act as UAPs.
In particular, we demonstrate that different DCN architectures are sensitive to Gabor noise patterns.
This behaviour, its causes, and implications deserve further in-depth study.
1. Introduction
Deep Convolutional Networks (DCNs) have enabled deep learning to become one the primary tools for computer vision tasks.
However, adversarial examples–slightly altered inputs that change the model’s output–have raised concerns on their reliability and security.
Adversarial perturbations can be deﬁned as the noise patterns added to natural inputs to generate adversarial examples.
Some of these perturbations are universal , i e. the same pattern can be used to fool the classiﬁer on a large fraction of the tested dataset (MoosaviDezfooli et al., 2017; Khrulkov & Oseledets, 2018).
As shown in Fig.
1, it is interesting to observe that such Universal Adversarial Perturbations (UAPs) for DCNs contain structure in their noise patterns.
Results from (Co et al., 2018) together with our results here suggest that DCNs are sensitive to procedural noise perturbations, and more speciﬁcally here to Gabor noise.
Existing UAPs have some visual similarities with Gabor noise as in Figure 2.
Convolutional layers induce a prior on DCNs to learn local spatial information (Goodfellow et al., 2016), and DCNs trained on natural image datasets, such as ImageNet, learn convolution ﬁlters that are similar Imperial College London, United Kingdom. Contact: Kenneth T. Co < k.co@imperial.ac.uk > , Luis Mu ˜ noz-Gonz ´ alez < l.munoz@imperial.ac.uk > , Emil C. Lupu < e.c.lupu@imperial.ac.uk > .
Accepted to the Workshop on Identifying and Understanding Deep Learning Phenomena within the 36 th International Conference on Machine Learning , Long Beach, California. Copyright 2019 by the authors. in appearance to Gabor kernels and colour blobs (Yosinski et al., 2014; Olah et al., 2017).
Gabor noise is a convolution between a Gabor kernel 2 and a sparse white noise.
Thus, we hypothesize that DCNs are sensitive to Gabor noise, as it exploits speciﬁc features learned by the convolutional ﬁlters.
In this paper we demonstrate the sensitivity of 3 different
DCN architectures ( Inception v3 , ResNet-50 , and VGG-19 ), to Gabor noise on the ImageNet image classiﬁcation task.
We empirically observed that even random Gabor noise patterns can be effective to generate UAPs.
Understanding this behaviour is important, as the generation and injection of Gabor noise is computationally inexpensive and, therefore, can become a threat to the security and reliability of DCNs.
2. Background
Compared to standard adversarial examples, UAPs reveal more general features that the DCN is sensitive to.
In contrast, adversarial perturbations generated for speciﬁc inputs, though less detectable in many cases, can “overﬁt” and evade only on inputs they were generated for (Zhou et al., 2018).
Previous approaches to generate UAPs use knowledge of the model’s learned parameters.
Moosavi-Dezfooli et al.
(2017) use the DeepFool algorithm (Moosavi-Dezfooli et al., 2016) iteratively over a set of images to construct a UAP.
A different approach is proposed in (Mopuri et al., 2018), where UAPs are computed using Generative Adversarial Nets (GANs).
Khrulkov & Oseledets (2018) proposed the singular vector method to generate UAPs targeting speciﬁc layers of DCNs, learning a perturbation s that maximises the L p -norm of the differences in the activations for that speciﬁc layer, f i :
[Equation]
where the L q -norm of s is constrained to ε .
This can approximated using the Jacobian for that layer:
[Equation]
The solution s that maximizes this is the ( p, q )
-singular vector can be computed with the power method (Boyd, 1974).
Then, s is effective to generate UAPs targeting a speciﬁc layer in the DCN.
The solutions obtained with this method for the ﬁrst layers of DCNs (see Fig. 1) resemble the Gabor noise patterns shown in Fig.
However none of these works highlight the interesting visual patterns that manifest from these UAPs.
In contrast, we show that procedural noise can generate UAPs targeting DCNs in a systematic and efﬁcient way.
3. Gabor Noise
Gabor noise is the convolution of a sparse white noise and a Gabor kernel, making it a type of Sparse Convolution
Noise (Lagae et al., 2009; 2010).
The Gabor kernel g with parameters { κ, σ, λ, ω } is the product of a circular Gaussian and a harmonic function
[Equation]
] where κ and σ are the magnitude and width of the Gaussian, and λ and ω are the frequency and orientation of the Harmonic (Lagae et al., 2010).
The value of the Gabor noise at point ( x, y ) is given by
[Equation]
where ( x i , y i ) are the coordinates of sparse random points and w i are random weights.
Gabor noise is an expressive noise function and has exponentially many parameterizations to explore.
To simplify the analysis, we choose anisotropic Gabor noise, where the Gabor kernel parameters and weights are the same for each i .
This results in noise patterns that have uniform orientation and thickness.
We also normalize the variance spectrum of the Gabor noise using the algorithm in (Neyret & Heitz, 2016) to achieve min-max oscillations within the pattern.
4. Experiments
For our experiments we use the validation set from the
ILSVRC2012 ImageNet image classiﬁcation task (Russakovsky et al., 2015) with 1,000 distinct categories.
We use 3 pre-trained ImageNet DCN architectures from keras.applications : Inception v3 (Szegedy et al., 2016), ResNet-50 (He et al., 2016), and VGG-19 (Simonyan & Zisserman, 2014).
Inception v3 take input images with dimensions 299 × 299 × 3 while the other two networks take images with dimensions 224 × 224 × 3 .
The kernel size κ = 23 is ﬁxed so that the Gabor kernels will ﬁll the entire image regardless of the distribution of points.
The number of points i distributed will be proportional to the image dimensions, which is independent of the Gabor kernel parameters.
The resulting Gabor noise parameters we control are Θ = { σ, ω, λ } .
We test the sensitivity of the models with 1,000 random Gabor noise perturbations generated from uniformly drawn parameters Θ with σ, λ ∈ [1 . 5 , 9] and ω ∈ [0 , π ] .
We evaluate our Gabor noise on 5,000 random images from the validation set with an (cid:96) ∞ norm constraint of ε =on the noise.
The choice of 12256 ≈ 0 . 047 is consistent with other attacks on ImageNet-scale models with less than 5% perturbation magnitude.
To provide a baseline, we also measure the sensitivity of the models to 1,000 uniform random noise perturbations from {− ε, ε } D × D × 3 where D is the image’s side length.
This is useful for showing that the sensitivity to Gabor noise is not trivial.
4.1. Metrics
Given model output f , input x ∈ X , perturbation s , and small ε > 0 , we deﬁne the universal sensitivity of a model on perturbation s over X as
[Equation]
The norm constraint on s ensures that the perturbation is small.
For this paper, we choose ∞ -norm as it is straightforward to impose for Gabor noise perturbations and is often used in the adversarial machine learning literature.
For classiﬁcation tasks, it is also useful to consider the universal evasion rate of a perturbation s over X
[Equation]
This corresponds to the deﬁnition that an adversarial perturbation is a small change that alters the predicted output label.
Note that we are not interested in the ground truth labels for x or x + s .
We focus instead on how small changes to the input result in large changes to the model’s original predictions .
It is worth using both the universal sensitivity and the universal evasion metrics, as the former gives a continuous measure of the sensitivity, while the latter tells us on how much of the dataset that perturbation changes the decision of the model.
4.2. Sensitivity to Gabor Noise
Our results show that the order from least to most sensitive models are Inception v3, ResNet-50, and then VGG-19.
This is not surprising as the validation accuracies of these models also appear in the same order.
Overall, our experiments show that the three models are signiﬁcantly more sensitive to the Gabor noise than random noise.
The universal sensitivity and evasion rates of random noise have very small variance and their values are clustered around their medians.
Table 1 shows how close the quartiles of random noise’s are for VGG-19.
Inception v3 is also insensitive to random noise, but has a moderate sensitivity to Gabor noise.
ResNet-50 appears to be more sensitive to the random noise than VGG-19, but
VGG-19 is more sensitive to Gabor noise than ResNet-50.
This implies that when comparing models higher sensitivity to one type of perturbation does not imply the same relationship for another type of perturbation.
The results in Fig.
3 suggest that across the three models a random Gabor noise is likely to affect the model outputs on a third or more of the input dataset.
From the histograms, the Gabor noise perturbations appear to centre around relatively high modes for both metrics.
As an example, the ﬁrst quartile of Gabor noise, as seen in Table 1, has 49.3% universal evasion, i.e. about 75% of the Gabor noise perturbations change VGG-19’s decision on about half or more of the input dataset.
For the remainder of this analysis we focus on VGG-19 as it is the most sensitive model.
Similar ﬁgures and statistics for the other two models are in the appendix.
[Table]
“Best” Parameters.
Taking the top 10 perturbations that
VGG-19 is most sensitive to, we see that the other two models are also very sensitive to these noise patterns.
The ranges of the universal evasion rate for these are 69.7% to 71.4% for VGG-19, 50.7% to 53.4% for ResNet-50, and 37.9% to 39.4% for Inception v3.
These values are all above the 3rd quartile for each of these models, showing its generalizability to the other models.
In Fig.
5 we see a strong correlation ( ≥ 0.74) between the universal sensitivity and evasion rates across models.
This further suggests that strong perturbations transfer across these models.
We also see a weak correlation between λ and the sensitivity and evasion rates for Inception v3, though there appears to be none between λ and the sensitivity values for ResNet50.
The universal evasion rate of the perturbations appears to be insensitive to its Gaussian width σ and orientation ω .
However, the sensitivity for small λ < 0 .
3 appears to fall below the average, suggesting that below a certain value the Gabor noise does not affect the model’s decision.
Interestingly, λ corresponds to the width or thickness of the bands in the image.
Examples of Gabor noise perturbations can be seen in the appendix.
Sensitivity of Inputs.
The model’s sensitivity could vary across the input dataset, meaning that the model’s predictions is stable on some inputs while more susceptible to small perturbations on others.
To measure this, we look at the sensitivity of single inputs over all perturbations.
Given a set of perturbations s ∈ S , we deﬁne the average sensitivity of a model on input x over S as
[Equation]
and the average evasion rate on x over S as
[Equation]
The bimodal distribution of the average evasion rate in Fig.shows that for each model there are two large subsets of the data: One that is very sensitive and another that is very insensitive.
The remaining data points are somewhat uniformly spread in the middle.
Note that for Inception v3, there is a much larger fraction of data points whose prediction is not affected by Gabor perturbations.
The distribution for the average sensitivity appears to have similar shape, but with more inputs in the 0-20% range for Inception v3.
The dataset is far less sensitive against random noise with upwards of 60% of the dataset being insensitive to that noise across all models.
5. Conclusion
The results show that the tested DCN models are sensitive to Gabor noise for a large fraction of the inputs, even when the parameters of the Gabor noise are chosen at random.
This hints that it may be representative of patterns learned at the earlier layers as Gabor noise appears visually similar to some UAPs targeting earlier layers in DCNs (Khrulkov & Oseledets, 2018).
This phenomenon has important implications on the security and reliability of DCNs, as it can allow attackers to craft inexpensive black-box attacks.
On the defender’s side, Gabor noise patterns can also be used to efﬁciently generate data for adversarial training to improve DCNs robustness.
However, both the sensitivity exploited and the potential to mitigate it require a more in-depth understanding of the phenomena at play.
In future work, it may be worth analyzing the sensitivity of hidden layer activations across different families of procedural noise patterns and to investigate techniques to reduce the sensitivity of DCNs to perturbations. Acknowledgements
Kenneth Co is partially supported by the Data Spartan research grant DSRD201801. Example code is available at https://github.com/kenny-co/procedural-advml
A. Sensitivity to Gabor Noise
As seen in Figure [Table] sensitivity metric values for random noise fall in a narrow range and are signiﬁcantly smaller than the metric values of the Gabor noise.
This is further shown when comparing the quartiles of the universal evasion and sensitivity in Tables 2 and 3.
Figures 9, 10, 11, 12, and 13 show some adversarial examples with the top perturbations.
[Table] [Table]
B. Sensitivity of Inputs
Large part of the input dataset is insensitive to random noise as shown in Tables 4, 5, 6 and Figure 7.
With about 60% of the dataset on having near 0% average evasion over the random noise perturbations for all three models.
[Table]
[Table] [Table]