Improving Visual Quality of Unrestricted Adversarial Examples with Wavelet-VAE
Abstract
Traditional adversarial examples are typically generated by adding perturbation noise to the input image within a small matrix norm.
In practice, unrestricted adversarial attack has raised great concern and presented a new threat to the AI safety.
In this paper, we propose a wavelet-VAE structure to reconstruct an input image and generate adversarial examples by modifying the latent code.
Different from perturbation-based attack, the modiﬁcations of the proposed method are not limited but imperceptible to human eyes.
Experiments show that our method can generate high quality adversarial examples on ImageNet dataset.
1. Introduction
Despite the great success of deep neural networks, they have been shown to be vulnerable to adversarial examples (Biggio et al., 2013; Szegedy et al., 2013).
By adding small perturbations, the well-designed adversarial examples are indistinguishable from the original ones, while the prediction labels of deep models can be confused.
As the application of DNNs penetrates into various ﬁelds, The existence of adversarial examples has raised great concern about safety and robustness of DNNs.
For long, perturbation-based adversarial examples have been the focus of attention.
Various adversarial attack methods have been proposed, including Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014), Projected Gradient Descent (PGD) (Madry et al., 2017), etc.
However, in the actual scene, more threats to the DNNs come from the unrestricted adversarial examples.
To be speciﬁc, the attacker makes large and visible modiﬁcations to the original images, which causes the model misclassiﬁcation, while preserving [Table] the normal observation from human perspective.
The unrestricted adversarial examples put a new threat to the AI safety.
Therefore, it is necessary to explore the scene and novel methods of unrestricted adversarial attacks.
Previous works have already given a formal deﬁnition of unrestricted adversarial attacks, Song (Song et al., 2018)
ﬁrst introduced the new threat and generates some adversarial examples from AC-GAN structure.
Stutz (Stutz et al., 2019) divided the adversarial examples into two groups according to the deﬁnition of data manifold.
However, limited by the model generation capability and coupling phenomenon of the latent code, most of the previous works are based on low resolution images.
The image quality of the generated adversarial examples has not been taken into well consideration.
To address the aforementioned image quality problem, we introduce wavelet transform and Variational Autoencoder (VAE) (Kingma & Welling, 2013) structure to the reconstruct process.
The intuition behind wavelet transform is that human eyes are not sensitive to high frequency signal changes.
The function of wavelet transform is to decouple the input images into different frequency bands.
Besides, to make the manipulations work in a more continuous way, we introduce the VAE structure.
In our work, we propose a novel adversarial attack algorithm based on wavelet-VAE network to encode and reconstruct the original images.
By manipulating the latent space codes, we can generate adversarial examples which are imperceptible to human eyes.
To be speciﬁc, we decompose the original images into different frequency bands, and a VQVAE (Razavi et al., 2019) network is trained to approximate the wavelet coefﬁcients of different scales.
Conditioned on the encoded latent space, we can formulate the attack problem as an optimization problem of ﬁnding a constrained latent code to maximize the adversarial loss.
Experiments on the ImageNet validation dataset (Deng et al., 2009)
show that, when compared with classic perturbationbased attack methods, our method can achieve higher attack success rate with better image quality, which is measure by two metrics, FID and LPIPS.
In summary, we make the technical contributions as:
[Figure]
• We project the real data distribution to a latent space with a wavelet-VAE network, which can decouple the human imperceptible high frequency bands by combining the wavelet transform with the Variational Autoencoder structure.
• We propose an unrestricted adversarial attack method based on the wavelet-VAE network, which is modeled as a optimization problem to the encoded latent code.
2. Related works
Song (Song et al., 2018)
ﬁrst proposed unrestricted adversarial examples
and they generate new adversarial examples with AC-GAN.
Kakizaki (Kakizaki & Yoshida, 2019) proposed a method to generate unrestricted adversarial examples against face recognition systems.
Shamsabadi (Shamsabadi et al., 2020) proposed a blackbox unrestricted adversarial attack, which modiﬁes the color of semantic regions.
Stutz (Stutz et al., 2019) provided a new way to understand the function of unrestricted adversarial attack from the manifold perspective, and the adversarial examples are generated from AC-GAN.
3. Method
In this section, we will describe in detail about the adversarial attack algorithm based on wavelet-VAE network.
3.1. Problem formulation
We start by introducing the deﬁnition of unrestricted adversarial attacks.
Let I denotes the set of all digital images taken into consideration.
The ground truth prediction can be formulated as a mapping function from the input set to the label set, i.e., g : X ⊆
I → { 1 , 2 , · · · , K } .
In addition, a well trained classiﬁer,denoted as f : X → { 1 , 2 , · · · , K } , will approximate but not equal to the ground truth function g .
With these notations, the unrestricted adversarial examples can be deﬁned as:
Deﬁnition 1 (Unrestricted Adversarial Examples)
Unrestricted adversarial examples to a target classiﬁer f can be deﬁned as any element in A
[Equation]
The unrestricted adversarial attacks set no constraint to the modiﬁcation range.
However, the perception distance between the original images and the modiﬁed ones should not be too large.
One possible way to achieve this goal is projecting the original high-dimensional pixel level space to a lower-dimensional manifold with VAE.
Besides, another advantage of VAE is introducing noise to the latent code, which means small perturbations to the latent code will not greatly change the semantic meaning of the decoded images.
However, the VAE structure can only maintain the semantic meaning, the perceptual distance may vary greatly.
Based on the fact that human eyes are not sensitive to signal changes in high frequency bands, we introduce wavelet transform to decouple the high frequency components and the low frequency ones.
With the combination of wavelet transform and VAE structure, the original adversarial attack problem can be formulated as an optimization problem to the latent code of wavelet coefﬁcients
[Equation]
in which, W − 1 is the inverse wavelet transform, dec ( · ) is the decoder of the VAE structure, the latent code is obtained from z = enc ( W ( x )) , ζ is the perturbation added to the latent code, f is the target classiﬁer, L is the loss function.
The adversarial examples with wavelet-VAE can be recognized as a projection from the original pixel-level manifold to the latent space of wavelet coefﬁcients.
The adversarial examples can be generated from the optimized latent code
[Equation]
in which, ζ ∗ is obtained from the optimal of Eq.
3.2. Wavelet-VAE network
In this section, we will provide the detailed description about the wavelet-VAE network, which is used as the reconstruction model of the input image.
The backbone network comes from VQ-VAE, which is a hierarchical learning architecture.
The structure of the network is shown in Fig.
We choose wavelet packet transform (WPT) to be the frequency analysis tool, which decomposes an input 2D image into four coefﬁcients, LL (low frequency component), (HL,
LH, HH) (high frequency components) , during the each decomposition level.
The wavelet coefﬁcients obtained from WPT are sent into the VQ-VAE network.
In practice, to get better image quality, we select to encode and reconstruct the high frequency components, while the low frequency components are directly sent to the inverse wavelet transform module.
The overall loss function is speciﬁed in Eq.
(3) as
[Equation]
in which, sg ( · ) refers to a stop-gradient operation that blocks gradients from ﬂowing into its argument, β is a hyperparameter which controls the reluctance to change the code corresponding to the encoder output, e is the quantized code from enc ( W ( x )) .
The ﬁrst term in the loss function is the reconstruction loss of the selected high frequency coefﬁcients decomposed by WPT.
The second and third term are borrowed from the original VQ-VAE network, which use
VQ to learn the embedding space.
3.3. Attack algorithm
The proposed method follows a two-stage approach: ﬁrst, we train the wavelet-VAE network to encode the wavelet coefﬁcients into a latent space, and then we add noise to the latent code to obtain an adversarial example according to the adversarial loss of the target classiﬁer.
At the ﬁrst stage, we utilize the proposed wavelet-VAE to learn the latent code distribution of the input image.
At the second stage, we ﬁrst ﬁx the wavelet-VAE parameters and encode the target image to obtain the latent code z .
Then the latent code is optimized using a gradient-based approach.
do 5: + loss 7: end 9: end
Although the VAE structure can keep the latent code staying on the manifold, we still need to set a constraint to the l ∞ norm of the latent code.
The reason of doing so is that the latent code space is not strictly compact with limited dataset and VAE model.
As the perturbation step gets larger, the probability of the latent code leaving the real image manifold will get higher.
4. Experiments 4.1. Experimental settings
Dataset and Models.
The tested dataset are crafted onrandomly selected ImageNet validation images.
As for target models, we choose 5 state-of-the-art DNNs: VGG19 (Simonyan & Zisserman, 2014), ResNet-152 (RN152)
(He et al., 2016), DenseNet-201 (Huang et al., 2017), Inception
V3 (IncV3) (Szegedy et al., 2016) and Inception-ResNet V (IncResV2) (Szegedy et al., 2017).
Evaluation Metrics.
In our experiments, we use Attack
Success Rate (ASR) as the evaluation metrics for attack ability, which can be expressed as
[Equation]
The image quality is measure from two metrics fr´echet inception distance (FID) (Heusel et al., 2017) and perceptual distance (LPIPS)
(Zhang et al., 2018).
We normalize these two metrics as
[Equation]
in which, f lhw and f l 0 hw are the l th layer feature output of
VGG with the input x and x adv , d is a distance metric.
Comparison Methods.
We compared our method with the classic perturbation-based attack method, which are
[Figure]
[Table] [Table]
FGSM (Goodfellow et al., 2014), PGD (Madry et al., 2017),
MIM (Dong et al., 2017) and DIM (Xie et al., 2019), for both the attack success rate and the quality of the adversarial examples.
Hyper-parameters.
We set the maximum perturbation of all the perturbation-based attack method to be 8 with pixel value ∈ [0 , 255] .
For the iterative methods, the number of iteration steps is to be 100.
The decay factor of momentum is set to be 1 . 0 for methods with momentum.
And the transformation probability of DIM is set to be 0 .
For our method, we set the modiﬁcation constrain of the latent vector to be η = 0 . 3 .
4.2. Experimental Results
We adopt a white-box attack with different attack methods on the selected classiﬁcation models.
And we compare the ASR and image quality of the adversarial generated by these attack methods separately.
Table 1 shows the results of the image quality (FID and LPIPS score) and Tableshows the results of the ASR.
Results from both tables show that our method can obtain higher attack success rate and higher image quality for most of the models.
From
Fig.2, it is shown that the adversarial examples generated by traditional perturbation-based methods have perceptible [Table] noise-like patterns, while the ones generated by our method are more natural from human perspective.
5. Conclusion
We propose an unrestricted adversarial attack algorithm based on wavelet-VAE network.
The VAE structure can make the latent code space more compact and wavelet transform can decouple the input image into different frequency bands, in which the higher frequency bands can be used to modify the image in an imperceptible way to human eyes.
By adding perturbations to the latent code of high frequency wavelet coefﬁcients, we can obtain high quality adversarial examples.
Acknowledgements
We thank the anonymous reviewers for their valuable comments.
This material is based upon CVPR-2021 AIC Phase
VI Track2: Unrestricted Adversarial Attacks on ImageNet.
We have won the second place in the competition.