Detecting Adversaries, yet Faltering to Noise? Leveraging Conditional Variational AutoEncoders for Adversary Detection in the Presence of Noisy Images
Abstract
With the rapid advancement and increased use of deep learning models in image identiﬁcation, security becomes a major concern to their deployment in safety-critical systems.
Since the accuracy and robustness of deep learning models are primarily attributed from the purity of the training samples, therefore the deep learning architectures are often susceptible to adversarial attacks.
Adversarial attacks are often obtained by making subtle perturbations to normal images, which are mostly imperceptible to humans, but can seriously confuse the state-of-the-art machine learning models.
What is so special in the slightest intelligent pertur- bations or noise additions over normal images that it leads to catastrophic classiﬁcations by the deep neural networks?
Using statistical hypothesis testing, we ﬁnd that Conditional
Variational AutoEncoders (CVAE) are surprisingly good at detecting imperceptible image perturbations.
In this paper, we show how CVAEs can be effectively used to detect adversarial attacks on image classiﬁcation networks.
We demonstrate our results over MNIST, CIFAR-10 dataset and show how our method gives comparable performance to the state-of-the-art methods in detecting adversaries while not getting confused with noisy images, where most of the existing methods falter.
Introduction
The phenomenal success of deep learning models in image identiﬁcation and object detection has led to its wider adoption in diverse domains ranging from safety-critical systems, such as automotive and avionics (Rao and Frtunikj 2018) to healthcare like medical imaging, robot-assisted surgery, genomics etc.
(Esteva et al. 2019), to robotics and image forensics (Yang et al. 2020), etc.
The performance of these deep learning architectures are often dictated by the volume of correctly labelled data used during its training phases.
Recent works (Szegedy et al. 2013)
(Goodfellow,
Shlens, and Szegedy 2014) have shown that small and carefully chosen modiﬁcations (often in terms of noise) to the input data of a neural network classiﬁer can cause the model to give incorrect labels.
This weakness of neural networks allows the possibility of making adversarial attacks on the input image by creating perturbations which are imperceptible to humans but however are able to convince the neural network in getting completely wrong results that too with very high conﬁdence.
Due to this, adversarial attacks may pose a serious threat to deploying deep learning models in real-world safety-critical applications.
It is, therefore, imperative to devise efﬁcient methods to thwart such adversarial attacks.
Many recent works have presented effective ways in which adversarial attacks can be avoided.
Adversarial attacks can be classiﬁed into whitebox and blackbox attacks.
White-box attacks (Akhtar and Mian 2018) assume access to the neural network weights and architecture, which are used for classiﬁcation, and thereby speciﬁcally targeted to fool the neural network.
Hence, they are more accurate than blackbox attacks (Akhtar and Mian 2018) which do not assume access the model parameters.
Methods for detection of adversarial attacks can be broadly categorized as – (i) statistical methods, (ii) network based methods, and (iii) distribution based methods.
Statistical methods (Hendrycks and Gimpel 2016)
(Li and Li 2017) focus on exploiting certain characteristics of the input images or the ﬁnal logistic-unit layer of the classiﬁer network and try to identify adversaries through their statistical inference.
A certain drawback of such methods as pointed by (Carlini and Wagner 2017) is that the statistics derived may be dataset speciﬁc and same techniques are not generalized across other datasets and also fail against strong attacks like CW-attack.
Network based methods (Metzen et al.
(Gong, Wang, and Ku 2017) aim at speciﬁcally training a binary classiﬁcation neural network to identify the adversaries.
These methods are restricted since they do not generalize well across unknown attacks on which these networks are not trained, also they are sensitive to change with the amount of perturbation values such that a small increase in these values makes this attacks unsuccessful.
Also, potential whitebox attacks can be designed as shown by (Carlini and Wagner 2017) which fool both the detection network as well as the adversary classiﬁer networks.
Distribution based methods (Feinman et al. 2017)
(Gao et al. 2021)
(Song et al. 2017)
(Xu, Evans, and Qi 2017)
(Jha et al.
2018) aim at ﬁnding the probability distribution from the clean examples and try to ﬁnd the probability of the input example to quantify how much they fall within the same distribution.
However, some of the methods do not guarantee robust separation of randomly perturbed and adversarial perturbed images.
Hence there is a high chance that all these methods tend to get confused with random noises in the image, treating them as adversaries.
To overcome this drawback so that the learned models are robust with respect to both adversarial perturbations as well as sensitivity to random noises, we propose the use of
Conditional Variational AutoEncoder (CVAE) trained over a clean image set.
At the time of inference, we empirically establish that the input example falls within a low probability region of the clean examples of the predicted class from the target classiﬁer network.
It is important to note here that, this method uses both the input image as well as the predicted class to detect whether the input is an adversary as opposed to some distribution based methods which use only the distribution from the input images.
On the contrary, random perturbations activate the target classiﬁer network in such a way that the predicted output class matches with the actual class of the input image and hence it falls within the high probability region.
Thus, it is empirically shown that our method does not confuse random noise with adversarial noises.
Moreover, we show how our method is robust towards special attacks which have access to both the network weights of CVAE as well as the target classiﬁer networks where many network based methods falter.
Further, we show that to eventually fool our method, we may need larger perturbations which becomes visually perceptible to the human eye.
The experimental results shown over MNIST and CIFAR-10 datasets present the working of our proposal.
In particular, the primary contributions made by our work is as follows.
(a) We propose a framework based on CVAE to detect the possibility of adversarial attacks.
(b) We leverage distribution based methods to effectively differentiate between randomly perturbed and adversarially perturbed images.
(c) We devise techniques to robustly detect specially targeted BIM-attacks (Metzen
et al. 2017) using our proposed framework.
To the best of our knowledge, this is the ﬁrst work which leverages use of Variational AutoEncoder architecture for detecting adversaries as well as aptly differentiates noise from adversaries to effectively safeguard learned models against adversarial attacks.
Adversarial Attack Models and Methods
For a test example X , an attacking method tries to ﬁnd a perturbation, ∆ X such that | ∆ X | k ≤ (cid:15) atk where (cid:15) atk is the perturbation threshold and k is the appropriate order, generally selected as 2 or ∞ so that the newly formed perturbed image, X adv = X +∆ X .
Here, each pixel in the image is represented by the (cid:104) R , G , B (cid:105) tuple, where R , G , B ∈ [0 , 1] .
In this paper, we consider only white-box attacks, i.e. the attack methods which have access to the weights of the target classiﬁer model.
However, we believe that our method should work much better for black-box attacks as they need more perturbation to attack and hence should be more easily detected by our framework.
For generating the attacks, we use the library by (Li et al. 2020).
Random Perturbation (RANDOM)
Random perturbations are simply unbiased random values added to each pixel ranging in between − (cid:15) atk to (cid:15) atk .
Formally, the randomly perturbed image is given by,
[Equation]
where, U ( a, b ) denote a continuous uniform distribution in the range [ a, b ] .
Fast Gradient Sign Method (FGSM)
Earlier works by (Goodfellow, Shlens, and Szegedy 2014) introduced the generation of malicious biased perturbations at each pixel of the input image in the direction of the loss gradient ∆ X L ( X, y ) , where L ( X, y ) is the loss function with which the target classiﬁer model was trained.
Formally, the adversarial examples with with l ∞ norm for (cid:15) atk are computed by,
[Equation]
FGSM perturbations with l 2 norm on attack bound are calculated as,
[Equation]
Projected Gradient Descent (PGD)
Earlier works by (Kurakin, Goodfellow, and Bengio 2017) propose a simple variant of the FGSM method by applying it multiple times with a rather smaller step size than (cid:15) atk .
However, as we need the overall perturbation after all the iterations to be within (cid:15) atk -ball of X , we clip the modiﬁed
X at each step within the (cid:15) atk ball with l ∞ norm.
[Equation]
4b)
Given α , we take the no of iterations, n to be (cid:98) 2 (cid:15) atk α + 2 (cid:99) .
This attacking method has also been named as Basic
Iterative Method (BIM) in some works.
Carlini-Wagner (CW) Method
(Carlini and Wagner 2017) proposed a more sophisticated way of generating adversarial examples by solving an optimization objective as shown in Equation 5.
Value of c is chosen by an efﬁcient binary search.
We use the same parameters as set in (Li et al. 2020) to make the attack.
[Equation]
DeepFool method DeepFool
(Moosavi-Dezfooli, Fawzi, and Frossard 2016) is an even more sophisticated and efﬁcient way of generating adversaries.
It works by making the perturbation iteratively towards the decision boundary so as to achieve the adversary with minimum perturbation.
We use the default parameters set in (Li et al. 2020) to make the attack.
Proposed Framework Leveraging CVAE
In this section, we present how Conditional Variational AutoEncoders (CVAE), trained over a dataset of clean images, are capable of comprehending the inherent differentiable attributes between adversaries and noisy data and separate out both using their probability distribution map.
Conditional Variational AutoEncoders (CVAE)
Variational AutoEncoder is a type of a Generative Adversarial Network (GAN) having two components as Encoder and Decoder.
The input is ﬁrst passed through an encoder to get the latent vector for the image.
The latent vector is passed through the decoder to get the reconstructed input of the same size as the image.
The encoder and decoder layers are trained with the objectives to get the reconstructed image as close to the input image as possible thus forcing to preserve most of the features of the input image in the latent vector to learn a compact representation of the image.
The second objective is to get the distribution of the latent vectors for all the images close to the desired distribution.
Hence, after the variational autoencoder is fully trained, decoder layer can be used to generate examples from randomly sampled latent vectors from the desired distribution with which the encoder and decoder layers were trained.
Conditional VAE is a variation of VAE in which along with the input image, the class of the image is also passed with the input at the encoder layer and also with the latent vector before the decoder layer (refer to Figure 1).
This helps Conditional VAE to generate speciﬁc examples of a class.
The loss function for CVAE is deﬁned by Equation 6.
The ﬁrst term is the reconstruction loss which signiﬁes how closely can the input
X be reconstructed given the latent vector z and the output class from the target classiﬁer network as condition, c .
The second term of the loss function is the KL-divergence ( D KL ) between the desired distribution, P ( z | c ) and the current distribution ( Q ( z | X, c ) ) of z given input image X and the condition c .
L ( X, c ) = E (cid:2)
[Equation]
Training CVAE Models
For modeling log P ( X | z, c ) , we use the decoder neural network to output the reconstructed image, X rcn where we utilize the condition c which is the output class of the image to get the set of parameters, θ ( c ) for the neural network.
We calculate Binary Cross Entropy ( BCE ) loss of the reconstructed image, X rcn with the input image, X to model log P ( X | z, c ) .
Similarly, we model Q ( z | X, c ) with the encoder neural network which takes as input image
X and utilizes condition, c to select model parameters, θ ( c ) and outputs mean, µ and log of variance, log σ 2 as parameters assuming Gaussian distribution for the conditional distribution.
We set the target distribution P ( z | c ) as unit Gaussian distribution with mean 0 and variance 1 as
N (0 , 1) .
The resultant loss function would be as follows,
[Equation]
[Equation]
The model architecture weights, θ ( c ) are a function of the condition, c .
Hence, we learn separate weights for encoder and decoder layers of CVAE for all the classes.
It implies learning different encoder and decoder for each individual class.
The layers sizes are tabulated in Table I.
We train the Encoder and Decoder layers of CVAE on clean images with their ground truth labels and use the condition as the predicted class from the target classiﬁer network during inference.
[Table]
Determining Reconstruction Errors
Let X be the input image and y pred be the predicted class obtained from the target classiﬁer network.
X rcn,y pred is the reconstructed image obtained from the trained encoder and decoder networks with the condition y pred .
We deﬁne the reconstruction error or the reconstruction distance as in Equation 8.
The network architectures for encoder and decoder layers are given in Figure 1.
[Equation]
Two pertinent points to note here are:
• For clean test examples, the reconstruction error is bound to be less since the CVAE is trained on clean train images.
As the classiﬁer gives correct class for the clean examples, the reconstruction error with the correct class of the image as input is less.
• For the adversarial examples, as they fool the classiﬁer network, passing the malicious output class, y pred of the classiﬁer network to the CVAE with the slightly perturbed input image, the reconstructed image tries to be closer to the input with class y pred and hence, the reconstruction error is large.
As an example, let the clean image be a cat and its slightly perturbed image fools the classiﬁer network to believe it is a dog.
Hence, the input to the CVAE will be the slightly perturbed cat image with the class dog.
Now as the encoder and decoder layers are trained to output a dog image if the class inputted is dog, the reconstructed image will try to resemble a dog but since the input is a cat image, there will be large reconstruction error.
Hence, we use reconstruction error as a measure to determine if the input image is adversarial.
We ﬁrst train the Conditional Variational
AutoEncoder (CVAE) on clean images with the ground truth class as the condition.
Examples of reconstructions for clean and adversarial examples are given in Figureand Figure 3.
[Figure] [Figure]
Obtaining p -value
As already discussed, the reconstruction error is used as a basis for detection of adversaries.
We ﬁrst obtain the reconstruction distances for the train dataset of clean images which is expected to be similar to that of the train images.
On the other hand, for the adversarial examples, as the predicted class y is incorrect, the reconstruction is expected to be worse as it will be more similar to the image of class y as the decoder network is trained to generate such images.
Also for random images, as they do not mostly fool the classiﬁer network, the predicted class, y is expected to be correct, hence reconstruction distance is expected to be less.
Besides qualitative analysis, for the quantitative measure, we use the permutation test from (Efron and Tibshirani 1993).
We can provide an uncertainty value for each input about whether it comes from the training distribution.
Speciﬁcally, let the input
X (cid:48) and training images X 1 , X 2 , . . .
, X N .
We ﬁrst compute the reconstruction distances denoted by Recon ( X, y ) for all samples with the condition as the predicted class y =
Classifier ( X ) .
Then, using the rank of Recon ( X (cid:48) , y (cid:48) ) in { Recon ( X 1 , y 1 ) , Recon ( X 2 , y 2 ) , . . .
, Recon ( X N , y N ) } as our test statistic, we get,
[Equation]
Where I [ . ] is an indicator function which returns 1 if the condition inside brackets is true, and 0 if false.
By permutation principle, p -value for each sample will be,
[Equation]
Larger p -value implies that the sample is more probable to be a clean example.
Let t be the threshold on the obtained p -value for the sample, hence if p X,y < t , the sample X is classiﬁed as an adversary.
Algorithm 1 presents the overall resulting procedure combining all above mentioned stages.
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
Algorithm 1 ﬁrst trains the CVAE network with clean training samples (Line 2) and formulates the reconstruction distances (Line 3).
Then, for each of the test samples which may contain clean, randomly perturbed as well as adversarial examples, ﬁrst the output predicted class is obtained using a target classiﬁer network, followed by ﬁnding it’s reconstructed image from CVAE, and ﬁnally by obtaining it
’s p -value to be used for thresholding (Lines 58).
Images with p -value less than given threshold ( t ) are classiﬁed as adversaries (Lines 9-10).
Experimental Results
We experimented our proposed methodology over MNIST and CIFAR-10 datasets.
All the experiments are performed in Google Colab GPU having 0 .
82 GHz frequency, 12 GB
RAM and dual-core CPU having 2 .
3 GHz frequency, 12 GB
RAM.
An exploratory version of the code-base will be made public on github.
Datasets and Models
Two datasets are used for the experiments in this paper, namely MNIST (LeCun, Cortes, and Burges 2010) and
CIFAR-10 (Krizhevsky 2009).
MNIST dataset consists of hand-written images of numbers from 0 to 9 .
It consists of 60 , 000 training examples and 10 , 000 test examples where each image is a 28 × 28 gray-scale image associated with a label from 1 of the 10 classes.
CIFAR-10 is broadly used for comparison of image classiﬁcation tasks.
It also consists of 60 , 000 image of which 50 , 000 are used for training and the rest 10 , 000 are used for testing.
Each image is a 32 ×coloured image i.e. consisting of 3 channels associated with a label indicating 1 out of 10 classes.
We use state-of-the-art deep neural network image classiﬁer, ResNet18 (He et al. 2015) as the target network for the experiments.
We use the pre-trained model weights available from (Idelbayev ) for both MNIST as well as
CIFAR-10 datasets.
Performance over Grey-box attacks
If the attacker has the access only to the model parameters of the target classiﬁer model and no information about the detector method or it’s model parameters, then we call such attack setting as Grey-box.
This is the most common attack setting used in previous works against which we evaluate the most common attacks with standard epsilon setting as used in other works for both the datasets.
For MNIST, the value of (cid:15) is commonly used between 0.15-0.3 for
FGSM attack and 0.1 for iterative attacks (Samangouei,
Kabkab, and Chellappa 2018)
(Gong, Wang, and Ku 2017)
(Xu, Evans, and Qi 2017).
While for CIFAR10, the value of (cid:15) is most commonly chosen to be 8255 as in (Song et al. 2017)
(Xu, Evans, and Qi 2017) (Fidel, Bitton, and
Shabtai 2020).
For DeepFool (Moosavi-Dezfooli, Fawzi, and Frossard 2016) and Carlini Wagner (CW) (Carlini and
Wagner 2017) attacks, the (cid:15) bound is not present.
The standard parameters as used by default in (Li et al. 2020) have been used for these 2 attacks.
For L 2 attacks, the (cid:15) bound is chosen such that success of the attack is similar to their L ∞ counterparts as the values used are very different in previous works.
Reconstruction Error Distribution
The histogram distribution of reconstruction errors for MNIST and CIFAR10 datasets for different attacks are given in Figure 4.
For adversarial attacked examples, only examples which fool the network are included in the distribution for fair comparison.
It may be noted that, the reconstruction errors for adversarial examples is higher than normal examples as expected.
Also, reconstructions errors for randomly perturbed test samples are similar to those of normal examples but slightly larger as expected due to reconstruction error contributed from noise.
[Figure] [Figure]
From the reconstruction error values, the distribution histogram of p-values of test samples for MNIST and CIFAR-10 datasets are given in Figure 5.
It may be noted that, in case of adversaries, most samples have p -value close to 0 due to their high reconstruction error; whereas for the normal and randomly perturbed images, p -value is nearly uniformly distributed as expected.
ROC Characteristics:
Using the p -values, ROC curves can be plotted as shown in Figure 6.
As can be observed from ROC curves, clean and randomly perturbed attacks can be very well classiﬁed from all adversarial attacks.
The values of (cid:15) atk were used such that the attack is able to fool the target detector for at-least 45% samples.
The percentage of samples on which the attack was successful for each attack is shown in Table II.
Statistical Results and Discussions:
The statistics for clean, randomly perturbed and adversarial attacked images for MNIST and CIFAR datasets are given in Table II.
Error rate signiﬁes the ratio of the number of examples which were misclassiﬁed by the target network.
Last column (AUC) lists the area under the ROC curve.
The area for adversaries is expected to be close to 1 ; whereas for the normal and randomly perturbed images, it is expected to be around 0 .
It is worthy to note that, the obtained statistics are much comparable with the state-of-the-art results as tabulated in
[Figure]
[Table]
AUC : Area Under the ROC Curve.
Error Rate (%) :
Percentage of samples mis-classiﬁed or Successfully-attacked
Table V (Given in the Appendix ).
Interestingly, some of the methods (Song et al. 2017) explicitly report comparison results with randomly perturbed images and are ineffective in distinguishing adversaries from random noises, but most other methods do not report results with random noise added to the input image.
Since other methods use varied experimental setting, attack models, different datasets as well as (cid:15) atk values and network model, exact comparisons with other methods is not directly relevant primarily due to such varied experimental settings.
However, the results reported within the Table V (Given in the Appendix ) are mostly similar to our results while our method is able to statistically differentiate from random noisy images.
In addition to this, since our method does not use any adversarial examples for training, it is not prone to changes in value of (cid:15) or with change in attacks which network based methods face as they are explicitly trained with known values of (cid:15) and types of attacks.
Moreover, among
[Figure]
distribution and statistics based methods, to the best of our knowledge, utilization of the predicted class from target network has not been done before.
Most of these methods either use the input image itself (Jha et al. 2018)
(Song et al. 2017)
(Xu, Evans, and Qi 2017), or the ﬁnal logits layer (Feinman et al. 2017) (Hendrycks and Gimpel 2016), or some intermediate layer (Li and Li 2017)
(Fidel, Bitton, and Shabtai 2020) from target architecture for inference, while we use the input image and the predicted class from target network to do the same.
Performance over White-box attacks
In this case, we evaluate the attacks if the attacker has the information of both the defense method as well as the target classiﬁer network.
(Metzen et al. 2017) proposed a modiﬁed
PGD method which uses the gradient of the loss function of the detector network assuming that it is differentiable along with the loss function of the target classiﬁer network to generate the adversarial examples.
If the attacker also has access to the model weights of the detector CVAE network, an attack can be devised to fool both the detector as well as the classiﬁer network.
The modiﬁed PGD can be expressed as follows :-
[Equation]
Where y target is the target class and L det is the reconstruction distance from Equation 8.
It is worthy to note that our proposed detector CVAE is differentiable only for the targeted attack setting.
For the non-targeted attack, as the condition required for the CVAE is obtained from the target classiﬁer output which is discrete, the differentiation operation is not valid.
We set the target randomly as any class other than the true class for testing.
Effect of σ :
To observe the effect of changing value of σ , we keep the value of (cid:15) ﬁxed at 0.1.
As can be observed in
Figure 7, the increase in value of σ implies larger weight on fooling the detector i.e. getting less reconstruction distance.
Hence, as expected the attack becomes less successful with larger values of σ 8 and gets lesser AUC values 7, hence more effectively fooling the detector.
For CIFARdataset, the detection model does get fooled for higher cvalues but however the error rate is signiﬁcantly low for those values, implying that only a few samples get attacks on setting such value.
[Figure]
Effect of (cid:15) :
With changing values of (cid:15) , there is more space available for the attack to act, hence the attack becomes more successful as more no of images are attacked as observed in Figure 10.
At the same time, the trend for
AUC curves is shown in Figure 9.
The initial dip in the value is as expected as the detector tends to be fooled with larger (cid:15) bound.
From both these trends, it can be noted that
[Figure]
for robustly attacking both detector target classiﬁer for signiﬁcantly higher no of images require signiﬁcantly larger attack to be made for both the datasets.
[Figure]
Related Works
There has been an active research in the direction of adversaries and the ways to avoid them, primarily these methods are statistical as well as machine learning (neural network) based which produces systematic identiﬁcation and rectiﬁcation of images into desired target classes.
Statistical Methods:
Statistical methods focus on exploiting certain characteristics of the input images and try
[Figure]
to identify adversaries through their statistical inference.
Some early works include use of PCA, softmax distribution of ﬁnal layer logits (Hendrycks and Gimpel 2016), reconstruction from logits (Li and Li 2017) to identify adversaries.
Carlini and Wagner (Carlini and Wagner 2017) showed how these methods are not robust against strong attacks and most of the methods work on some speciﬁc datasets but do not generalize on others as the same statistical thresholds do not work.
Network based Methods
Network based methods aim at speciﬁcally training a neural network to identify the adversaries.
Binary classiﬁcation networks (Metzen et al.
(Gong, Wang, and Ku 2017) are trained to output a conﬁdence score on the presence of adversaries.
Some methods propose addition of a separate classiﬁcation node in the target network itself (Hosseini et al. 2017).
The training is done in the same way with the augmented dataset.
(Carrara et al. 2018) uses feature distant spaces of intermediate layer values in the target network to train an LSTM network for classifying adversaries.
Major challenges faced by these methods is that the classiﬁcation networks are differentiable, thus if the attacker has access to the weights of the model, a speciﬁcally targeted attack can be devised as suggested by Carlini and Wagner (Carlini and Wagner 2017) to fool both the target network as well as the adversary classiﬁer.
Moreover, these methods are highly sensitive to the perturbation threshold set for adversarial attack and fail to identify attacks beyond a preset threshold.
Distribution based Methods:
Distribution based methods aim at ﬁnding the probability distribution from the clean examples and try to ﬁnd the probability of the input example to fall within the same distribution.
Some of these methods include using Kernel Density Estimate on the logits from the ﬁnal softmax layer (Feinman et al.
(Gao et al. 2021) used Maximum mean discrepancy (MMD) from the distribution of the input examples to classify adversaries based on their probability of occurrence in the input distribution.
PixelDefend
(Song et al. 2017) uses PixelCNN to get the Bits Per Dimension (BPD) score for the input image.
(Xu, Evans, and Qi 2017) uses the difference in the ﬁnal logit vector for original and squeezed images as a medium to create distribution and use it for inference.
(Jha et al.
2018) compares different dimensionality reduction techniques to get low level representations of input images and use it for bayesian inference to detect adversaries.
Someother special methods include use of SHAP signatures (Fidel, Bitton, and Shabtai 2020) which are used for getting explanations on where the classiﬁer network is focusing as an input for detecting adversaries.
A detailed comparative study with all these existing approaches is summarized through Table V in the Appendix .
Comparison with State-of-the-Art using Generative Networks
Finally we compare our work with these 3 works (Meng and Chen 2017)
(Hwang et al. 2019)
(Samangouei, Kabkab, and Chellappa 2018) proposed earlier which uses Generative networks for detection and puriﬁcation of adversaries.
We make our comparison on MNIST dataset which is used commonly in the 3 works (Table III).
Our results are typically the best for all attacks or are off by short margin from the best.
For the strongest attack, our performance is much better.
This show how our method is more effective while not being confused with random perturbation as an adversary.
More details are given in the Appendix .
[Table]
Conclusion
In this work we propose the use of Conditional Variational
AutoEncoder (CVAE) for detecting adversarial attacks.
We utilized statistical base methods to verify that the adversarial attacks usually lie outside of the training distribution
We demonstrate how our method can speciﬁcally differentiate between random perturbations and targeted attacks which is necessary for some applications where the raw camera image may contain random noises which should not be confused with an adversarial attack.
Furthermore, we demonstrate how it takes huge targeted perturbation to fool both the detector as well as the target classiﬁer.
Our framework presents a practical, effective and robust adversary detection approach in comparison to existing state-ofthe-art techniques which falter to differentiate noisy data from adversaries.
As a possible future work, it would be interesting to see the use of Variational AutoEncoders for automatically purifying the adversarialy attacked images
properties of neural networks
adversarial examples is (nearly) as hard as classifying them
Appendix Use of simple AutoEncoder (AE)
MagNet (Meng and Chen 2017) uses AutoEncoder (AE) for detecting adversaries.
We compare the results with our proposed CVAE architecture on the same experiment setting and present the comparison in AUC values of the
ROC curve observed for the 2 cases.
Although the paper’s claim is based on both detection as well as puriﬁcation (if not detected) of the adversaries.
MagNet uses their detection framework for detecting larger adversarial perturbations which cannot be puriﬁed.
For smaller perturbations,
MagNet proposes to purify the adversaries by a different
AutoEncoder model.
We make the relevant comparison only for the detection part with our proposed method.
Using the same architecture as proposed, our results are better for the strongest attack while not getting confused with random perturbations of similar magnitude.
ROC curves obtained for different adversaries for MagNet are given in Figure
Use of Variational AutoEncoder (VAE)
PuVAE (Hwang et al. 2019) uses Variational AutoEncoder (VAE) for purifying adversaries.
We compare the results with our proposed CVAE architecture on the same experiment setting.
PuVAE however, does not propose using
VAE for detection of adversaries but in case if their model is to be used for detection, it would be based on the reconstruction distance.
So, we make the comparison with our proposed CVAE architecture.
ROC curves for different adversaries are given in Figure
Use of Generative Adversarial Network (GAN)
Defense-GAN (Samangouei, Kabkab, and Chellappa 2018) uses Generative Adversarial Network (GAN) for detecting adversaries.
We used L = 100 and R = 10 for getting the results as per our experiment setting.
We compare the results with our proposed CVAE architecture on the same experiment setting and present the comparison in
AUC values of the ROC curve observed for the 2 cases.
Although the paper’s main claim is about puriﬁcation of the adversaries, we make the relevant comparison for the detection part with our proposed method.
We used the same architecture as mentioned in (Samangouei, Kabkab, and Chellappa 2018) and got comparable results as per their claim for MNIST dataset on FGSM adversaries.
As this method took a lot of time to run, we randomly chose 1000 samples out of 10000 test samples for evaluation due to time constraint.
The detection performance for other attacks is considerably low.
Also, Defense-GAN is quite slow as it needs to solve an optimization problem for each image to get its corresponding reconstructed image.
Average computation time required by Defense-GAN is 2 .
8 s per image while our method takes 0 .
17 s per image with a batch size of 16 .
Hence, our method is roughlytimes faster than Defense-GAN.
Refer to Figure 13 for the
ROC curves for Defense-GAN.
Reporting the results in robust detection risk form
(Tram`er 2021) argued that most of the results reported for detection form are inconsistent and there seems to be a fair chance for works to over-claim the detection results. (Tram`er 2021) shows a reduction from robust detection for a given (cid:15) bound to robust puriﬁcation of images within (cid:15) 2 by the same margin of error. This means that a robust detector being able to detect all adversaries within (cid:15) bound is equivalent to a robust (but inefﬁcient) puriﬁer that puriﬁes all adversaries within (cid:15) 2 bound. While, using Area Under the Curve (AUC) of the full ROC curves can be a good way for comparison of different detectors, we additionally present results in the robust detection risk form (Equation 12) as suggested by (Tram`er 2021).
The upper bound on value of robust risk ( R (cid:15)adv − det ) can be obtained by Equation 13.
We choose appropriate FPR from the ROC curve such that the robust risk ( R (cid:15),upperadv − det ) gets minimised.
The results for grey-box attacks are reported in table IV.
[Equation]
[Table]
References
[Table]
Binaryclassiﬁernetworktrainedwithinputimage
[Equation]
[Table]
Trained with generatedadversaries, hence does
[Table]
well on other
[Table] [Table]
to
[Table]
No
[Table]
, ability to differentiate from randomly perturbed images (
[Table]
LSTM on dis- tant features at each layer of target classiﬁer network
ILSVRCdataset
[Equation]
Not evaluated for differentiation from random noisy images, on special attack which has access to network weights
No use of adversaries for training, ability to differentiate from randomly perturbed images, evaluaion on l 2 attacks (Feinmanetal.2017)
Bayesiandensityestimate on ﬁnal softmax layer
[Equation]
No explicit test for random noisy images
Ability to differentiate between randomly perturbed images, better AUC values (
[Table]
Using PixelDe- fend to get reconstruction er-
ror on input image
FashionMNIST,
CIFARFGSM,BIM,
DeepFool, CW ( L ∞ )
ROC curves given, AUC not given
Cannot differentiate random noisy images from adversaries
Ability to differenti- ate between randomly perturbed and clean images (Xu,
[Table]