Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions

1 INTRODUCTION . Adversarial examples ( Szegedy et al. , 2013 ) are inputs that are designed by an adversary to cause a machine learning system to make a misclassification . A series of studies on adversarial attacks have shown that it is easy to cause misclassifications using visually imperceptible changes to an image under ` p-norm based similarity metrics ( Goodfellow et al. , 2014 ; Kurakin et al. , 2016 ; Madry et al. , 2017 ; Carlini & Wagner , 2017b ; Goodfellow et al. , 2018 ) . Since the discovery of adversarial examples , there has been a constant “ arms race ” between better attacks and better defenses . Many new defenses have been proposed ( Song et al. , 2017 ; Gong et al. , 2017 ; Grosse et al. , 2017 ; Metzen et al. , 2017 ) , only to be broken shortly thereafter ( Carlini & Wagner , 2017a ; Athalye et al. , 2018 ) . Hinton et al . ( 2018 ) showed that capsule models are more robust to simple adversarial attacks than CNNs but Michels et al . ( 2019 ) showed that this is not the case for all attacks . The cycle of attacks and defenses motivates us to rethink both how we can improve the general robustness of neural networks as well as the high-level motivation for this pursuit . One potential path forward is to detect adversarial inputs , instead of attempting to accurately classify them ( Schott et al. , 2018 ; Roth et al. , 2019 ) . Recent work ( Jetley et al. , 2018 ; Gilmer et al. , 2018b ) argue that adversarial examples can exist within the data distribution , which implies that detecting adversarial examples based on an estimate of the data distribution alone might be insufficient . Instead , in this paper we develop methods for detecting adversarial examples by making use of class-conditional reconstruction networks . These sub-networks , first proposed by Sabour et al . ( 2017 ) as part of a Capsule Network ( CapsNet ) , allow a model to produce a reconstruction of its input based on the identity and instantiation parameters of the winning capsule . Interestingly , we find that reconstructing ∗Equal Contributions . an input from the capsule corresponding to the correct class results in a much lower reconstruction error than reconstructing the input from capsules corresponding to incorrect classes , as shown in Figure 1 ( a ) . Motivated by this , we propose using the reconstruction sub-network in a CapsNet as an attack-independent detection mechanism . Specifically , we reconstruct a given input from the pose parameters of the winning capsule and then detect adversarial examples by comparing the difference between the reconstruction distributions for natural and adversarial ( or otherwise corrupted ) images . We extend this detection mechanism to standard convolutional neural networks ( CNNs ) and show its effectiveness against black box and white box attacks on three image datasets ; MNIST , FashionMNIST and SVHN . We show that capsule models achieve the strongest attack detection rates and accuracy on these attacks . We then test our method against a stronger attack , the Reconstructive Attack , specifically designed to attack our detection mechanism by generating adversarial examples with a small reconstruction error . With this attack we are able to create undetected adversarial examples , but we show that this attack is less successful in fooling the classifier than a non-reconstructive attack . Among all these attacks , we find CapsNets perform the best in detecting adversarial examples . To explain the success of CapsNets over CNNs , we further diagnose the adversarial examples for CapsNets and find that 1 ) the success of the targeted reconstructive attack is highly dependent on the visual similarity between the source image and the target class . 2 ) many of the resultant attacks resemble members of the target class and so cease to be “ adversarial ” – i.e. , they may also be misclassified by humans . These findings suggest that CapsNets with class conditional reconstructions have the potential to address the real issue with adversarial examples – networks should make predictions based on the same properties of the image that people use rather than using features that can be manipulated by an imperceptible adversarial attack . In summary , our main contributions are : • We propose a class-conditional capsule reconstruction based detection method to detect standard white-box/black-box adversarial examples on three datasets . This detection mechanism is attack-agnostic and is successfully extended to standard convolutional neural networks . • We test our detection mechanism on the corrupted MNIST dataset and show that it can work as a general out-of-distribution detector . • A stronger reconstructive attack is specifically designed to attack our detection mechanism but becomes less successful in fooling the classifier . • We perform extensive qualitative studies to explain the superior performance of CapsNets in detecting adversarial examples compared to CNNs . The results suggest that the features captured by CapsNets are more aligned with human perception . 2 RELATED WORK . Adversarial examples were first introduced in ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ) , where a given image was modified by following the gradient of a classifier ’ s output with respect to the image ’ s pixels . Goodfellow et al . ( 2014 ) then developed the more efficient Fast Gradient Sign method ( FGSM ) , which can change the label of the input image X with a similarly imperceptible perturbation that is constructed by taking an step in the direction of the gradient . Later , the Basic Iterative Method ( BIM ) ( Kurakin et al. , 2016 ) and Projected Gradient Descent ( Madry et al. , 2017 ) can generate stronger attacks improved on FGSM by taking multiple steps in the direction of the gradient . In addition , Carlini & Wagner ( 2017b ) proposed another iterative optimization-based method to construct strong adversarial examples with small perturbations . An early approach to reducing vulnerability to adversarial examples was proposed by ( Goodfellow et al. , 2014 ) , where a network was trained on both clean images and adversarially perturbed ones . Since then , there has been a constant “ arms race ” between better attacks and better defenses ; Kurakin et al . ( 2018 ) provide an overview of this field . However , many defenses against adversarial examples have been demonstrated to be an effect of “ obfuscated gradients ” and can be further circumvented under the white-box setting ( Athalye et al. , 2018 ) . Another line of work attempts to circumvent adversarial examples by detecting them with a separatelytrained classifier ( Gong et al. , 2017 ; Grosse et al. , 2017 ; Metzen et al. , 2017 ) or using statistical properties ( Hendrycks & Gimpel , 2016 ; Li & Li , 2017 ; Feinman et al. , 2017 ; Grosse et al. , 2017 ) . However , many of these approaches were subsequently shown to be flawed ( Carlini & Wagner , 2017a ; Athalye et al. , 2018 ) . The most recent work in detecting adversarial examples ( Roth et al. , 2019 ) that has a 99 % true positive rate on CIFAR-10 dataset ( Krizhevsky , 2009 ) has also been fully bypassed by later work ( Hosseini et al. , 2019 ) which decreased the true positive rate to less than 2 % . Similar to our work , Schott et al . ( 2018 ) also investigated the effectiveness of a class-conditional generative model as a defense mechanism for MNIST digits . However , we differ in some important ways . Their model is in some ways the opposite of ours - they first attempt to generate the input , and then make a classification on the resulting generated images , whereas our method attempts to first classify the input , making use of an otherwise unchanged capsule classification model , and then generates the input from a high level representation . As such , our method does not increase the computational overhead of classifying the input , compared to the approach of Schott et al . ( 2018 ) . In addition , the work of Schott et al . ( 2018 ) is only applied to MNIST , so our results on the more complex datasets represent an improvement . 3 PRELIMINARIES . Adversarial Examples Given a clean test image x , its corresponding label y , and a classifier f ( · ) which predicts a class label given an input , we refer to x′ = x+ δ as an adversarial example if it is able to fool the classifier into making a wrong prediction f ( x′ ) 6= f ( x ) = y . The small adversarial perturbation δ ( where “ small ” is measured under some norm ) causes the adversarial example x′ to appear visually similar to the clean image x but to be classified differently . In the unrestricted case where we only require that f ( x′ ) 6= y , we refer to x′ as an “ untargeted adversarial example ” . A more powerful attack is to generate a “ targeted adversarial example ” : instead of simply fooling the classifier to make a wrong prediction , we force the classifier to predict some targeted label f ( x′ ) = t 6= y . In this paper , the target label t is selected uniformly at random as any label which is not the ground-truth correct label . As is standard practice in the literature , in this paper we test our detection mechanism on three ` ∞ norm based attacks ( fast gradient sign method ( FGSM ) ( Goodfellow et al. , 2014 ) , the basic iterative method ( BIM ) ( Kurakin et al. , 2016 ) , projected gradient descent ( PGD ) ( Madry et al. , 2017 ) ) and one ` 2 norm based attack ( Carlini-Wagner ( CW ) ( Carlini & Wagner , 2017b ) ) . Capsule Networks Capsule Networks ( CapsNets ) are an alternative architecture for neural networks ( Sabour et al. , 2017 ; Hinton et al. , 2018 ) . In this work we make use of the CapsNet architecture detailed by ( Sabour et al. , 2017 ) . Unlike a standard neural network which is made up of layers of scalar-valued units , CapsNets are made up of layers of capsules , that output a vector or matrix . Intuitively , just as one can think of the activation of a unit in a normal neural network as the presence of a feature in the input , the activation of a capsule can be thought of as both the presence of a feature and the pose parameters that represent attributes of that feature . A top-level capsule in a classification network therefore outputs both a classification and pose parameters that represent the instance of that class in the input . This high level representation allows us to train a reconstruction network . Threat Model In this paper , we test our detection mechanism against both white-box and black-box attacks . For white-box attacks , the adversary has full access to the model as well as its parameters . In particular , the adversary is allowed to compute the gradient through the model to generate adversarial examples . To perform black-box attacks , the adversary is allowed to know the network architecture but not its parameters . Therefore , we retrain a substitute model that has the same architecture as the target model and generate adversarial examples by attacking the substitute model . Then we transfer these attacks to the target model . For ` ∞ based attacks , we always control the ` ∞ norm of the adversarial perturbation to be within a relatively small bound ∞ , specific to each dataset .

Summary: This paper studies the problem of detecting and generating adversarial images using class-conditional capsule networks. Specifically, this paper first introduced a novel method that detects adversarial examples by class-conditional image reconstruction. Motivated by this defense method, this paper further proposed a novel reconstructive attack that minimizes both classification and reconstruction loss. Experimental evaluations are conducted on MNIST, FashionMNIST, SVHN, and CIFAR-10 dataset. Results demonstrate the effectiveness of the proposed defense and the novel reconstructive attack method.