Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions

1 INTRODUCTION . Adversarial examples ( Szegedy et al. , 2013 ) are inputs that are designed by an adversary to cause a machine learning system to make a misclassification . A series of studies on adversarial attacks have shown that it is easy to cause misclassifications using visually imperceptible changes to an image under ` p-norm based similarity metrics ( Goodfellow et al. , 2014 ; Kurakin et al. , 2016 ; Madry et al. , 2017 ; Carlini & Wagner , 2017b ; Goodfellow et al. , 2018 ) . Since the discovery of adversarial examples , there has been a constant “ arms race ” between better attacks and better defenses . Many new defenses have been proposed ( Song et al. , 2017 ; Gong et al. , 2017 ; Grosse et al. , 2017 ; Metzen et al. , 2017 ) , only to be broken shortly thereafter ( Carlini & Wagner , 2017a ; Athalye et al. , 2018 ) . Hinton et al . ( 2018 ) showed that capsule models are more robust to simple adversarial attacks than CNNs but Michels et al . ( 2019 ) showed that this is not the case for all attacks . The cycle of attacks and defenses motivates us to rethink both how we can improve the general robustness of neural networks as well as the high-level motivation for this pursuit . One potential path forward is to detect adversarial inputs , instead of attempting to accurately classify them ( Schott et al. , 2018 ; Roth et al. , 2019 ) . Recent work ( Jetley et al. , 2018 ; Gilmer et al. , 2018b ) argue that adversarial examples can exist within the data distribution , which implies that detecting adversarial examples based on an estimate of the data distribution alone might be insufficient . Instead , in this paper we develop methods for detecting adversarial examples by making use of class-conditional reconstruction networks . These sub-networks , first proposed by Sabour et al . ( 2017 ) as part of a Capsule Network ( CapsNet ) , allow a model to produce a reconstruction of its input based on the identity and instantiation parameters of the winning capsule . Interestingly , we find that reconstructing ∗Equal Contributions . an input from the capsule corresponding to the correct class results in a much lower reconstruction error than reconstructing the input from capsules corresponding to incorrect classes , as shown in Figure 1 ( a ) . Motivated by this , we propose using the reconstruction sub-network in a CapsNet as an attack-independent detection mechanism . Specifically , we reconstruct a given input from the pose parameters of the winning capsule and then detect adversarial examples by comparing the difference between the reconstruction distributions for natural and adversarial ( or otherwise corrupted ) images . We extend this detection mechanism to standard convolutional neural networks ( CNNs ) and show its effectiveness against black box and white box attacks on three image datasets ; MNIST , FashionMNIST and SVHN . We show that capsule models achieve the strongest attack detection rates and accuracy on these attacks . We then test our method against a stronger attack , the Reconstructive Attack , specifically designed to attack our detection mechanism by generating adversarial examples with a small reconstruction error . With this attack we are able to create undetected adversarial examples , but we show that this attack is less successful in fooling the classifier than a non-reconstructive attack . Among all these attacks , we find CapsNets perform the best in detecting adversarial examples . To explain the success of CapsNets over CNNs , we further diagnose the adversarial examples for CapsNets and find that 1 ) the success of the targeted reconstructive attack is highly dependent on the visual similarity between the source image and the target class . 2 ) many of the resultant attacks resemble members of the target class and so cease to be “ adversarial ” – i.e. , they may also be misclassified by humans . These findings suggest that CapsNets with class conditional reconstructions have the potential to address the real issue with adversarial examples – networks should make predictions based on the same properties of the image that people use rather than using features that can be manipulated by an imperceptible adversarial attack . In summary , our main contributions are : • We propose a class-conditional capsule reconstruction based detection method to detect standard white-box/black-box adversarial examples on three datasets . This detection mechanism is attack-agnostic and is successfully extended to standard convolutional neural networks . • We test our detection mechanism on the corrupted MNIST dataset and show that it can work as a general out-of-distribution detector . • A stronger reconstructive attack is specifically designed to attack our detection mechanism but becomes less successful in fooling the classifier . • We perform extensive qualitative studies to explain the superior performance of CapsNets in detecting adversarial examples compared to CNNs . The results suggest that the features captured by CapsNets are more aligned with human perception . 2 RELATED WORK . Adversarial examples were first introduced in ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ) , where a given image was modified by following the gradient of a classifier ’ s output with respect to the image ’ s pixels . Goodfellow et al . ( 2014 ) then developed the more efficient Fast Gradient Sign method ( FGSM ) , which can change the label of the input image X with a similarly imperceptible perturbation that is constructed by taking an step in the direction of the gradient . Later , the Basic Iterative Method ( BIM ) ( Kurakin et al. , 2016 ) and Projected Gradient Descent ( Madry et al. , 2017 ) can generate stronger attacks improved on FGSM by taking multiple steps in the direction of the gradient . In addition , Carlini & Wagner ( 2017b ) proposed another iterative optimization-based method to construct strong adversarial examples with small perturbations . An early approach to reducing vulnerability to adversarial examples was proposed by ( Goodfellow et al. , 2014 ) , where a network was trained on both clean images and adversarially perturbed ones . Since then , there has been a constant “ arms race ” between better attacks and better defenses ; Kurakin et al . ( 2018 ) provide an overview of this field . However , many defenses against adversarial examples have been demonstrated to be an effect of “ obfuscated gradients ” and can be further circumvented under the white-box setting ( Athalye et al. , 2018 ) . Another line of work attempts to circumvent adversarial examples by detecting them with a separatelytrained classifier ( Gong et al. , 2017 ; Grosse et al. , 2017 ; Metzen et al. , 2017 ) or using statistical properties ( Hendrycks & Gimpel , 2016 ; Li & Li , 2017 ; Feinman et al. , 2017 ; Grosse et al. , 2017 ) . However , many of these approaches were subsequently shown to be flawed ( Carlini & Wagner , 2017a ; Athalye et al. , 2018 ) . The most recent work in detecting adversarial examples ( Roth et al. , 2019 ) that has a 99 % true positive rate on CIFAR-10 dataset ( Krizhevsky , 2009 ) has also been fully bypassed by later work ( Hosseini et al. , 2019 ) which decreased the true positive rate to less than 2 % . Similar to our work , Schott et al . ( 2018 ) also investigated the effectiveness of a class-conditional generative model as a defense mechanism for MNIST digits . However , we differ in some important ways . Their model is in some ways the opposite of ours - they first attempt to generate the input , and then make a classification on the resulting generated images , whereas our method attempts to first classify the input , making use of an otherwise unchanged capsule classification model , and then generates the input from a high level representation . As such , our method does not increase the computational overhead of classifying the input , compared to the approach of Schott et al . ( 2018 ) . In addition , the work of Schott et al . ( 2018 ) is only applied to MNIST , so our results on the more complex datasets represent an improvement . 3 PRELIMINARIES . Adversarial Examples Given a clean test image x , its corresponding label y , and a classifier f ( · ) which predicts a class label given an input , we refer to x′ = x+ δ as an adversarial example if it is able to fool the classifier into making a wrong prediction f ( x′ ) 6= f ( x ) = y . The small adversarial perturbation δ ( where “ small ” is measured under some norm ) causes the adversarial example x′ to appear visually similar to the clean image x but to be classified differently . In the unrestricted case where we only require that f ( x′ ) 6= y , we refer to x′ as an “ untargeted adversarial example ” . A more powerful attack is to generate a “ targeted adversarial example ” : instead of simply fooling the classifier to make a wrong prediction , we force the classifier to predict some targeted label f ( x′ ) = t 6= y . In this paper , the target label t is selected uniformly at random as any label which is not the ground-truth correct label . As is standard practice in the literature , in this paper we test our detection mechanism on three ` ∞ norm based attacks ( fast gradient sign method ( FGSM ) ( Goodfellow et al. , 2014 ) , the basic iterative method ( BIM ) ( Kurakin et al. , 2016 ) , projected gradient descent ( PGD ) ( Madry et al. , 2017 ) ) and one ` 2 norm based attack ( Carlini-Wagner ( CW ) ( Carlini & Wagner , 2017b ) ) . Capsule Networks Capsule Networks ( CapsNets ) are an alternative architecture for neural networks ( Sabour et al. , 2017 ; Hinton et al. , 2018 ) . In this work we make use of the CapsNet architecture detailed by ( Sabour et al. , 2017 ) . Unlike a standard neural network which is made up of layers of scalar-valued units , CapsNets are made up of layers of capsules , that output a vector or matrix . Intuitively , just as one can think of the activation of a unit in a normal neural network as the presence of a feature in the input , the activation of a capsule can be thought of as both the presence of a feature and the pose parameters that represent attributes of that feature . A top-level capsule in a classification network therefore outputs both a classification and pose parameters that represent the instance of that class in the input . This high level representation allows us to train a reconstruction network . Threat Model In this paper , we test our detection mechanism against both white-box and black-box attacks . For white-box attacks , the adversary has full access to the model as well as its parameters . In particular , the adversary is allowed to compute the gradient through the model to generate adversarial examples . To perform black-box attacks , the adversary is allowed to know the network architecture but not its parameters . Therefore , we retrain a substitute model that has the same architecture as the target model and generate adversarial examples by attacking the substitute model . Then we transfer these attacks to the target model . For ` ∞ based attacks , we always control the ` ∞ norm of the adversarial perturbation to be within a relatively small bound ∞ , specific to each dataset .

Summary: This paper proposed a new defense method for capsule networks. For both white-box and black-box settings, the proposed CapNets has shown superior performance than two variants of CNNs. The visualizations of adversarial examples generated by the CapNets are more aligned with the human perception which is very insightful. On the corrupted MNIST dataset, the results show the proposed defense method can also be used well as an out-of-distribution detector. Overall the paper is clearly written and easy to follow.