Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Authors: Yao Qin
Nicholas Frosst
Sara Sabour
Colin Raffel
Garrison Cottrell
Geoffrey Hinton
Venue:  (2020)
Keywords: Adversarial Examples
Detection of adversarial attacks

Abstract: Adversarial examples raise questions about whether neural network models are sensitive to the same visual features as humans. In this paper, we first detect adversarial examples or otherwise corrupted images based on a class-conditional reconstruction of the input. To specifically attack our detection mechanism, we propose the Reconstructive Attack which seeks both to cause a misclassification and a low reconstruction error. This reconstructive attack produces undetected adversarial examples but with much smaller success rate. Among all these attacks, we find that CapsNets always perform better than convolutional networks. Then, we diagnose the adversarial examples for CapsNets and find that the success of the reconstructive attack is highly related to the visual similarity between the source and target class. Additionally, the resulting perturbations can cause the input image to appear visually more like the target class and hence become non-adversarial. This suggests that CapsNets use features that are more aligned with human perception and have the potential to address the central issue raised by adversarial examples.

Published as a conference paper at ICLR 2020
DETECTING AND DIAGNOSING ADVERSARIAL IMAGES WITH CLASS-COND CONDITIONL CAPSULE RECONSTRUCTIONS
Yao Qin'
Nicholas Frosst* Google Brain f $rosst&google..o com
Sara Sabour
UC San Diego
Google Brain
yaq007@eng ucsd. edu
sasabour@google..om
Colin Raffel
Garrison Cottrell UC San Diego
Geoffrey Hinton
Google Brain
Google Brain
craffel@google: com
garygeng. uwsd.eddee
geoffhinton@googee..
ABSTRACT
Adversarial examples raise questions about whether neural network models are sensitive to the same visual features humans. In this paper, first detect adversarial examples or otherwise corrupted images based on a class-conditional reconstruction of the input. To specifically attack our detection mechanism, we propose the Reconstructive Attack which seeks both to cause misclassification and low reconstruction error. This reconstructive attack produces undetected adversarial examples but with much smaller success rate. Among all these attacks, we find that CapsNets always perform better than convolutional networks. Then, we diagnose the adversarial examples for CapsNets and find that the success of the reconstructive attack highly related the visual similarity between the source and target class. Additionally, the resulting perturbations can cause the input image to appear visually more like the target class and hence become non-adversarial. This suggests that CapsNets use features that are more aligned with human perception and have the potential to address the central issue raised by adversarial examples.
INTRODUCTION
Adversarial examples (Szegedy et al. 2013) are inputs that are designed by an adversary to cause machine learning system to make misclassification. A series of studies adversarial attacks have shown that it is easy to cause misclassifications using visually imperceptible changes to an image under 2,-norm based similarity metrics (Goodfellow et al.) 2014] Kurakin et al.) 2016 Madry et al. 2017] Carlini & Wagner 2017b] Goodfellow et al., 2018). Since the discovery of adversarial examples, there has been constant "arms race" between better attacks and better defenses. Many new defenses have been proposed (Song et al. 2017 Gong et al. 2017 Grosse et al. | broken shortly thereafter (Carlini Wagner 2017a Athalye al. (0018). Hinton et al. 2018) showed that capsule models are more robust to simple adversarial attacks than CNNs but Michels al. (2019) showed that this not the case for all attacks.
al.) (0017). only
The cycle of attacks and defenses motivates to rethink both how we can improve the general robustness of neural networks as well as the high-level motivation for this pursuit. One potential path forward is to detect adversarial inputs, instead of attempting to accurately classify them (Schott) al. 2018 Roth al. 2019). Recent work (Jetley et al., 2018 Gilmer et al., 2018b) argue that exist within the data distribution, which implics that detecting adversarial examples based on an estimate of the data distribution alone might be insufficient. Instead, this paper develop methods for detecting adversarial examples by making reconstruction networks. These sub-networks, first proposed by [abour et al.] (2017) as part of a Capsule Network (CapsNet), allow model produce a reconstruction of its input based on the identity and instantiation parameters of the winning capsule. Interestingly, we find that reconstructing
adversarial examples
of class-conditional
"Equal Contributions.

---

Published as a conference paper at ICLR 2020
CapsNet
CapsNet
CNN+CR
CNN+R
10000
3000
5000
5000
Clean
8000
PCD
4030
3000
3000
4000
2000
2000
2000
2000
1000
1000
20
40
66
80 100
120
140
20
60
80
100
120
140
20
40
60
20
100
120
140
20
40
60
80
100
120
140
(8)
(b)
Figure I: (a) The histogram of & distances between the input and the reconstruction using the correct capsule or other capsules in CapsNet on the real MNIST images. Notice the stark difference between the distributions of reconstructions of the capsule corresponding the correct class and other capsules. (b) The histograms of lg distances between the reconstruction and the input for real and adversarial images for the three models explored in this paper on the MNIST dataset. We use PGD (aadry et al. 2017) with the foo bound 0.3 to create the attacks.
an input from the capsule corresponding to the correct class results in a much lower reconstruction error than reconstructing the input from capsules corresponding to incorrect classes, as shown in Figure a)). Motivated by this, we propose using the reconstruction sub-network in CapsNet as an attack-indes sitec--indeeeneen detection mechanism. Specifieally, parameters of the winning capsule and then detect adversarial examples by comparing the difference between the reconstruction distributions for natural and adversarial (or otherwise corrupted) images.
reconstruct given input from the pose
We extend this detection mechanism to standard convolutional neural networks (CNNs) and show its effectiveness against black box and white box attacks on three image datasets; MNIST, Fashion- MNIST and SVHN. We show that capsule models achieve the strongest attack detection rates and accuracy on these attacks. We then test our method against stronger attack, the Reconstructive Attack, specifically designed to attack our detection mechanism by generating adversarial examples with a small reconstruction error. With this attack we are able to create undetected adversarial examples, but we show that this attack is less successful in fooling the classifier than a non-reconstructive attack.
Among all these attacks, we find CapsNets perform the best in detecting adversarial examples. To explain the success of CapsNets over CNNs, we further diagnose the adversarial examples for CapsNets and find that 1) the success of the targeted reconstructive attack highly dependent on the visual similarity between the source image and the target class. 2) many of the resultant attacks resemble members of the target class and so cease to be "adversarial" i..., they may also be misclassified by humans. These findings suggest that CapsNets with class conditional reconstructions have the potential to address the real issue with adversarial examples networks should make predictions based the same properties of the image that people rather than using features that can be manipulated by an imperceptible adversarial attack.
In summary, our main contributions are:
We propose class-conditional capsule reconstruction based detection method to detect stan- dard biite-box/black-box adversarial examples on three datasets. This detection mechanism is attack-agnostic and is successfully extended to standard convolutional neural networks.
We test our detection mechanism on the corrupted MNIST dataset and show that it can work general diitr--diitiiittion detector.
A stronger reconstructive attack is specifically designed to attack our detection mechanism but becomes less successful in fooling the classifier.
We perform extensive qualitative studies to explain the superior performance of CapsNets in detecting adversarial examples compared CNNs. The results suggest that the features captured by CapsNets are more aligned with human perception.
2
RELATED WORK
Adversarial examples were first introduced in (Biggio et al., 2013 Szegedy et al. (013), where given image modified by following the gradient of classifier's output with respect to the
2

---

Published as a conference paper at ICLR 2020
image's pixels. Goodfellow et al. (2014) then developed the more efficient Fast Gradient Sign method (FGSM), which can change the label of the input image X with a similarly imperceptible perturbation that is constructed by taking an step in the direction of the gradient. Later, the Basic Iterative Method (BIM) (Kurakin et al. 2016) and Projected Gradient Descent (Madry et al.] 2017) can generate stronger attacks improved on FGSM by taking multiple steps in the direction of the gradient. In addition. Carlini & Wagner (2017b) proposed another iterative optimization-bn based method to construct strong adversarial examples with small perturbations.
An early approach to reducing vulnerability to adversarial examples was proposed by (Goodfellow jet al., 2014). where network was trained on both clean images and adversarially perturbed ones. Since then, there has been a constant "arms race" between better attacks and better defenses:|Kurkkin Kurakin jet al. 2018)) provide an overview of this field. However, many defenses against adversarial examples have been demonstrated to be an effect of "obfuscated gradients" and can be further circumvented under the white-box setting (Athalye et al. 2018).
Another line of work attempts to circumvent adversarial examples by detecting them with separately- trained classifier (Gong et al. 2017: Grosse et al. 2017, Metzen et al. (0117) or using statistical
properties (Hendrycks & Gimpel 2016)
200 6 Li & Lij 2017: einman et al.] Grosse et al. 2017).
However, many of these approaches were subsequently shown to be flawed (Carlini & Wagner 2017a
Athalye al., 2018). The most recent work in detecting adversarial examples (Roth et al. (2119) 9) that has a 99% true positive rate on CIFAR-10 dataset (Krizhevsky) 2009) has also been fully bypassed by later work (Hosseini et al. 2019) which decreased the true positive rate to less than 2%.
Similar to our work, |Schott et al. 2018) also investigated the effectiveness of a cansscoondiiioall generative model as defense mechanism for MNIST digits. However, we differ in some important ways. Their model is in some ways the opposite of ours they first attempt to generate the input, and then make a classification on the resulting generated images, whereas our method attempts to first classify the input, making use of an otherwise unchanged capsule classification model, and then generates the input from a high level representation. As such, our method does not increase the computational overhead of classifying the input, compared to the approach Schott et al. (201 8). In addition, the work of Schott et al. (2018) is only applied to MNIST, so our results on the more complex datasets represent an improvement.
PRELIMINARIES
Adversarial Examples Given clean test image x, its corresponding label y, and a classifier f(() which predicts a class label given an input, refer to x' + 5 as adversarial example if it is able to fool the classifier into making a wrong prediction f(x') ) * f(x) = y. The small adversarial perturbation 5 (where "small" is measured under some norm) causes the adversarial example x' to appear visually similar to the clean image x but to be classified differently. In the unrestricted case where we only require that f f(x' F y. refer to x' as "untargeted adversarial example". A more powerful attack is to generate a "targeted adversarial example": instead of simply fooling the classifier to make a wrong prediction, this paper, the target label t is selected uniformly at random as any label which is not the ground truth correct label. As is standard practice in the literature, in this paper we test our detection mechanism on three foo norm based attacks (fast gradient sign method (FGSM) (Goodfellow et al. 2014), the basic iterative method (BIM) (Kurakin et al. 2016), projected gradient descent (PGD) (Madry et al.. )017 and one lg norm based attack (Carlini- -agner (CW) (Carlini & Wagner 2017b)
force the classifier to predict some targeted label f(x') j y. In
Capsule Networks Capsule Networks (CapsNets) are an alternative architecture for neural net- works (Sahour et al, 2017 Hinton et al., 2018). this work we make use of the CapsNet architecture detailed by (Sabour et al. 2017). Unlike a standard neural network which is made up of layers of scalar-valued units, CapsNets are made up of layers of capsules, that output a vector or matrix. Intuitively, just one can think of the activation of a unit in a normal neural network as the presence of a feature in the input, the activation of a capsule can be thought of as both the presence of a feature and the pose parameters that represent attributes of that feature. A levele capsule in classification network therefore outputs both a classification and pose parameters that represent the instance of that class in the input. This high level representation allows to train reconstruction network.
3

---

Published as a conference paper at ICLR 2020
Threat Model In this paper, test our detection mechanism against both white-box and black-box attacks. For white-box attacks, the adversary has full access to the model as well as its parameters. In particular, the adversary is allowed to compute the gradient through the model to generate adversarial examples. To perform black-box attacks, the adversary is allowed to know the network architecture but not its parameters. Therefore, we retrain substitute model that has the same architecture as the target model and generate adversarial examples by attacking the substitute model. Then we transfer these attacks to the target model. For foo based attacks, we always control the &oo norm of the adversarial perturbation to be within relatively small bound 6oo, specific to each dataset.
DETECTING ADVERSARIAL IMAGES BY RECONSTRUCTION
To detect adversarial images, we make use the reconstruction network proposed in (abour et al. 2017), which takes pose parameters v as input and outputs the reconstructed image r(æ). The reconstruction network is simply fully connected neural network with two ReLU hidden layers with 512 and 1024 units respectively, with sigmoid output with the same dimensionality as the dataset. The reconstruction network is trained to minimize the lg distance between the input image and the reconstructed image. This same network architecture is used for all the models and datasets we explore. The only difference is what is given to the reconstruction network as input.
4.1 MODELS
The reconstruction network of the CapsNet is class-conditional: It takes in the pose parameters of all the class capsules and masks all values to except for the pose parameters of the predicted class. We use this reconstruction network for detecting adversarial attacks by measuring the Euclidean distance between the input and a class conditional reconstruction. Specifically, for any given input x, the CapsNet outputs a prediction f()) as well the pose parameters for all classes. The reconstruction network takes in the pose parameters and then selects the pose parameter corresponding to the predicted class, denoted f((x)- generate a reconstruction r((r(=)). Then compute the lg reconstruction distance d(x) [|r()) 2212 between the reconstructed image and the input image, and compare it with a pre-defined detection threshold 0 (described below in Section 4.2p. If the reconstruction distance d(x)) is higher than the detection threshold 0, flag the input as an adversarial example. Figure (b) shows an example of histograms of reconstruction distances for natural images and typical adversarial examples.
CapsNet
CNN+CR Although our strategy is inspired by the reconstruction networks used CapsNets, the strategy can be extended to standard convolutional neural networks (CNNs). We create a similar architecture, CNN with conditional reconstruction (CNN+CR). by dividing the penultimate hidden layer of a CNN into groups corresponding to each class. The sum of each neuron group serves as the logit for that particular class and the group itself serves the same purpose as the pose parameters the CapsNet. We use the same masking mechanism as Sabour et al. (2017) to select the pose parameter corresponding to the predicted label (w) and generate the reconstruction based on the selected pose parameters. In this way extend the class-conditional reconstruction network standard CNNs.
We can also create a more naive implementation of our strategy by simply computing the reconstruction from the activations in the entire penultimate layer without any masking mechanism. We call this model the "CNN+R" model. In this way we are able to study the effect of conditioning on the predicted class.
CNN+R
4.2 DETECTION THRESHOLD
We find the threshold 0 for detecting adversarial inputs by measuring the reconstruction error between a validation input image and its reconstruction. the distance between the input and the reconstruction is above the chosen threshold 0, we classify the data as adversarial. Choosing the detection threshold 0 involves a trade-ofi off between false positive and false negative detection rates. The optimal threshold depends on the probability of the system being attacked. Such trade-off is discussed by Gilmer et al.) (2018a). In our experiments we don't 't tune this parameter and simply set it as the 95th percentile of validation distances. This means our false positive rate real validation data is 5%.
4

---

Published as a conference paper at ICLR 2020
Table 1: Success Rate Undetected Rate of white-box targeted and untargeted attacks on the MNIST
dataset. In the table. R, is shown for targeted attacks and Su/ R. is presented for untargeted
attacks. A smaller success rate and undetected rate means a stronger defense model. Full results for FashionMNIST and SVHN can be seen in Table [ in the Appendix.
Targeted (%)
Untargeted (%)
Networks
FGSM
BIM
PGD
CW
FGSM
BIM
PGD
CW
CapsNet
3/0
82/0
86/0
99/2
11/0
99/0
99/0
100/19
CNN+CR
16/0
93/0
95/0
89/8
85/0
100/0
100/0
100/28
CNN+R
37/0
100/0
100/0
100/47
64/0
100/0
100/0
100/63
4.3 EVALUATION METRICS
We use Success Rate to measure the success of attacks. For targeted attacks, the success rate S, is
defined as the proportion of inputs which are classified as the target class, S; .V Σ; (f(z²)
4,), while the success rate for untargeted attacks defined as the proportion of inputs which are misclassified, Su
Σ (f(z)) A y²). Previous work (Carlini & Wagner 2017a Hosseini 2019) used the True Positive Rate to measure the proportion adversarial examples that are detected, which alone is insufficient to measure the ability of different detection mechanism because the unsuccessful adversarial examples do not have be detected. Therefore, in this paper, we propose to use the Undetected Rate: the proportion of attacks that are successful and undetected evaluate the detection mechanism. For targeted attacks, the undetected rate is defined as Rg N Σ; (f())) = f2) (d(x*) S 0), where d(-)) computes the reconstruction distance of the input and 0 denotes the detection threshold introduced in Section 44.2, Similarly, the undetected rate for untargeted attacks R., can be defined as R. x ()'((())) # ye) n (d(x4) S 0). The smaller the undetected rate R2 or R., is, the stronger the model is in detecting adversarial examples. The undetected rate can also be used evaluate the attacks (higher is better). We also plot the Undetected Rate vs. False Positive Rate curve to compare the detection performance between different models, where False Positive Rate defined as the proportion of clean examples that are misclassified as the adversarial example by the detection method.
(f(x.)
4.4 TEST MODELS AND DATASETS
In all experiments, all three models (CapsNet, CNN+R, and CNN+CR) have the same number of parameters and were trained with Adam (Kingma & Ba 2014) for the same number of epochs. In general, all models achieved similar test accuracy. We did not do exhaustive hyperparameter search on these models, instead we chose hyperparameters that allowed each model to perform roughly cquivalently on the test sets. We run experiments on three datasets: MNIST (LeCun et al., 1998). FashionMNIST (Xiao al. (017)). and SVHN (Netzer et al., 2011). The test error rate for each model on these three datasets, as well as details of the model architectures, can be seen in Section A and Section B in the Appendix.
EXPERIMENTS
We first demonstrate how reconstruction networks can detect standard white and black-box attacks in addition naturally corrupted images. Then, introduce the "reconstructive attack", which is specifically designed to circumvent our defense and show that it is more powerful attack in this setting. Based on this finding, we qualitatively study the kind of misclassifiaatios caused by the reconstructive attack and argue that they suggest that CapsNets learn features that are better aligned with human perception.
5.1 STANDARD ATTACKS
White Box We present the success and undetected rates for several targeted and untargeted attacks on MNIST (Table I) FaSHionMNIST, and SVHN (Table presented in the Appendix). Our method
5

---

Published as a conference paper at ICLR 2020
Table Error Rate/Undetected Rate on the Corrupted MNIST dataset. A smaller error rate and undetected rate means better defense model.
Gaussian Noise
Gaussian Blur
Dotted Line
Elastic Transform
Corruption
Clean
Line
CapsNet
0.6/0.2
12.1/0.0
10.3/4.1
19.6/0.1 1
4.3/0.0
11.3/0.8
CNN+CR
0.7/0.3
9,8/0.0
6.7/4.2
17.6/0.1
4.2/0.0
11.1/1.1
CNN+R
0.6/0.4
6.7/0.0
8,9/6.4
18.9/0.1
3.1/0.0
12.2/2.1
Corruption
Saturate
JPEG
Quantize
Sheer
Spatter
Rotate
CapsNet
3.5/0.0
0.8/0.4
0.7/0.1
1.6/0.4
1.9/0.2
6.5/2.2
CNN+CR
1.5/0.0
0.8/0.5
0.9/0.1
2.1/0.4
1.8/0.4
6.1/1.6
CNN+R
1.2/0.0
0.7/0.5
0.7/0.2
2.2/0.7
1.8/0.4
6.5/3.4
Corruption
Contrast
Inverse
Canny Edge
Fog
Frost
Zigzag
CapsNet 92.0/0.0
91.0/0.0
21.5/0.0
83.7/0.0
70.6/0.0
16.9/0.0
CNN+CR 72.0/32.6
1/0.0
34.6/0.0
66.0/0.5
37.6/0.0
18.4/0.0
CNN+R
73.4/49.4
88.1/0.0
23.4/0.0
65.6/0.1
36.2/0.0
17.5/0.0
is able to accurately detect many attacks with very low undetected rates. Capsule models almost always have the lowest undetected rates out of our three models. is worth noting that this method performs best with the simplest dataset, MNIST, and that the highest undetected rates are found with the Carlini- Carlini-Wagner attack on the SVHN dataset. This illustrates both the strength of this attack and shortcoming of our defense, namely that our detection mechanism relies on & image distance proxy for visual similarity, and the case of higher dimensional color datasets such as SVHN, this proxy is less meaningful.
Black Box We also tested our detection mechanism results black box attacks. Given the low undetected rates in the white-box settings, it is not surprising that our detection method able to detect black box attacks as well. In fact, the MNIST dataset the capsule model is able to detect all targeted and untargeted PGD attacks. Both the CNN-R and the CNN-CR models are able to detect the black box attacks as well, but with relatively higher undetected rate. A table of these results can be seen in Table 7 in the Appendix.
5.2 CORRUPTION ATTACKS
Recent work has argued that improving the robustness of neural networks Ep norm bounded adversarial attacks should not come at the expense of increasing error rates under distributional shifts that do not affect human classification rates and are likely to be encountered in the "real- world" (Gilmer et al. 8a). For example, if an image is corrupted due to adverse weather, lighting, or occlusion, we might hope that our model can continue to provide reliable predictions or detect the distributional shift. We can test our detection method on its ability to detect these distributional shifts by making use of the Corrupted MNIST dataset (Mu Gilmer (019). This data set contains many visual transformations of MNIST that do not seem to affect human performance, but nevertheless are strongly misclassified by state-of-the-an MNIST models. Our three models can almost always detect these distributional shifts (in all corruptions CapsNets have either small undetected rate undetected rate of O). The error rate (the proportion misclassified input) and undetected rate of three test models the Corrupted MNIST dataset is shown in Table 2 Please refer to Figure 7 and Figure g in the Appendix for visualization of Corrupted MNIST.
5.3 RECONSTRUCTIVE ATTACKS
Thus far we have only evaluated dreviously-defined attacks. Following the suggestion in (Carlini & Wagner 2017a) that detection methods need to show effectiveness towards defense-aware attacks, introduce attack specifically designed to take into account defense mechanism. In order to construct adversarial examples that cannot be detected by the network, we propose a two-stage optimization method to generate "reconstructive attack".

---

Published as a conference paper at ICLR 2020
Table 3: Success rate and the worst case undetected rate of white-box targeted and untargeted reconstructive attacks. Re/R, is shown for targeted attacks and Su/Ra, R., is presented for untargeted attacks. The worst case undetected rate is reported via tuning the hyperparameter in Equ and Eqn[] The best defense models are shown in bold (smaller success rate and undetected rate is better).
All the numbers are shown in %. A full table with more attacks can be seen Table in Appendix.
MNIST
FASHION
SVHN
Targeted R-PGD
Untargeted R-PGD
Targeted R-PGD
Untargeted R-PGD
Targeted R-PGD
Untargeted R-PGD
CapsNet CNN+CR
50.7/33.7
88.1/37.9
53.7/29.8
84.9/75.5
82.0/79.2
98.9/97.5
98.6/68.1 1
99.4/87.7
89.8/84.4
91.5/86.0
99.0/97.9
99.9/99.5
CNN+R
95.5/71.2
95.1/70.5
94.6/88.4
98.9/90.0
99.5/99.3
100.0/99.9
(a) MNIST
(b) Fashion MNIST
(c) SVHN
100
100
100
Undetected Rate (%5
Undetected Rate (%)
Rate (56)
Caprkie
75
50
50
50
25
25
25
0
20
40
60
80
100
20
40
60
80
100
20
40
60
80
100
False Positive Rate (%)
False Positive Rate (96
False Positive Rate (%)
Figure 2: The undetected rate of the white-box targeted defense-aware R-PGI) attack versus the False Positive Rate on the MNIST, Fashion-MNIST and SVHN datasets.
Untargeted Reconstructive Attacks the perturbation based on the gradient of the cross-entropy loss function following standard FGSM attack (Goodfellow et al. 2014), that is:
To construct untargeted reconstructive attacks, we first update
(1) where (ner(f()) y) is the cross-entropy loss function, is the foo bound for our attacks, e hyperparameter controlling the step size in each iteration and 8 is a hyperparameter which balances the importance of the cross-entropy loss and the reconstruction loss (explained further below). In the second stage, focus on constraining the reconstructed image from the newly predicted label to have a small reconstruction distance by updating ó according to
8 + (ó c sign
+ 8).))))
8 f clip, - c 21
(1- - 3) sign( Vign(V (V((I((())+)))) - (x: + 8)1l2)),
where r(0)()))) is the class-conditional reconstruction based on the predicted label fxr + 5) in a CapsNet or CNN+CR network. The used here the optimized ó from the first stage. (]+(()(=+)) (x + d) is the l reconstruction distance between the reconstructed image and the input image. Since the CNN+R network does not use the class conditional reconstruction, we simply use the reconstructed image without the masking mechanism. According to Eqn and Eqn 2 that / balances the importance between the success rate of attacks and the reconstruction distance. This hyperparameter was tuned for each model and each dataset in order to create the strongest attacks. The success rate and undetected rate change as this parameter, which is shown in Figure 9 in Appendix.
we can see
Targeted Reconstructive Attacks We perform a similar two-stage optimization to construct targeted reconstructive attacks, by defining target label and attempting to maximize the clas- sification probability of this label, and minimize the reconstruction error from corresponding capsule. Because the targeted label is given, another way to construct targeted reconstruc- tive attacks to combine these two stages into one stage via minimizing the loss function {= B- (fe 8), y) + (1 B)- (0)-II((uzae (V)(((+))))))))))))))))))))()))) (x + + We implemented both of these targeted reconstructive attacks and found that the two-stage version is a stronger attack. Therefore, all the Reconstructive Attack experiments performed in this paper are based on two-stage optimization.
7

---

Published as a conference paper at ICLR 2020
CapsNet
White-box Defense-aware R-PGD
White-box R-PGD
CapsNet
Undetected Rate (%)
80
Black-box R-PGD
Undetected Rate (%)
80
DeepCaps
CapsNet All
60
60
40
40
20
20
0
0
20
40
60
80 100
20
40
60
80
100
False Positive Rate (%)
False Positive Rate (%)
Figure 3: The defense-avee aware R-PGD attack is tested on the CIFAR-10 dataset with coc 8/255. aefense-awware R-PGD versus the Fasle Positive Rate for clean examples. The test model our CapsNet. Right: The undetected rate of white-box defense-aware R-PGD versus the Fasle Positive Rate for the clean examples. The test model is our CapsNet using consiioandiionall reconstruction, *CapsNet All" using all capsule information, and the DeepCaps (Rajasegaran et al., 2019) using class-independent capsule information.
Left: The undetected rate of seitt--bx/bblak-boor
We build our reconstructive attack based on the standard PGD attack, denoted R-PGD, and test the performance of our detection models against this reconstructive attack in a white-box setting (white-box Reconstructive FGSM and BIM are reported Table in the Appendix). Comparing Table [ and Table 3 we can see that the Reconstructive Attack is significantly less successful at changing the models prediction (lower success rates than the standard attack). However, this attack is more successful at fooling our detection method. For all attacks and datasets the capsule model has the lowest attack success rate and the lowest undetected rate. We report results for black-box R-PGD attacks in Table 7 in the Appendix, which suggest similar conclusions.
In addition, we report the undetected rate of the white-box targeted defense-aware R-PGD attack versus the False Positive Rate on the MNIST, Fashion-MNIST and SVHN datasets in Figure We can clearly see that the undetected rate the defense-aware attack against CapsNet is significantly smaller than the CNN-based networks, which suggests that CapsNets are more robust against adversarial attacks. Furthermore, CNN with condicooniiionall reconstruction (CNN+CR) has smaller undetected rate at the same False Positive Rate compared to the CNN without class-conditional reconstruction (CNN+R), which suggests the class-conditional information is helpful in our models to improve the robustness against adversarial attacks.
5.4 CIFAR-10 DATASET
In order to show that method based on CapsNet is capable to scale up more complex datasets, we test our detection method with a deeper reconstruction network on CIFAR- Krizhevsky 2209)). The classification accuracy on the clean test dataset is 92.2%. In addition, we display the undetected rate of the shite-box/black-bor defense-aware R-PGD attack against CapsNets versus the False Positive Rate in Figure (Left), where can significant drop of the undetected rate of black-box R-PGD compared to the white-box setting. This indicates the CapsNets greatly reduce the attack transferability and the threat of black-box attacks.
Class-conditional Information To investigate the effectiveness of the class-conditional informa- tion in the reconstruction network, we compare our CapsNet based on (Sabour et al.] (017) with the other two variants of CapsNets: "CapsNet All" and "DeepCaps" (Rajasegaran et al., 2019). In "CapsNet All", we remove the masking mechanism in the CapsNet and use all the capsules to do the extract the winning-capsule information
reconstruction. In "DeepCaps", used it as the input for the reconstruction network instead of using masking mechanism to mask out the losing capsules information. In this way, the class information in DeepCaps more explicitly fed into the reconstruction network. As shown in Figure 3 (right), our CapsNet has the best detection performance (the lowest undetected rate at the same False Positive Rate) compared the other two
single vector and
8

---

Published as a conference paper at ICLR 2020
Capsule Medal
Saterthtpp
Trouser
Original class
Pallover
Dress
Sandal
Sneaker
Arice toot
Broset
Steaker
Dig
boot
Tabertop
dreetee
book
bosta
Target class
Target
Target class
Figure 4: This diagram visualizes the adversarial success rates for each source/target pair for targeted R-PGD attacks on Aushion-MNIST with Eoe = 25/ The size of the box at position x, y represents the success rate of adversarially perturbing inputs of class x to be classified as class y. We can see that there is significantly higher variance for the CapsNet model than for the two CNN models.
Source
CapsNet, R-PGD
CNN+CR, R-PGD
CNN+R, R-PGD
CNN+R, PGD
Figure 5: These are randomly sampled (not cherry picked) successful and undetected adversarial attacks created by R-PGD with a target class of for each model on the SVHN dataset(<<>: = 25/255) We can see that for the capsule model, many of the attacks are not "adversarial" as they resemble members of the target class.
Capsule models. "DeepCaps" performs slightly worse than our "CapsNet" and "CapsNet All" has the worst detection performance. Therefore, we conclude that the class-conditional information used in the reconstruction network increases the model's 's robustness to adversarial attack. This also holds true to CNN-based networks because CNN+CR has a better detection performance than CNN+R, shown in Figure 2
VISUAL COHERENCE OF THE RECONSTRUCTIVE ATTACK
The great success of CapsNet over CNN-based models motivates to further diagnose the generated adversarial examples for CapsNets. If our true aim in adversarial robustness research is to create models that make predictions based on reasonable and human-observable features, then we would prefer models that are more likely to misclassify "shirt" as a "t-shirt" (in the case of FashionMNIST) than to misclassify "bag" as a "sweater". For model behave ideally, the success of an adversarial perturbation would be related to the visual similarity between the source and the target class. By visualizing a matrix of adversarial success rates between each pair of classes (shown in Figure 4 we can see that for the capsule model there is great variance between the source and target class pairs and that the success rate of attacks is highly related to the visual similarity of the classes. However, this is not the case for either of the other two CNN- models.
Thus far we have treated all attacks as equal. However, a key component an adversarial example is that it is visually similar to the source image, and that it does not resemble the adversarial target class. The adversarial research community makes use of a small epsilon bound as a mechanism for ensuring that the resultant adversarial attacks are visually unchanged from the source image. For standard attacks against CNN-based models this heuristic is sufficient, because taking gradient steps in the image space in order to have network misclassify an image normally results in something

---

Published as a conference paper at ICLR 2020
visually similar to the source image. But this is not the case for adversarial attacks against CapsNets. As shown in Figure [5] when we use R-PGD to attack the CapsNet, many of the resultant attacks resemble members of the target class. In this way, they stop being "adversarial". As such, attack detection method which does not detect them as adversarial is arguably behaving correctly. This puts the previously undetected rates presented earlier in a new light, and illustrates difficulty in the evaluation of ad adversarial attacks and defenses. In addition, it should be noted that this phenomenon rarely occurs in a standard convolutional neural network, which suggests that the features captured by CapsNet are more aligned with human perception.
7 DISCUSSION
Our detection mechanism relies on similarity metric (i.c. a measure of reconstruction error) between the reconstruction and the input. This metric is required both during training in order to train the reconstruction network and during test time in order to flag adversarial examples. In the four datasets we have evaluated, the distance between examples roughly correlates with semantic similarity, However, this may not be the case for images in more complex datasets such as the SUN dataset (Xiao) et al., 2010) and ImageNet (Deng al. 2009), in which two images may be similar in terms of semantic content but nevertheless have significant &g distance. A better similarity metric (Theis et al.) 2013: Zhang et al., 2018) can be further explored extend our methods to more complex problems. Furthermore our reconstruction network is trained on a hidden representation of one class but is trained to reconstruct the entire input. In datasets without distractors backgrounds, this is not problem. But in the case of ImageNet, in which the object responsible for the classification is not the only object in the image, attempting to reconstruct the entire input from class encoding seems misguided.
8 CONCLUSION
We have presented consiiooniiional reconstruction-basee detection method that does not rely on specific predefined adversarial attack. We have shown that by reconstructing the input from the internal caasiioondiio representation, our system is able to accurately detect black-box and white-box FGSM, BIM, PGD, and CW attacks. We then proposed new attack to beat our defense the Reconstructive Attack in which the adversary optimizes not only the classification loss but also minimizes the reconstruction loss. We showed that this attack was able to fool our detection mechanism but with a much smaller success rate than a standard attack.
Compared to CNN-based models, we showed that the CapsNet was able to detect adversarial examples with greater accuracy on all the datasets we explored. To further explain the success of CapsNet, qualitatively showed that the success of the reconstructive attack was highly related to the visual similarity between the target class and the source class for the CapsNet. In addition, we showed that images generated by this reconstructive attack to attack the CapsNet are not typically adversarial, i.e. many of the resultant attacks resemble members of the target class even with a small Eg norm bound. These are not the case for the CNN-based models. The extensive qualitative studies indicate that the capsule model relies on visual features similar to those used by humans. We believe this is step towards solving the true problem posed by adversarial examples.
REFERENCES
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give false sense of security: Cireumventing defenses to adversarial examples. In International Conference on Machine Learning, 2018
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Nicholas Carlini and David Wagner. Adversarial examples not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14, ACM, 2017a.
10

---

Published as a conference paper at ICLR 2020
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. JEEE, 2017b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In JEEE Conference on Computer Vision and Puttern Recognition, 2009.
Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. Detecting adversarial samples from artifacts, arXiv preprini arXiv: 170..00410, 2017.
Justin Gilmer, Ryan Adams, Ian Goodfellow, David Andersen, and George Dahl. Motivating the rules of the game for adversarial example research. arXiv preprint orXiv:1807.06733 20.
Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S. Schoenholz, Maithra Raghu, Martin Wattenberg. and Ian Goodfellow. Adversarial spheres. In International Conference on Learning Representations, 2018b.
Zhitao Gong, Wenlu Wang, and Wei-Shinn Ku. Adversarial and clean data are not twins. arXiv preprint arXiv: 170400000 2017.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2014.
Ian Goodfellow, Yao Qin, and David Berthelot. Evaluation methodology for attacks against confidence thresholding models. 2018.
Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. On the (statistical) detection of adversarial examples. arXiv preprint 1xXi::7702.06800 2017.
Dan Hendrycks and Kevin Gimpel. Early methods for detecting adversarial images. International Conference on Learning Representations, 2016.
Geoffrey E. Hinton, Sara Sabour, and Nicholas Frosst. Matrix capsules with em routing, In International Conference on Learning Representations, 2018.
Hossein Hosseini, Sreeram Kannan, and Radha Poovendran. Are odds really odd? bypassing statistical detection of adversarial examples, arXiv preprint arXiv:1907.12133 2019.
Saumya Nicholas A. Lord, and Philip H. S. Torr. With friends like these, who needs adversaries? In Advances in Neural Information Processing Systems, 2018.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2014.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In International Conference on Learning Representations, 2016.
Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, et al. Adversarial attacks and defences competition. arXiv preprint IrXiv:1804 1804,00097 2018
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324 1998.
Xin Li and Fuxin Li. Adversarial examples detection in deep networks with convolutional filter statistics. In Proceedings of the IEEE International Conference on Computer Vision, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2017.
11

---

Published as a conference paper at ICLR 2020
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. In International Conference on Learning Representations, 2017.
Felix Michels, Tobias Uelwer, Eric Upschulte, and Stefan Harmeling. On the vulnerability of capsule networks adversarial attacks. arXiv preprint arXiv: 00012. 2019.
Norman Mu and Justin Gilmer. Mnist-e: robustness benchmark for computer vision. In ICML 20/9 Workshop on Uncertainty and Robustness in Deep Learning, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 5, 2011.
Jathushan Rajasegaran, Vinoj Jayasundara, Sandaru Jayasekara, Hirunima Jayasekara, Suranga Seneviratne, and Ranga Rodrigo. Deepcaps: Going deeper with capsule networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10725-10733, 2019.
Kevin Roth, Yannic Kilcher, and Thomas Hofmann. The odds are odd: statistical test for detecting adversarial examples. In International Conference on Machine Learning, 2019.
Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. Dynamic routing between capsules. In Advances in Neural Information Processing Systems. pp. (856-3866, 2017.
Lukas Schott, Jonas Rauber, Wieland Brendel, and Matthias Bethge. Robust perception through analysis by synthesis. arXiv preprint
2018
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Lever- aging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations, 2017.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference Learning Representations, 2013.
Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. In International Conference on Learning Representations. 2015.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1008.07747, 2017.
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3485-3492. IEEE, 2010.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586-595, 2018
12

---

Published as a conference paper at ICLR 2020
APPENDIX
A NETWORK ARCHITECTURES
Figure shows shows the architecute of the capsule network, the CNN reconstruction model and the CNN conditional reconstruction model used for experiments on MNIST, FashionMNIST and SVHN dataset. MNIST and Fashion MNIST have exactly the SVHN. Note that the only difference between the CNN reconstruction (CNN+R) and the CNN conditional reconstruction (CNN+CR) the masking procedure on the input to the reconstruction network based on the predicted class, All three models have the same number of parameters.
architectures while
larger models for
Capsule
FachicMNISST
CNN Model used for MNIST and FishionMNiST
Irpal
Cotv 2 14 12
C 7x7x
Con'
Cone
Resonstrztion
FUB Caroare FAIG DOLDEI
Cansdda Model SVNN
CNN Model used for SV1NN
15 L
Conv
Comv2
10 18 1254
8x
64
Roamea
15
15
CO
Figure 6: The architecture for the CapsNet, CNN+R and CNN+CR model used for our experiments on MNIST (LeC'n et al. 1998). FashionMNIST (Xiao et al.) 2017). and SVHN (Netzer et al. 2011)
B TEST MODELS
The error rate of each model used in the paper are presented in Table We ensure that they have similar performance.
Table 4: Error of each model when the input clean test images in each dataset.
Dataset
CapsNet
CNN+CR
CNN+R
MNIST
0.6%
0.7%
0.6%
FashionMNIST
9.6%
9.5%
9.3%
SVHN
10.7%
9.3%
9.5%
13

---

Published as a conference paper at ICLR 2020
C IMPLEMENTATION DETAILS
For all the Loo based adversarial examples, the fcc norm of the perturbations is bound by c, which is to 0.3, 0.1, 0.1 for MNIST, Fashion MNIST and SVHN dataset respectively following previous work (Madry et al. 2017 Song et al. 2017). In FGSM based attacks, the step size c is 0,05. In based (Kurakin al. 2016) and PGD-based (Madry al. 2017) attacks, the step size is 0.01 for all the datasets and the number of iterations are 1000, 500 and 200 for MNIST, Fashion MNIST and SVHN dataset respectively. We choose sufficiently large number of iterations to ensure attacks has converged.
We use the publicly released code from the authors of (Carlini Wagner 201 to perform the CW attack for our models. The number of iterations are set to 1000 for all three datasets.
D WHITE BOX STANDARD ATTACKS
The results of four white box standard attacks on the three datasets are shown in Table 5
Table Success rate and undetected rate of white-box targeted and untargeted attacks. In the table, S R is shown for targeted attacks and Su/ R,, is presented for untargeted attacks.
Targeted (%)
Untargeted (%)
Networks
FGSM
BIM
PGD
CW
FGSM
BIM
PGD
CW
MNIST Dataset
CapsNet CNN+CR
3/0
82/0
86/0
99/2
11/0
99/0
99/0
100/19
16/0
93/0
95/0
89/8
85/0
100/0
100/0
100/28
CNN+R
37/0
100/0
100/0
100/47
64/0
100/0
100/0
100/63
FASHION MNIST Dataset
CapsNet CNN+CR
7/5
54/9
55/10
100/26
35/29
86/50
87/51
100/68
19/13
89/28
89/28
87/37
74/33
100/25
100/24
100/72
CNN+R
23/16
98/19
98/19
99/81
62/48
100/35
100/34
100/87
SVHN Dataset
CapsNet
22/20
83/45
84/46
100/90
74/67
99/70
99/68
100/94
CNN+CR
24/23
99/90
99/90
99/93
87/82
100/90
100/89
100/90
CNN+R
26/24
100/86
100/86
100/94
88/82
100/92
100/92
100/95
14

---

Published as a conference paper at ICLR 2020
E
VISUALIZATION OF CORRUPTED MNIST DATASET
Visualization of examples from Corrupted MNIST dataset (Mu Gilmer 2019 and the corresponding
reconstructed images for each model are shown in Figure and Figure g
Clean
7 2 0 4 9 9
CapsNet
7 7 2 4 / 4 9
CapsNet
72/64
CNN+R
7 400444440 9
CNN+R
1 72 62/0919559
CNN+CR
1210414994
CNN+CR
72/04/
Gaussian Noise
72 4
Dotted Line
7.2.4444444444
CapsNet
72 7210414AS6
CapsNet
7
CNN+R
7 7 f. 1.4 S
CNN+R
72104/4AC7
CNN+CR
7
z O 4
4 5
CNN+CR
Gaussian Blur
7 2
0
4
4
4
Transform
7 O
4
4
CapsNet
7 2
4
CapsNet
7
2
5
4
9
9
CNN+R
7
2
0
4
1
CNN+R
2
1
4
14
5
CNN+CR
0
A
4
CNN+CR
7
2
1
4 4 69
Saturate
7 4 4 9
Sheer
7
6
4 14959
CapsNet
7 02
CapsNet
7
4 41444499
CNN+R
7 9
CNN+R
7
0
1499
4
CNN+CR
2 04/4969
CNN+CR
7
0 1 14959
JPEG
7 2 0
Spatter
CapsNet
7 2 04/4444
CapsNet
72
0 4
9
5
CNN+R
7 72/0414A69
CNN+R
2
7
04
4
9 6
CNN+CR
72/0414467
CNN+CR
7
44
4 9
5
Quantize
7 7 0
Rotate
72
0 4
8
8
CapsNet
7 2 / 0 4 4
CapsNet
7
2
4
8 5
CNN+R
7 4
9969
CNN+R
2
0 9
CNN+CR
7 7 0
4
4
5999
CNN+CR
2
0 9
3 à S
9
Figure 7: Examples of Corrupted MNIST and the reconstructed image for each model. box represent that this input is flagged adversarial example while green box represent this input has been misclassified and not been detected.
15

---

Published as a conference paper at ICLR 2020
Contrast
7
2
4
9
Fog
7 0
CapsNet
?
?
CapsNet
33 3 1
8
3
9
3
CNN+R
9
CNN+R
CNN+CR
CNN+CR
8 B
Inverse
7
0
4
/ 4
9
5 9
Frost
2
4
CapsNet
2 2
2
3 2
7
CapsNet
3 3
7
CNN+R
3
3 5
CNN+R
7 2 / 2 Q
f
8 5
CNN+CR
4
2
9
5
CNN+CR
E.
R
5
A
8
63
Canny Edge
7 2 / O 4 4 $
Zigzag
77 2 - 6 4 7 4 R
CapsNet
7 7 O 4 9999999
CapsNet
7 4 04 / 9
CNN+R
7 7. 0
CNN+R
7 2 1. 1
CNN+CR
72.4 9 0
CNN+CR
7 721047
0
4 4
Figure 8: Examples of Corrupted MNIST and the reconstructed image for each model. red box represents that this input is flagged adversarial example while green box represents that this input has been misclassified and not been detected.
F RECONSTRUCTIVE ATTACKS
The results of Reconstructive FGSM, BIM and PGD on the three datasets are reported in Table 6
Table 6: Success rate and the worst case undetected rate of white-box targeted and untargeted reconstructive attacks. Below S,/R, R2 is shown for targeted attacks and Su/R,, presented for untargeted attacks,
Targeted (%) R-BIM
Untargeted (%) R-BIM
Networks
R-FGSM
R-PGD
R-FGSM
R-PGD
MNIST Dataset
CapsNet CNN+CR
1.8/0.3
51.0/33.8
50.7/33.7
6.1/1.0
84.5/35.1
88.1/37.9
7.6/0.5
98.0/68.1
98.6/68.1
41.7/3.2
96.5/86.8
99.4/87.7
CNN+R
16.9/3.3
86.3/65.9
95.5/71.2
25.9/8.1
82.9/67.8
95.1/70.5
FASHION MNIST Dataset
CapsNet CNN+CR
6,5/5.8
53.3/28.4
53.7/29.8
33.3/29.9
85.3/75.9
84.9/75.5
17.7/14.0
80.3/72.4
78.1/72.0
68.0/57.3
89.8/84.4
91.5/86.0
CNN+R
19.4/17.6
95.2/88.8
94.6/88.4
58.6/53.5
98.8/90.1
98.9/90.0
SVHN Dataset
CapsNet
21.6/21.2
81.1/78.3
82.0/79.2
71.6/68.3
98.9/97.5
98.9/97.5
CNN+CR
24.2/22.6
98.5/97.6
99.0/97.9
86.0/82.3
99.9/99.5
99.9/99.5
CNN+R
26.6/25.8
99.6/99.4
99.5/99.3
87.1/8
100.0/99.9
100.0/99.9
16

---

Published as a conference paper at ICLR 2020
F.1 HYPERPARAMETER |
Figure 9 shows the plot of success rate and undetected rate versus the hyperparameter 3 which balances the importance between attacking the classifier and fooling the detection mechanism in the targeted reconstructive PGD attacks on the MNIST dataset.
Targeted Reconstructive PGD Attack on the MNIST Dateset
100
CapsNet
CNN+CR
80
Undetected Rate
CNN+R
Success Rate
60
40
CapsNet CN
20
00 02 06
08 10
0.4
0.6
beta (a)
beta (b)
Figure 9: An example shows the plot of the success rate in (a) and undetected rate in (b) of targeted reconstructive PGD attack vesus the hyperparameter beta / for each model on the MNIST test set. We set the max loo norm 0.3 to create the attacks.
G BLACK BOX ATTACKS
Table 7: Success rate and undetected rate of black-box targeted and untargeted attacks. In the table, S R2 shown for targeted attacks and S./Ri Ry is presented for untargeted attacks. All the numbers shown in %.
MNIST Dataset
Targeted
CapsNet
CNN-CR
CNN-R
Untargeted
CapsNet
CNN-CR
CNN-R
PGD
1.5/0.0 4.2/1.0
7.8/0.0
7.4/0.0
PGD
8.5/0.0
32,6/0.0
27.6/0.0
R-PGD
18.3/11.0
11.3/4.8
R-PGD
10.4/2.4
42.7/24.9
25.2/8.9
H VISUALIZATION OF ADVERSARIAL EXAMPLES AND RECONSTRUCTIONS
29
2
3 7 S 3 3
Cleen:
Cler
8 223
22
11816
2
9
3
8
213
2
1816
Cansie
Copaneel
Capateer
PCD:
2
f
RPCD:
OWN
2
2)
11816
1 9 7
8
3
H
Recinnc:
8
223
24
11816
Racau::
1 5 t4
7
Racoes:
Becoms:
2 9
RPOD:
8
2255
22
1866
0 3
Figure 10: The source clean image is presented the first row with its reconstruction the second row. For each model, the top the targeted adversarial examples and the bottom the corresponding reconstruction image when the input are the PGD on the MNIST (left), R-PGD on the Fashion-MNiST (middle), CW the SVHN (right).
17

---

Published as a conference paper at ICLR 2020
source
LP8
41
220
0
1
2
042
3
4
Lf8
5
6
7F7
7
8P7
8
9
Figure 11: These are randomly sampled (not cherry picked) inputs (top row) and the result of adversarially perturbing them with targeted R-PGD against the CapsNet model (other rows). Many of these attacks are not successful. Note the visual similarity between many of the attacks and the target class.
18


Reviews:
[{"review_id": "rkeJjZa0tB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposes to detect adversarial examples or otherwise corrupted images with the reconstruction network, which is used to regularise CapsNets. To further confirm the effectiveness of their detection method, they propose the Reconstructive Attack, which seeks both to cause a misclassification and a low reconstruction error. The comprehensive experiments are conducted.\n\nAlthough the idea is not very novel, the paper makes enough contributions to get accepted. Especially, Section 6 diagnoses the adversarial examples for CapsNets and shows the relationship between the success of the attack and the visual similarity between the source and target class.\n\nWe have the following question for authors about this work:\n\n1. Baseline models: This work creates the baseline model CNN+CR, by dividing the penultimate hidden layer of a CNN into groups corresponding to each class. The sum of each neuron group serves as the logit for that particular class. Why is the sum operation used to create the logit? The sum operation is pretty rare the existing CNN architectures. A linear combination may be a better choice?\n\n2. Detection Threshold: this paper empirically chooses the 95th percentile of validation distances as the threshold to detect adversary samples. Why not report the Area Under Curve (AOC) score? It is a more comprehensive evaluation metric for such a problem.\n\n3. Class-conditional information: This paper follows the architecture in Sabour et al. (2017) where the reconstruction is class-conditional. The work DeepCaps [1] shows that the reconstruction without class-conditional information leads to the better disentanglement of instantiation parameters. Is the class-conditional information necessary to achieve the robustness to adversary attacks?\n[1] Rajasegaran, Jathushan, et al. \"DeepCaps: Going Deeper with Capsule Networks.\" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SJxlSj5VcS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the problem of detecting and generating adversarial images using class-conditional capsule networks. Specifically, this paper first introduced a novel method that detects adversarial examples by class-conditional image reconstruction. Motivated by this defense method, this paper further proposed a novel reconstructive attack that minimizes both classification and reconstruction loss. Experimental evaluations are conducted on MNIST, FashionMNIST, SVHN, and CIFAR-10 dataset. Results demonstrate the effectiveness of the proposed defense and the novel reconstructive attack method.\n\nOverall, this paper is well-motivated and presentation is clear. It proposed a smart way of using the class-conditional generative model to improve the adversarial robustness. Please address the following questions.\n\n(1) Reviewer’s major concern is that this method is not very scalable to large-scale real-world datasets such as ImageNet and SUN database. First, training a class-conditional generative model on MNIST is relatively easy compared to ImageNet. The generative model could potentially create image artifacts on higher resolution images. \n\n-- SUN Database: Large-scale Scene Recognition from Abbey to Zoo. Xiao et al. In CVPR 2010.\n\nSecond, the proposed proxy based on l_2 image distance might not be effective at all for higher-resolution images.\n\n-- A note on the evaluation of generative models, Theis et al. In ICLR 2016.\n-- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, Zhang et al. In CVPR 2018.\n\n(2) Reviewer is not fully convinced by the argument that features learned by CapsNets are superior to features learned by CNN baselines. To draw such conclusion, it is necessary to conduct systematic experiments with different CapsNets and CNNs architectures (e.g., number of layers) and other hyper-parameters related to the adversarial optimization.\n\n(3) It looks like the proposed method is not specific to generative models use class labels as condition. reviewer is curious whether the method generalizes to other conditions (e.g., image-to-image translation) as well.\n\n-- Learning Structured Output Representation using Deep Conditional Generative Models, Sohn et al. In NIPS 2015.\n-- Image-to-Image Translation with Conditional Adversarial Nets, Isola et al. In CVPR 2017.\n-- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, Zhu et al. In ICCV 2017.\n-- Semantic Image Synthesis with Spatially-Adaptive Normalization, Park et al. In CVPR 2019.\n\n(4) Can you possibly comment on the attack transferability compared to other existing attacks evaluated in this paper?\n\n(5) Detection threshold: Can you possibly draw the AOC curve or report the Area Under Curve (AOC) score?\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "ByeNI910FS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed a new defense method for capsule networks. For both white-box and black-box settings, the proposed CapNets has shown superior performance than two variants of CNNs. The visualizations of adversarial examples generated by the CapNets are more aligned with the human perception which is very insightful. On the corrupted MNIST dataset, the results show the proposed defense method can also be used well as an out-of-distribution detector. Overall the paper is clearly written and easy to follow.\n\n\n\nHere I have some concerns:\n\nThe major one is the limited comparisons. For the defense of CNNs, the author only implemented two types of the strategy, which is derived from the characteristics of the capsule network. However, there are also other a lot of defense methods for CNNs which are designed according to the characteristics of themselves. I think the author should also compare the defense performance with those methods to hold the strong claims that CpasNets always perform better than convolutional networks. Also, the experiments performed on Cifar-10 is very limited.\n\n\n\nThe adversarial examples generated by the CapsNets shown in Figure 3 and Figure 11 are indeed changing the number shape which is aligned with human perception. However, some studies for CNNs has also found similar results on MNIST(Towards Deep Learning Models Resistant to Adversarial Attacks) and CIFAR-10(RANDOM MASK: Towards Robust Convolutional Neural Networks). The author should provide more visualizations on other datasets such as CIFAR-10 to support the contribution that the features captured by CapsNets are more aligned with human perception than CNNs.\n\n=========================================================\nAfter Rebuttal:\n\nI thank the author for the response. \n\nI still think the argument that CpasNets always perform better than convolutional networks is an overstatement since you only performed a few defense methods. A milder one is more suitable.\n\nAlso, the high variance of the Capsule Network in Figure 10 can tell something but it is not enough. Could you find similar visible semantic changes in CIFAR-10 dataset as Figure 5?  If yes, you should also list some results. The mentioned CNN works could find similar phenomena on both MNIST and CIFAR-10 dataset. I do not see the strong evidence for the argument that features captured by CapsNets are more aligned with human perception than CNNs.\n\nTo sum up, I think some arguments are overstatements. But this is a good work to analyze the robustness of Capsule Net, I would like to rate 6.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a mechanism for capsule networks to defend against adversarial examples, and a new attack, the reconstruction attack. The differing success of this attacks on capsnets and convnets is used to argue that capsnets find features that are more similar to what humans use.

Reviewers generally like the paper, but took instance with the strength of the claim (about the usefulness of the examples) and argued that the paper might not be as novel as it claims.

Still, this seems like a valuable contribution that should be published.
Decision: Accept (Poster)