Effective and Efficient Vote Attack on Capsule Networks
Authors: Jindong Gu
Baoyuan Wu
Volker Tresp
Venue: ICLR 2021 Poster (2021)
Keywords: Capsule Networks
Adversarial Attacks
Adversarial Example Detection

Abstract: Standard Convolutional Neural Networks (CNNs) can be easily fooled by images with small quasi-imperceptible artificial perturbations. As alternatives to CNNs, the recently proposed Capsule Networks (CapsNets) are shown to be more robust to white-box attack than CNNs under popular attack protocols. Besides, the class-conditional reconstruction part of CapsNets is also used to detect adversarial examples. In this work, we investigate the adversarial robustness of CapsNets, especially how the inner workings of CapsNets change when the output capsules are attacked. The first observation is that adversarial examples misled CapsNets by manipulating the votes from primary capsules. Another observation is the high computational cost, when we directly apply multi-step attack methods designed for CNNs to attack CapsNets, due to the computationally expensive routing mechanism. Motivated by these two observations, we propose a novel vote attack where we attack votes of CapsNets directly. Our vote attack is not only effective, but also efficient by circumventing the routing process. Furthermore, we integrate our vote attack into the detection-aware attack paradigm, which can successfully bypass the class-conditional reconstruction based detection method. Extensive experiments demonstrate the superior attack performance of our vote attack on CapsNets.

Published as a conference paper at ICLR 2021
EFFECTIVE AND EFFICIENT VOTE ATTACK ON CAP- SULE NETWORKS
Jindong Gu Baoyuan Wu?.3, Volker Tresp 1:4
University of Munich, Germany'
The Chinese University of Hong Kong, Shenzhen, China²
Shenzhen Research Institute of Big Data, Shenzhen, China
Corporate Technology, Siemens AG, Munich, Germany"
jindong gu@out com, wubaoyuan& cuhk. edu. en, volker. www.:etseeepeeeess....mmm com
ABSTRACT
Standard Convolutional Neural Networks (CNNs) be easily fooled by images with small quasi-imperces inuereimppec artificial perturbations. As alternatives to CNNs. the recently proposed Capsule Networks (CapsNets) are shown be more ro- bust to white-box attacks than CNNs under popular attack protocols. Besides, the clndiicontiiional reconstruction part of CapsNets is also used to detect adversar- ial examples. In this work, we investigate the adversarial robustness of Caps especially how the inner workings of CapsNets change when the output capsules are attacked. The first observation is that adversarial examples misled CapsNets by manipulating the votes from primary capsules. Another observation the high computational cost, when directly apply multi-step attack methods designed for CNNs to attack CapsNets, due to the computationally expensive routing mech- anism. Motivated by these two observations, we attack votes of CapsNets directly. Our vote attack is not only effective but also efficient by circumventing the routing process. Furthermore, we integrate our vote attack into the detection-aa -wwaee aware attack paradigm, which can successfully bypass the class-conditional reconstruction based detection method. Extensive experiments demonstrate the superior attack performance of our vote attack on CapsNets.
propose novel vote attack where
INTRODUCTION
A hardly perceptible small artificial perturbation can cause Convolutional Neural Networks (CNNs) to misclassify an image. Such vulnerability of CNNs can pose potential threats to security-sensitic applications, e.g., face verification (Sharif et al. 2016) and autonomous driving (Eykholt et al. 2018). Besides, the existence of adversarial images demonstrates that the object recognition process CNNs is dramatically different from that in human brains. Hence, the adversarial examples have received increasing attention since it was introduced (Szegedy et al. 2014 Goodfellow et al. 2015).
Many works show that network architectures play an important role in adversarial robustness (Madry et al. 2018: Su et al. 2018 Xie & Yuille, 2020: Guo et al.) 2020). As alternatives to CNNs, Cap- sule Networks (CapsNetss) have also been explored resist adversarial images since they are more biologically inspired (Sabour et al. 2017). The CapsNet architectures are significantly different from those of CNNs. Under popular attack protocols, CapsNets are shown be more robust to white-box attacks than counter-part CNNs (Hinton et al.) 2018 Hahn et al. (019). Furthermore, the reconstruction part of CapsNets is also applied detect adversarial images (Qin et al. 2020).
In image classifications, CapsNets first extract primary capsules from the pixel intensities and trans- form them to make votes. The votes reach an agreement via an iterative routing process. It is not clear how these components change when CapsNets are attacked. By attacking output capsules di- rectly, the robust accuracy of CapsNets is 17.3%, while is reduced to on the counter- cuuterr-part CNNs in the same setting. Additionally, it is computationally expensive to apply multi-step attacks (e.g., PGD (Madry et al.] 2018)) to CapsNets directly. due to the costly routing mechanism. The two observations motivate propose effective and efficient vote attack on CapsNets.

---

Published as a conference paper at ICLR 2021
Primary Capsules Votes
Clese-Conditionl Reconstruction
Output Capsules
Routing
Figure I: The overview of Capsule Networks: the CapsNet architecture consists of four components, Le., primary capsule extraction, voting, routing, and class-conditional reconstruction.
The contributions our work can be summarised as follows: 1). We investigate the inner working changes of CapsNets when output capsules are attacked; 2). Motivated by the findings, propose an effective and efficient vote attack; 3). We integrate the vote attack in the detection-aware attack bypass class-conditional reconstruction based adversarial detection. The next section introduces background knowledge and related work. Sec. [ and investigate capsule attack and introduce our vote attack, respectively. The last sections show experiments and our conclusions.
BACKGROUND KNOWLEDGE AND RELATED WORK
Capsule Networks The overview of CapsNets is shown in Figure I CapsNets first extract primary capsules u, from the input image x with pure convolutional layers (or CNN backbones). Each primary capsule is then transformed to make votes for high-level capsules. The voting process, also called transformation process, is formulated as
3 W W4
(1)
Next, dynamic routing process is applied to identify weights Cij for the votes م with i E (.. N} corresponding to indices of primary capsules and E f1,2...., M to indices of high-level capsules. Specifically, the routing process iterates over the following three steps
(E++1) exp(b,, + Σí 0f 25)))))
exp(bba + -; 2 ((())))))))))))))))))))))))
where the superscript indicates the index of iterations starting from 1, and g(-) function (Sabour et al., 2017) that maps the length of the vector 8, into the range of [0, 1). The bu the log prior probability. Note that the routing process is the most expensive part of CapsNets.
squashing
The final output capsules are computed as v, = 5 Cij * uyji) where Gj is the output of the last routing iteration. The output capsules represented by vectors, the length of which indicates the confidence of the entitys' existence. In the training phase, the class-conditional reconstruction reconstructs the input image from the capsule corresponding to the ground-truth class i.e., a = r(v,). The reconstruction етror d(z, , . = a 2|l2 works as regularization term. All above notations will be used across this manuscript.
To improve CapsNets (Saoour et al., 2017), various routing mechanisms have been proposed, such (Hinton al., 2018; Zhang et al., 2018 Hahn al. 2019; Tsai al. 2020). The advanced techniques of building CNNs or GNNs have also been integrated into CapsNets successfully. For example, the multi head ireention-based graph pooling is applied replace the routing mechanism
(Gu & Tresp, 2020b). The CNN backbones are applied to extract more accurate primary capsules RRaassgaaran et 2019: Phaye et al., 2018). To understand CapsNets, (Gu & Tresp) 2020a) inves- tigates the contribution of dynamic routing to the input affine transformation robustness. This work focuses on its contribution to the adversarial robustness.
HHnton al. 2018 Hahn et al., 2019) demonstrated the high adversarial robustness of CapsNets. However, it has been shown in (Michels et al.) 2019) that the robustness does not hold for all attacks. In addition, many defense strategies proposed for CNNs
cireumvented by later defense-awe aware white-box attacks (Athalye et al.) 2018). Given the previous research line, we argue that it is nec- essary explore CapsNet architecture-aware attacks, before give any claim on the robustness
2

---

Published as a conference paper at ICLR 2021
of CapsNets. To the best of our knowledge, there is attack specifically designed for CapsNets in current literature.
Adversarial Attacks Given the outputs f(x) of an imput in a CNN, attacks fool the model by creating perturbations to increase the loss [f + 5).y) where C(-) is the standard cross-entropy loss and 6 indicates a Ep-bounded perturbation. The one-step Fast Gradient Sign Method (FGSM
(Goodfellow et al. 2015 ) creates perturbations as
8 = (V + 8).
The multi-step Projected Gradient Descent (PGI) Maadr et al. 2018 is defined
c clip, (5 + a sign( VoL(S (0)[((++))))))))))))))))))))))))
(4)
Other popular multi-step attacks also include Basic Iteractive Method (BIM (Kurakin et al. 2017 Momentum Iterative Method (MIM (Dong et al. 2018)). Besides, C& W attack (Carlini & Wagner 2017b) and Deepfool MMoosvvi-Dezoool et al. constraint.
2016) are popular strong attacks on the 2--morm
Adversarial Detection Besides adversarial attack and defense (Madry et al. 2018 Chen et al. 2020 Li et al., 2020), adversarial detection has also received much attention (Xu et al.) 2017 Ma et al. 2020). Many CNN-based adversarial detection methods were easily bypassed by constructing new loss functions (Carlini Wagner] 2017a). Adversarial images are not easily detected. The most recent work (Qin et al., (2200) leverages the class-conditional reconstruction net of CapsNets to detect adversarial images.
Given any input the predictions and the corresponding capsule are f(x) and V. respectively. The input is flagged as an adversarial image, if the reconstruction error is bigger than a pre-defined threshold [|((p)) - x > 0 where p arg max f(x) is the predicted class. The reconstruction net r(.) reconstructed the input from the capsule Up of the predicted class. The choice 0 involves a trade-off between false positive and false negative detection rates. Instead of tuning this parameter, the work (Qin et al. 2020) simply sets it as the 95th percentile of benign validation distances. A strong detection-aware reconstructive attack also proposed verify the effectiveness of the pro- posed detection method in (Qin et al., 2020). The reconstructive attack is a two-stage optimization method where it first creates a perturbation 8 to fool the prediction as in Equation (5). and updates the perturbation further to reduce the reconstruction error as in Equation (
8 - clip, (6 + a B sign( V&L(f(x - 8),8))), 8 - clip. (6 a. (1 (1-3)
(5)
8) () sign( V |)(U)()) x =12))
(6)
where is the step size, and / is hyper-parameter to balance the losses in the stages.
CAPSULE ATTACK ON CAPSULE NETWORKS
Attack Formulation. In CNNs, under certain constraints, the adversary finds adversarial perturba- tion of an instance by maximizing the classification loss. In CapsNets, the length of output capsules corresponds to the output probability of the classes. Similarly, the adversarial perturbation can be obtained by first mapping the length of output capsules to logits Z(z); = log(I the maximization problem in Equation 7 In this formulation, output capsules are attacked directly, which is called Caps-Attack
(e(lv, (111)))))))) and solving
8'
arg max H(Z(x + 8). = [(soltmax(Z(r + 9)), 8EN,
where NC. {8 ||8|, < {) with 0 being the maximal perturbation. This optimization problem can be naturally solved using the algorithm designed for the attack against CNNs, such as FGSM (see Equation (3) (Goodfellow et al. 2015) and PGD (see Equation
(Madry et al. 2018).
Analysis. In CapsNets, the primary capsule can make a positive or negative vote for the j-th class or abstain from voting. It depends on the relationship between v, and ū The vote from u, for the j-th class is positive if cos ú > otherwise negative cos(u,, (zpr) 0. The similarity value cos (v, ( 0 corresponds abstention of the primary capsules.
3

---

Published as a conference paper at ICLR 2021
a) GT Classes on Clean Images
b) GT Classes under Caps-Attak
<) CT Classes under Vote-Attack
10.0%
length
percentage
0.8
0.3
10.0%
0.8
0.3
10.0%
0.8
0.3
8.0%
coefficients
8.0%
8.0%
6.0%
0.2
6.0%
0.2
6.0%
0.2
0.4
4.0%
4.0%
4.0%
0.2 0.1
0.2 0.1
0.2
0.1
2.0%
2.0%
2.0%
0.
-0.5
0.0 0.0
0.0%
-1.0
-0.5
0
0.5
1.0
0.0 0.0
0.0
-0.5
0.0
Coaire Similarity
Cosire Simiarit
Cosire Siniarit
L-NGT Classes Clean Images
e) L-NGT Classes under Caps-Attack
L-NGT Classes under Vote-Artack
10.0%
0.5 0.3
10.0%
0.6 0.3
10.0%
0.6.0
8.0%
8.0%
8.0%
6.0%
0.4 0.2
6.0%
0.4 0.2
6.0
0.4 0.2
2.0%
2.0%
2.0%
0.0%
0.0 0.0%
-0.5
0.05
Codire Similartty
Cosire Similrity
Cosire Similaity
Figure 2: The left-to-right columns correspond to statistics of predictions on clean images, under Caps-Attack, and under Vote-Attack, respectively. The first corresponds the statistics ground-truth classes, and the second row corresponds to the classes with the largest output proba- bilities that are not ground-truth (L-NGT) classes. In each subplot, the x-axis indicates the cosine similarity value between the vote دز and the output capsule Vj. The blue histogram shows the percentage of votes falling in bins divided by the similarity values in x-axis. The green histogram corresponds to the strength of votes (the averaged length of the votes úzj(). The red curve presents the averaged weight (i.e., Cij. see Equation (2)) of votes at each bin. Please refer to the main context for more in-depth analysis of this figure.
How do the votes change when CapsNets are attacked by adversarial images? We investigate this question with experiments and visualize the results. We firstly train CapsNet with Dynamic Rout- ing (DR-CapsNet) (Sabour et al. 2017) on the CIFAR10 10 dataset (Krizhevsky et al. 2009). With the standard well-trained DR-CapsNet (92.8% test accuracy),
classify all clean images in the test dataset and extract all votes ر and output capsules v, of the ground-truth (GT) classes. We compute (U, }.) in all classifications and split them into 100 equal-width bins in the range of [],, In each bin, we compute the averaged length of all رزرز and average of all coupling coeffi- cients Ctj therein. Note that cý identified by the routing process stands for the weights of the vote dzlt- The results are visualized in Figure h. The majority of primary capsules make positive votes (more votes with positive similarity values blue bins).
To obtain adversarial images, we apply PGD attack to the clean image classifications on the DR- CapsNet where 17.3% robust accuracy is obtained. Similarly, we extract corresponding information from the classifications of adversarial images on the ground-truth class and visualize the results in Figure The votes corresponding to cos (Dj âgjx) are invalid since they have only tiny impact on final prediction. The adversarial images make votes invalid by manipulating the votes and the weights of them. Concretely, the votes on adversarial images are f/ The voting weights identified by the routing process are d'y- Both are manipulated by adversarial images so that the output capsule v' EN- Clj ( is orthogonal most votes 6//1"- Namely, the adversarial images make the majority votes invalid for the ground -trtt class (the concentration of votes around the zero).
To understand how votes change non non-ground-trun classes, also visualize the corresponding information on the classes with the Largest output probabilities that are Not Ground- Truth classes (L-NGT classes) in Figure 2 and 2e We mark differences between the two plots with dashed gray boxes. We can observe that the votes for L-NGT classes become stronger since both the coupling coefficients (the red line) and the strength of their positive votes (the green bins) become larger.
Drawbacks. The above analysis explains why the attack method originally designed for CNNs still works for CapsNets. The first drawback of Caps- Attack is its limited effectiveness. As will
4

---

Published as a conference paper at ICLR 2021
be shown in later experiments, under the same attack method, CapsNets are much more robust than CNNs. Since the routing process is the main difference between CapsNets and CNNs, we attribute the higher robustness of CapsNets to the conjecture that the routing process obfuscates the gradients used to generate adversarial examples. One intuitive way to mitigate it is to approximate the routing process, e-g-, with Backward Pass Differentiable Approximation (BPDA) (Athalye al.) 2018). However, it is non-trivial to approximate the routing process with several routing iterations. The second drawback of Caps-Attack is the low efficiency. The widely used multi-step gradient- based attacks require many times forward and backward passes on the whole CapsNet to generate adversarial examples, e.g., under PGD attack. Caps-Attack are computationally expensive due to the costly iterative routing mechanism of CapsNets.
VOTE ATTACK ON CAPSULE NETWORKS
The above two drawbacks of Caps-Attack inspire that it is necessary to develop adversarial attack methods specifically for CapsNets, rather than directly applying the attack methods designed for CNNs attack CapsNets. In this work, we propose directly attack the votes (see Equation (8) rather than the final output capsules of CapsNets, dubbed Vote-Attack. The behind rationale is that the vote a exactly corresponds
the output class j. though is an intermediate activation of CapsNets. Besides, when the votes from primary capsules are attacked, the corresponding weights (i.e., Gjj, see Equation (2)) identified by the routing process will also be changed. Thus, the attacked votes could mislead the corresponding outputs of CapsNets.
Specifically, given an input-label pair (z. y), the votes from primary capsules are uر = fa where i (1, .,,, N). The average of the N votes is first computed and then squashed wtih the squashing fucntion g()). The vector lengths of the squashed one correspond to output probabilities. Formally, the Vote-Attace on is defined as
N Σ fc ( +))),
8 = arg max (( sEN,
(8)
N
In the formulation above, we first average the votes and squash the averaged vote. There are two intuitive variants of the proposed Vote-attack. The one is to first squash their votes and then average the squashed votes. The other to average the loss caused by all votes. Instead of opimizing on the loss computed the squahed averaged vote, we compute the loss of individual vote seperatedly and average them. More details about these two variants of our Vote-Attack can be found in Appendix A
The maximization problem of Equation (8) can be approximately solved with popular attack method, e.g., PGD attack. When PGD is taken the underlying attack, the proposed Vote-Attack method can reduce the robust accuracy of DR-CapsNets from 17.3% (with Caps-Artack) to 4.83%.
Our Vote-Attack also be extended targeted attack by simply modifying the attack loss function of Equation (8) into 8" arg maxsex. ((log(g() x ΣN fi + 5))). t) where is the target class.
Analysis. We also visualize the votes on the adversarial images created by our Vote-Attack. On the GT classes (see Figure (2), our Vote-Attack increase the negative votes and decrease the positive votes, when compared to Caps-Attack in Figure On the L-NGT classes, the positive votes are strengthened further by our Vote-Attace which leads to more misclassifications See the difference in dashed gray boxes, where both the length of positive votes and the weights become larger (where the similarity values are about 1.0).
Advantages. It is interesting to find that the proposed Vote-Attack could alleviate the drawbacks of CapsNets. Firstly, since the routing process is excluded, Vote-Attack could mitigate the gradient obfuscation when computing the gradient to generate adversarial samples. Hence, the attack perfor- mance of Vote-Attack expected to higher than Caps-Attack. Secondly, since the costly routing process is removed from the attack method, Vote-Attack will be more efficient than Caps-Attack.
5

---

Published as a conference paper at ICLR 2021
5 EXPERIMENTS
In this section, we verify our proposal via empirical experiments. We first show the effectiveness of Vote-Attack on CapsNets in the regular training scheme and the adversarial training one. We also show the efficiency of Vote-Attack. Besides, we apply Vote-Attack to bypass the recently proposed CapsNet-basee based adversarial detection method. All the reported scores are averaged over 5 runs.
5.1 EFFECTIVENESS OF VOTE ATTACK ON CAPSNETS
We take 8 as CNN baseline. In couter-part CapsNets, we apply resnet18 back- bone to extract primary capsules u E (64 x 4 x 4, 8) where the outputs of the backbone are feature maps of the shape (512, 4, 4) and 64 is the number of capsule groups, 8 is the primary capsule size. The primary capsules are transformed to make 64 x 4 x 4 votes & E (64 x 4 x 4, 10, 16) with the learned transformation matrices W E (64 10 are the number of output classes. The votes û reach an agreement E (10, 16) via the dynamic routing meachnism. The length of 10 output capsules are the probabilites of 10 output classes.
Models:
x 4, 8, 160). The size of output capsule
16, and
The popular datasets CIFARIO (Krizhevsky et al. 2009) and SVHN (Netzer et al. 2011 are used in this experiment. The standard preprocess is applied on CIFARIO for training: 4 pixels are padded on an input of 32 x 32, and a 32 x 32 crop is randomly sampled from the padded image or its horizontal flip. For le based attacks, the perturbation range 0.031 (CIFAR10) and 0.047
Datasets:
(SVHN) for pixels ranging in [0. 1]. For la-based attacks, the &g norm of the allowed maximal perturbation is 1.0 for both datasets.
White-Box Attacks We train CNNs and CapsNets with the same standard training scheme where the models are trained with a batch size of 256 for 80 epochs using SGD with an initial learning rate of 0.1 and moment 0.9. The learning rate is set to 0.01 from the 50-th epoch. We apply popular -bo-ee attacks (FGSM (Goodfellow et al.] 2015). BIM (Kurakin et al. [2017). MIM (Dong et al.) P0I8).PGD (Madry et al., 2018)) and ba-ba attacks (C&W W attack (Carlini & Wagner] (017b), Deepfool (Moosavi-Dezlee
et al. (016)) to attack the well-trained models. The syper-parametere mainly follow the Foolbox tool (Rauber et al. (017)). In CapsNets, Capsules and Votes are taken as targets to attack, respectively.
Table 1: The robust accuracy of ResNets and CapsNets are shown under popular attacks on CIFARIO and SVHN datasets. Vote-Attack is much more effective than Caps-Attack and compatible with different underlying attacks.
Model
Target
FGSM
BIM
MIM
PGD
Deepfool-
C&W-2
On CIFARIO Dataset, the model accuracy are ResNet 92. 18 and CapsNet 42.80.
ResNet
Logits
16.6(20.70)
0.15c1005)
Or
Ocio)
0.08
0.24c2010
Caps
44.55
24.43
21.69(=232)
17.3
26.55:+0043)
18.91
CapsNet
Votes
26.21
8.12/1013)
9.20.13.00
4.83:1005)
20.83
2.36)
On SVHN Dataset, the model accuracy are ResNet 94. 94.46 and CapsNet 94. 16
ResNet
Logits
14.57(2279)
2.9(1047)
0.06.2000)
0.06(1900)
3.05r2045)
2. 1)
Caps
58.32(+1.4))
50,25(+088)
c0.09(=
34.82
45.76
c..m
CapsNet
Votes
49.16
31.46.4020
14.22
8.11
39.3
27.94
The standard test accuracy and the robust accuracy under different attacks are reported in Table The CapsNets and the counter-part CNNs achieve similar performance on normal test data. The strong attack PGD can mislead all the classifications of ResNet. However, is less effective to attack output capsules. Our Vote-Attack different attack methods. We also check the lo, f l2 norms of the perturbations created by different attack methods in Appendix B In most cases, the different norms of perturbations corresponding to Vote-attack is similar to the ones to Caps-attack.
reduce the robust accuracy of CapsNets significantly across
We also verify the effectiveness of Aote-Attack from other perspectives, such as, the targeted attacks, the transfcrability of adversarial examples and the adversarial robustness on trffne-rranfoormed in- puts. The experimental details of the targeted Vote Attack are in the Appendix C The transferability

---

Published as a conference paper at ICLR 2021
of the created adversarial examples is investigated in Appendix D The adversarial examples created by Vote-attack are more transferable than the ones by Caps-attack.
CapsNets are shown to be robust to input affine transformation (Sabour et al. 2017 Gu & Tresp 2020a). When inputs are affine transformed, the votes in CapsNets also change correspondingly We also verify the effectiveness of Vote-Attack in case of affine transformed inputs. We consider two cases: 1) The CapsNet built on standard convolutional layers (Sabour et al.) (017) is trained on MNIST dataset and tested on AfNNIST dataset. 2) The CapsNet built on a backbone (i.e. ResNet18) is trained on the standard CIFARIO training dataset and tested on affine-tranfformeed CIFARIO test images. In both cases, our Vote- Mote-Attack achieves higher attack success rates than Caps-Attack Attack. More details about this experiment can be found in Appendix E This experiment shows that our Vote- Attack is more effective than Caps-Attack when the inputs are affinettransfommed
Under Vote-Attack, the robust accuracy of CapsNets is still higher than that of counter-part CNNs. However, we did claim CapsNets are more robust for two reasons. 1) CapsNets possess more net- work parameters due to transformation matrices. 2) The potential attacks can reduce the robust accuracy further. This study demonstrates that the high adversarial robustness of CapsNets can be a fake sense, and we should be careful to draw any conclusion about the robustness of CapsNets.
Adversarial Training In this experiment, we verify the effectiveness of Vote-Attack in the context of Adversarial Training. We train models with adversarial examples created by Caps-Attack where PGD with 8 iterations is used. For training a more robust model, we also combine Vote-Attack and loss from the two attacks is used.
Caps-Attack to create adversarial examples where
The underlying attack method used in this experiment is PGD with 40 iterations. The model perfor- mance is reported in Table under different training schemes. We can observe that the Vote-Attach (corresponding to the last column) is more effective than Caps-Autack (corresponding to the second last column) under adversarial training. When include Vote-Attack improve the adversarial training (AT v.s. AT + Votes), the robust accuracy of CapsNets is increased under both Caps-Attack and Vote-Attack.
The Vote-Attack only attacks part of the model. During adversarial training, the model can adapt the routing process to cireumvent the adversarial perturbations. Therefore, not effective to do adversarial training only using Vote-Attack.
The robustness of CapsNets with different training schemes on CIFARIO and SVHN datasets: Vote-Attack is also effective to attack models with adversarial training; It can also be applied to improve adversarial training.
Table 2:
ResNet
CapsNet Caps
Dataset
Traning
Asta
Logits
Astd
Votes
Natural
92. 92
92.8(29.14)
17.30k
4.83
CIFAR1O
AT
79.45(2127) 43.91(2062)
75.0(2004)
45.49
43.65(20.85
AT + Votes
76.42.+03m
49.62c=030)
44. 14.1224-032
Natural
94,46(2014 87.92200)
0.06
94. 12
34,82(22.11)
1(20,00)
SVHN
AT
36.05(2033
86. 86.0(20.0)
33. 13.40(21.36))
30.44(:100)
AT + Votes
83.89(4073)
39.13r
34.92
5.2 EFFICIENCY OF VOTE ATTACK ON CAPSNETS
In the last subsection, we demonstrate the effectiveness of Vote-Attack from different perspectives. We now show the efficiency of Vote-Attack. In our Vote-Attack, no routing process is involved in both forward inferences and gradient backpropagations. To show the efficiency of Vote-Attack empirically, we record the time required by each attack to create a single adversarial example and average them across the CIFAR10 test dataset. A single Nvidia V100 GPU is used.
The required time is reported in Table 3 The time on SVHN dataset is almost the same as in CI- FAR10 since both input space dimensions the same (i.e., 32. 32, 3). The column corresponding to Astd shows the time required to classify a single input image. Compared to the logit attack in CNNs, Caps-Attack in CapsNets requires more time to create adversarial examples since the dy-
7

---

Published as a conference paper at ICLR 2021
namic routing is computationally expensive. Our using the routing part, and reduce the required time significantly. However, the required time is still more than that CNNs. The reason behind this is that the current deep learning framework is highly optimized on the convolutional operations, less on the voting process.
create adversarial images without
Table 3: The averaged time required by each attack to create an adversarial example is reported on CIFARIO test dataset. Vote-Attack requiring less time is more efficient than Caps-Attacks.
Model
Target
Aai
FGSM
BIM
MIM
PGD
Deepfool
C&W
ResNet
Logits
4.14ms
12. 1 3ms
83.34ms
165.76ms
324.53ms
186.77ms
409.38ms
Caps
17.45ms
120.75ms
242.97ms
471.81ma
607.84m
612.79ms
CapsNet
5.65ms
Votes
14.89ms
105.09ms
196.11ms
414.58ms
295.28ms
448.31ms
5.3 BYPASSING CLASS-CONDITIONAL CAPSULE RECONSTRUCTION BASED DETECTION
In this experiment, we demonstrate that class-conditional capsule reconstruction based detection be bypassed by integrating our Vote-Attack in the detection-aware attack method. Following the work (Qin et al., 2020)) we use the original CpasNet architecture (Sabour et al. 2017) for this experiment. The architecture details are shown as follows.
CapsNets, two standard convolutional layers, Conv (C226,, K9, S1), Conv2(C226,, K9, S2), are used extract primary capsules of shape (32x6x6, 8). The output capsule of shape (10, 16) can be obtained after the dynamic routing process. The output capsules will be taken as input for reconstruction net with (FC160-FC512 FC122 of the output capsules is activated, others are masked with zeros. Since the input contains the class information, the reconstruction is class-conditional. The capsules corresponding to the ground-truth class will be activated during training, while the winning capsule (the one with maximal length) will be activated in the test phase.
In the reconstruction process, only one
Two CNN baseline models are considered. CNN+CR uses the same architecture without routing and group 160 activations into 10 groups where the sum of 16 activations is taken as a logit. The same class-conditional reconstruction mechanism is used. CNN+R does not group 160 activations and reconstructs the input from activations without a masking mechanism. More details of the baseline models can be found in (Qin et al. 2020).
Given an input, it will be flagged as adversarial examples if its reconstruction error bigger than given threshold d(æ, 2) O. Following (Qin et al. 2020),
set 0 as 95th percentile of recon- struction errors of benign validation images, namely, 5% False postive rate. We report Success Rate
S = K Σ (f + 5) * y) and Undetected Rate R = R Σ: (f(x + 8) * y) n (d(xx .) < 0).
Both detection-agnnstic and detection-aware attacks introduced in Sec. are considered.
Table 4: Different attacks are applied to circumvent the caass-conditional reconstruction adversarial detection method FMNIST dataset. The attack success rate and undetected rate (S/R) are re- ported for each attack. The integration of Vote-Attack in the detection-aware attack increases both the attack success rate and the undetected rate significantly.
Attacks
Model
Target
Aata
FGSM
BIM
PGD
C&W
CNN+R
Logits
90,95
85,8/63.3
100/80.0
100/75.7
86,4/68.8
Detection-agnoste Attack
CNN+CR
Logits
91.79
89.4/66.4
97.4/70.4
97.9/67.9
77.3/77.1
Caps
40.2/29.3
88.8/53.1
90.6/51.4
70.7/54.1
CapsNet
91.85
Votes
74.8/46.1
94.6/59.2
94.7/55.3
90.5/50.1
CNN+R
Logits
90.95
85,3/77.3
99.7/95.0
100/92.1
Detection-aware
CNN+CR
Logits
91.79
89,3/75.9
96.3/82.3
96.2/81.2
Attack
Caps
41.8/37.2
87.977877
89.7778.2
CapsNet
91.85
Votes
76.8/66.5
95.1/85.2
95.6/86.1
The results on FMNIST dataset are reported in Table 4 In aetection-agnostic attacks, we apply our Vote-Attack to attack CapsNets directly without considering the detection mechanism. The CapsNet
8

---

Published as a conference paper at ICLR 2021
Clean
6
6
5
Caps-Attack
6
5
2
Vote-Attack
Figure 3: This figure shows the clean images and the corresponding adversarial images created by Caps-Attack and Vote-Attack in a targeted setting. The attack target class is set to the digit O. The adversarial images created by the two attack methods are visually similar. The observation also echoes the previous findings in Appendix [ where we show that the perturbations created by Caps-Attack and Vote-Attack have similar norms.
used in this experiment is built on standard convolutional layers instead of backbones in previous experiments. Our Vote-Attack still achieve higher success rate than Caps-Attack, It indicates that the Vote-Attack is effective across different architectures. Furthermore, the undetected rate is also increased correspondingly. In detection-aware attacks, the integration of our Vote-Attack increases the attack success rate and undetected rate significantly. More results on MNIST and SVHN datasets are shown Appendix F
Under the class-conditional capsule reconstruction based detection, some of the undetected exam- ples are not imperceptible anymore, as shown in (Qin et al. 2020). Some images are flipped into the attack target classes when attacked, although a small perturbation threshold is applied. Some images are hard flip, e.g., the ones with a big digit or thin strokes. We also visualize the adver- sarial examples created by Caps-Attack and our Vote-Attack in Figure 3 More figures and details are shown Appendix G We find that there no obvious visual difference between the adversarial examples created by the two attacks. This finding echoes a previous experiment, where we compute the different norms (i.c., the Eg, E1, &g norms) of the created perturbations. The perturbations have similar norms (see Appendix B Hence, the adversarial examples created by the two attacks visually similar.
6 CONCLUSIONS AND FUTURE WORK
We dive into the inner working of CapsNets and show how it is affected by adversarial examples. Our investigation reveals that adversarial examples can mislead CapsNets by manipulating the votes. Based on the investigation analysis, we propose an effective and efficient Vote-Attach to attack Cap- sNets. The Vote-Attack is more effective and efficient than Caps-Attach in both standard training and adversarial training settings. Furthermore, Vote-Attack also demonstrates the superiority in terms of the transferability of adversarial examples well the adversarial robustness tfafne-transformedd data. Last but not least, we apply our Vote-Attack to increase the undetected rate significantly of the class-conditional capsule reconstruction based adversarial detection.
The idea of attacking votes of CapsNet can also be applied to different versions of CapsNets. How- ever, some adaptions required since different CapsNet versions can have significantly different architectures. For instance, in EM-CapsNet (Hinton et al., 2018). capsule corresponding to an entity are represented by a matrix, and the confidence of the entity's existence is represented by the activation of a single neuron. The possible adaption could be attacking votes by flipping the neuron activations that represents the existence of entities. Recently, many capsule networks have been pro- posed, to name few (Hinton et al.) 2018 Zhang et al. 2018: Rawlinson et al.) 2018 Hahn et al. 2019 Ahmed Torresani] 20199 Gu & Tresp 2020a] Tsai et al. 2020) Ribeiro et al. 2020). We
leave the further exploration on different versions of CapsNet in future work.
Even though CapsNets still seem to be more robust than counter-part CNNs under our stronger Vote-Attack, it is too early to draw such conclusion. We conjecture that the robust accuracy of CapsNets can be reduced further. future work, we will explore more strong attacks as well as the certifications compare the robustness of CNNs and CapsNets.

---

Published as a conference paper at ICLR 2021
REFERENCES
Karim Ahmed and Lorenzo Torresani. Star-caps: Capsule networks with straight-through attentive routing. In Advances in Neural Information Processing Systems, pp. 9101-9110, 2019.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML), 2018.
Nicholas Carlini and David Wagner. Adversarial examples detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14, 2017a.
easily detected: Bypassing ten
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), Pp. 39-57. IEEE, 2017b.
Weilun Chen, Zhaoxiang Zhang, Xiaolin Hu, and Baoyuan Wu. Boosting decision-bast black- box adversarial attacks with random sign flip. In European Conference on Computer Vision, pp. Springer, 2020.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 9185-9193, 2018
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625-1634, 2018
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International conference learning represenlations (ICLR), 2015.
Jindong Gu and Volker Tresp. Improving the robustness of capsule networks to image affine trans- formations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 7285-7293, 2020a.
Jindong Gu and Volker Tresp. Interpretable graph capsule networks for object recognition. 2020b. To appear in AAAI 2021.
Minghao Guo, Yuzhe Yang, Rui Xu, and Ziwei Liu. When nas meets robustness: In search robust architectures against adversarial attacks. In The IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2020.
Taeyoung Hahn, Myeongjang Pyeon, and Gunhee Kim. Self-routing capsule networks. In Advances in Neural Information Processing Systems (NeurlPS), pp- 7658-7667, 2019.
Geoffrey E Hinton, Sara Sabour, and Nicholas Frosst. Matrix capsules with em routing. In Interna- tional conference on learning representations (ICLR), 2018
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In International Conference Learning Representations (ICLR), 2017.
Yiming Li, Baoyuan Wu, Yan Feng, Yanbo Fan, Yong Jiang, Zhifeng Li. and Shutao Xia. Toward adversarial robustness via semi-supervised robust training. arXiv preprint TBt::2003.06974 2020.
Chengeheng Ma, Weiliang Meng, Baoyuan Wu, Shibiao Xu, and Xiaopeng Zhang. Efficient joint gradient based attack against sor defense for 3d point cloud classification. In Proceedings of the 28th ACM International Conference Multimedia, pp. 1819-1827, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International conference on learning representations (ICLR), 2018.
10

---

Published as a conference paper at ICLR 2021
Felix Michels, Tobias Uelwer, Eric Upschulte, and Stefan Harmeling, On the vulnerability of capsule networks to adversarial attacks. 2019.
Seyed-Mohsen Moosavi-Dezfooli. Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 2574-2582, 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Sai Samarth R Phaye, Apoorva Sikka, Abhinav Dhall, and Deepti R Bathula. Multi-level dense capsule networks. In Asian Conference on Computer Vision, pp. 577-592. Springer, 2018.
Yao Qin, Nicholas Frosst, Sara Sabour, Colin Raffel, Garrison Cottrell, and Geoffrey Hinton. De- tecting and diagnosing adversarial images with classcooniitional capsule reconstructions. In In- ternational Conference on Learning Representations (ICLR), 2020.
Jathushan Rajasegaran, Vinoj Jayasundara, Sandaru Jayasekara, Hirunima Jayasekara, Suranga Seneviratne, and Ranga Rodrigo. Deepcaps: Going deeper with capsule networks. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10725-10733, 2019.
Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: python toolbox to benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop, 34th International Conference on Machine Learning, 2017. URL http:://aaatt org/abs/
7707.01131
David Rawlinson, Abdelrahman Ahmed, and Gideon Kowadlo. Sparse unsupervised capsules gen- eralize better. arXiv preprint
2018.
Fabio De Sousa Ribeiro, Georgios Leontidis, and Stefanos Kollias. Capsule routing via variational bayes. In 34-th Association for the Advancement of Artificial Intelligence, 2020.
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In Advances in neural information processing systems (NeurIPS), pp. 3856-3866, 2017.
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security, pp. 1528-1540, 2016.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR), 2015.
Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin- Yu Chen, and Yupeng Gao. robustness the cost of accuracy? a comprehensive study on the robustness of 18 deep image classification models. In European Conference on Computer Vision (ECCV), 2018
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International conference on learning representations (ICLR), 2014.
Yao-Hung Hubert Tsai, Nitish Srivastava, Hanlin Goh, and Ruslan Salakhutdinov. Capsules with inverted dot-product attention routing. In International Conference Leurning Representations (ICLR), 2020.
Cihang Xie and Alan Loddon Yuille. Intriguing properties of adversarial training at scale. In Inter- national conference learning representations (ICLR), 2020.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv: 155, 2017.
Liheng Zhang, Marzich Edraki, and Guo-Jun Qi. Cappronet: Deep feature learning via orthogo-
nal projections onto capsule subspaces. In Advances in Neural Information Processing Systems (NeurIPS), pp. 5814-5823, 2018.
11

---

Published as a conference paper at ICLR 2021
A TWo VARIANTS OF VOTE ATTACK
We have another two choices when attacking votes in CapsNet directly, Choices I: In Equation (8) we first average the votes and squash the averaged vote. Another choice is to first squash their votes and then average the squashed votes. Our experiments show that this option is similarly effective.
1 Σ gl + +))))
8" = arg max H(log( dev
Choices 2: Another choice is average the loss caused by all votes. Instead of opimizing on the loss computed on the squahed averaged vote, we can compute the loss of individual vote seperatedly and average them, namely.
1
8 arg max
Σ ( SEV N
(10)
The loss of each vote can differ from each other significantly. The large part of loss be caused by a small part of votes. In other words, the gradients of received by the input can be caused mainly by few too strong votes. This choice is less effective, compared to the one in Equation (8)
We use the same emperimental setting as in Sec. 5 Under the same PGD attack on CIFARIO dataset, the robust accuracy corresponding the choice 1 is 1.06, and is effective, similar to Equation (8) The choice 2 with the robust accuracy 43.31 does not work well since the gradients received by inputs are dominated only by a small part of votes.
B NORMS OF PERTURBATIONS CREATED BY DIFFERENT ATTACKS
On CIFARIO and SVHN datasets, we compute the different norms of perturbations created by dif- ferent attacks. On each dataset, we first select the examples that are successfully attacked by both Vote-Attack and Caps-Attack on CapsNets well as the corresponding attack ResNets from the test dataset. Then, we obtain the created perturbations created by the corresponding attacks. The to l and l2 norm of perturbations are shown in Table 5 on CIFARIO dataset and Table 6 on SVHN dataset.
In most cases, Vote-Attack and Caps-Attack create perturbations with similar norms. Under BIM attack, we can observe that l and l norms corresponding to Vote-Attack are higher than the ones Caps-Atrack. Both are smaller than the ones corresponding to other multi-step attacks (e.g., PGD). The reason behind this is that the BIM attack does not converge since only 10 iterations are used by default in FoolBox tool (50 iterations in PGD). Given the same iterations before the convergence, Vote-Attack accumulates the relatively consistent gradients. Vote-Attack converges faster than Caps-Attack, which explains our observation.
In addition, the l attack find the minimal perturbations to misled the classifier. The different norms of perturbations is small. Our Vote-Attack finds samller perturbations SVHN dataset and similar ones in CIFARIO 10 dataset. This obervation indicate that the performance of attack method can also depend on the datasets.
C VOTE TARGETED ATTACK
We create adversarial examples in targeted attack settings CIFARIO and SVHN datasets. The used models are the same as in the untargeted setting. The target classes are selected uniformly at random from the non-ground-truth classes. The attack is successful if the created adversarial examples are classified as the corresponding target classes by the underlying classifier.
The attack success rate (%) is reported in Table 7 In the targeted attack setting, Vote-Attack achieves a significantly higher attack success rate than Caps-Attack. This exeriment show that our Vote-Artack is still effective when extended the targeted attack setting.
12

---

Published as a conference paper at ICLR 2021
Table 5: The lo. E,, and & norms of perturbations created by different attacks are shown on CI- FAR10 dataset. Overall, the perturbations created by our Vote-Attack have similar norms to the ones by Caps-Attack.
Model
Target
FGSM
BIM
MIM
PGD
Deepfool-22
C&W-&2
ResNet
Logits
3054.7
2797.1
3071.9
3057.4
1533.5
2942.2
6 norm
Caps
3054.5
2489.2
3071.6
3066.3
1431.7
2977.4
CapsNet
Votes
3054.6
2741.1
2523.9
3065.7
1534.8
2978.1
ResNet
Logits
93.96
53.07
77.89
77.38
0.21
0.51
é norm
Caps Votes
93.93
28.79
78.86
53.36
0.32
0.42
CapsNet
93.91
43.68
78.71
54.03
0.32
0.51
ResNet
Logits
1.7041
1.1089
1.4974
1.4849
0.0059
0.0105
E norm
Caps
1.7037
0.6471
1,5066
1.1035
0,0092
0.0087
CapsNet
Votes
1.7035
0.6753
1.5047
1.1155
0.0091
0.0104
Table The lo. l,, and & norms of perturbationn created by different attacks are shown on SVHN dataset. In -ttack methods, the perturbations created by our Vote-Attack have similar norms to the ones by Caps-Attack, In lz-attack methods, our Vote-attack can find smaller perturbations to fool the underlying classifier.
Model
Target
FGSM
BIM
MIM
PGD
Deepfool-E2
C&W-2
ResNet
Logits
3066.9
2854.1
3071.9
3066.0
1754.4
2972.7
t norm
Caps
3067.4
2552.2
3071.8
3070.3
2103.4
2931.1
CapsNet
Votes
3067.2
2587.6
3071.8
3069.7
875.0
2924.9
ResNet
Logits
94.83
59.18
80.75
111.37
0.65
& norm
Caps
94.88
32.99
77.77
79.37
0.52
0.87
CapsNet
Votes
94.86
35.06
78.00
80.33
0.24
0.15
ResNet
Logits
1.7136
1.2041
1,5357
2.1657
0.0141
0.0149
le norm
Caps
1.7142
0.7283
1.4920
1.6464
0.0161
0.0172
CapsNet
Votes
1.7140
0.7677
1,4954
1.6442
0.0074
0.0029
D TRANSFERABILITY OF ADVERSARIAL EXAMPLES
We also investigate the transferability of adversarial examples created by Caps-Attack and Vote- Attack CIFAR10 dataset. We consider three models, VGG19 (Simonyan Zisserman 2015). ResNet18 and CapsNets. The PGD is used as the underlying attack. We measure the transferability using Transfer Sucess Rate (TSR).
The TSR of different adversarial examples is reported in Table The adversarial examples created on CNNs are more transferable. Especially, the ones created ResNet18 can be transferred to CapsNets very well. The reason behind this is that CapsNets also the ResNet18 bone to extract primary capsules. By comparing the last two columns in Table we can observe that the adversarial example created by Vote-Attack is more transferable than the ones created by Caps-Attack.
E ADVERSARIAL ROBUSTNESS ON AFFINETTRANSFORMED DATA
CapsNets learn equivariant visual representations. When inputs are affine transformed, the votes also changes correspondingly. this experiment, we aim to verify the effectiveness of Vote-Attack when inputs and their votes in Capsnets changed. The model is trained the same before. We translate the test images with pixels randomly and rotate the images within a given pre-defined degree.
13

---

Published as a conference paper at ICLR 2021
Table 7: The targeted attack success rates (%) are shown on CIFARIO and SVHN datasets, In
the targeted attack setting, our Vote-Attack is significantly more effective than Caps-Attack when combined with popular attacks.
Model
Target
FGSM
BIM
MIM
PGD
Deepfool-E2
C&W-&
On CIEARIO Dataset, the model accuracy are ResNet 92. 88:5 and CapsNet 42.80
ResNet
Logits
39.13r43.11
98.4 98.47(4008)
99.7 99.71c=0.3
99. 39.97
10.47:+0.11)
97.99(=1.89)
Caps
9.58(20.10)
27.91
48.38
(5.94
9.43(2048)
34.07(=138)
CapsNet
Votes
10.67(10.3))
32.66,1209
61.08.4771
75.35(100n)
9.55
41.41(25885
On SVHN Dataset, the model accuracy are ResNet 94 4014 and CapsNet 44.16.
ResNet
Logits
43.0612337)
91.72(1042))
98. 05
99.78(2004)
11.84(2045)
93.97(=083)
Caps
(.82
38.58:40.39)
49.04 0.89)
68,94
6.82(=1.12)
44.64c
CapsNet
Votes
(.27(2
48.25 o2)
(5.35
91.68
(.57
62.93:20.55)
Table 8: The transferability of adversarial examples created on CNNs and CapsNets on CIFARIO dataset: the ones created on CNNs are more transferable than on CapsNets; the ones created with Vote-Attack are more transferable than the ones with Caps-Attack.
Attacks on Source Model
VGG19 (Logits)
ResNet18 (Logits)
CapsNet (Caps)
CapsNet ( (Votes)
Target Models
VGG19
83.79(20.18)
93.94(1022))
35.64(1096)
41.491:0.19)
ResNet18
97.26r1180
37.59(1625)
43.45c22.13)
CapsNet
80.38(
97.53(2057)
46.43(4566)
55.34126626)
The robust accuracy of affinettaassfrmmdd images is shown in Table 0 on CIFAR10 dataset. Un- der different rotation degrees, our Vote-Attack still effective. It consistently reduces the robust accuracy of CapsNets, when compared to Caps-Attack.
Table 9:
When inputs are affine-trame transformed in CIFAR1O dataset, the Vote-Attack is still more effective to create adversarial examples than Caps-Attack.
Model
Target
(0, =100)
(422,
+15
(1.2, +.30%)
(4:2, 160")
(4:2, +.00)
Asta
92.18(2057)
85.64:1040
68. (8.1112)11))
48.47
42.07(2022)
ResNet
Logits
0
0
0
Asid
92.8.10.4)
86.09.1039)
1.944
49.37:42449
42.62(#1.60)
CapsNet
Caps
17.3
1.82(+18))
2.89)=1005)
1.63
1.11(_0.38)
Votes
4.83
(.15
0.54
0.32
0.23
We also conduct experiments AffNIST dataset. In this experiment, the original CapsNet archi- tecture and the original CNN baseline in (Sabour et al., 2017) are used. The modes are trained on standard MNIST dataset and tested AiINIST dataset. In AfINIST dataset, the MNIST images are transformed, namely, rotated, translated, scaled, or sheared. More details about this dataset are in this resource The perturbation threshold and the attack step size are set to 0.3 and 0.01, respectively. The other hyper-paue parameters are defaults in the Foolbox tool (Rauber et al. 2017).
The test accuracy on the untransformed test dataset (Asta), the accuracy on the transformed dataset (Aaff) and the robust accuracy under different attacks are reported in Table TO Our Vote-Attack achieve higher attack success rates than Caps-Attack.
http:::/www.cs..ooooooooooo http://wwwww....ooooooooo/////// tijmen/afiNISTY
14

---

Published as a conference paper at ICLR 2021
Table 10: The accuracy on the dataset with untransformed images and the one on the dataset with transformed images are reported (in %). CapsNet achieves better transformtation robustness than the original CNN baseline. The robust accuracy of different models are also reported under different attacks. We can observe that it is more effective to attack Votes instead of output capsules CapsNet.
Model
Target
Astd
Aaff
FGSM
BIM
MIM
PGD
ResNet
Logits
99.22
66.08
10.18
0
0
Caps Votes
15.61
4.27
1.01
0.48
CapsNet
99.22
79.12
10.43
1.33
0
0
F BYPASSING CLASS-CONDITIONAL RECONSTRUCTION ON MNIST, FMNIST AND SVHN
The integration of our Vote-attack into detection-aware attack effective to bypass the class- conditional reconstruction detection method. To verify this, also conduct experiments on dif- ferent datasets, such as MNIST and SVHN. The results are reported in Table [1] On the All three datasets, both detection-aware and detection-agnostic attacks achieve high attack success rate and undetected rate, when combined with our Vote-attack.
Table 11: Different attacks are applied to circumvent the class- class-conditional reconstruction adversarial detection method. The attack success rate and undetected rate (S/ R) are reported for each attack. On all the three popular datasets, the integration of Vote-Attack in the detection-aware attack increases both the attack success rate and the undetected rate significantly.
DataSet
Model
Asta
Attacks
Target
FGSM
BIM
PGD
Caps
13.3/6.3
73.3/31.7
77.9/33.
Detection-agnostic
Votes
38.8/15.1
92.3/35.4
93.4/34.1
MNIST
CapsNet
99.41
Caps
16.1/13.6
71.7/5.7
77.2/57.8
Detection-aweer aware
Votes
44.8/34.9
92.1/66.8
93.4/67.1
Caps
40.2/29.3
88.8/53.1
90.6/51.4
Detection-agnon
Votes
74.8/46.1
94.6/59.2
94.7/55.3
FMNIST
CapsNet
91.85
Caps
41.8/37.2
87.9778.7
89.7778.2
Detection-awe aware
Votes
76.8/66.5
95.1/85.2
95.6/86.1
Caps
83.2/78.1
99.1/92.5
99.6/92.2
Detection-agnostic
Votes
95.5/88.8
99.9/93.2
99.9/93.3
SVHN
CapsNet
91.32
Caps
84,2/80.
97.8/95
97.8/94.7
Detection-awart
Votes
90.6/90.8
100/96.7
100/96.8
G VISUALIZING UNDETECTED ADVERSARIAL EXAMPLES
We also visualize the adversarial examples created by Caps-Attack and Vote-Attack in Figure 4 In this experiment, following (Qin et al. 2020).
detection-aware attack method and set the attack target class is O. The standard setting 0.047 is used the case of input range [0, 1]. which corresponds to 12 of the pixel range of 255. In Figure Some adversarial examples are flipped to target class to human perception, although the perturbation threshold is small. For some examples, it is hard flip them, e.g., the ones with a big digit and thin strokes
By comparing the adversarial examples created by Caps- Atts-Attack and Vote-Attack, we can find that there is no obvious visual difference between the adversarial examples. The observation also echos with our experiment in Appendix B In that experiment, we compute the different norms of the per- turbations created by different methods. The results in aad the perturbations created by Caps-Attack and Attack have similar norms. Hence, the adversarial examples created by Caps-Atter Attack and Vote- Vote-Attack are also visually similar.
15

---

Published as a conference paper at ICLR 2021
5
9
18833
344
(a) Clean images in SVHN test dataset
0
9
18%
3
3
4
0
9
18189
3
4
3
3
4
3
4
3
4
3
4
3
344
344
(b) Adversarial images created by Vote-Attack
5
1100
3
3
4
5
9
18
181899
4
3
4
18:8999
3
344
4
1893
4
1838
3
4
3
4
3
4
0
344
34
344
(c) Adversarial images created by Vote-Attack
Figure 4: The first subfigure shows clean test images of the SVHN dataset. The second subfigure shows the adversarial images created by Caps-Attack. Different rows correspond to different weights to reduce reconstruction eтror in Equation (6) (i.e., the second attack step in detection- aware attack method). Some images are flipped, and some hard ones are not. The images the third subfigure are the adversarial images created by Vote-Atrack. There is no obvious visual difference between the adversarial examples created by the two attacks. To be noted that the images are randomly selected (not cherry picked).
16


Reviews:
[{"review_id": "yIM_GernJz", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think the approach proposed by the authors to attack CapsNets and show how they are affected by adversarial examples is solid. It is backed by a comprehensive experimental design and empirical evidence, demonstrating that attacking the voting process of routing is better than attacking output capsules directly. \nComments:\n-- I am not sure that this statement is accurate \"Since the routing process is the main difference between CapsNets and CNNs,....\"....Capsules are built under different premises one of which is the routing process.\n--   \"In couter-part[sic] CapsNets, we apply ResNet18 backbone to extract primary capsules\".......What's the point of using a ResNet-18 backbone with CapsNets when the whole point of CapsNets is to adhere to the premises they were built under. I understand that in this paper authors evaluate a certain property but how do you know that ResNet-18 does not have an effect on this? \n-- To understand table 1 better, is Resnet = Resnet-18 and CapsNets= ResNet-18 backbone + Capsules?\n\nImprovements:\n-- Generalise this approach to other proposed CapsNets architectures, beyond Dynamic Routing, such as VarCaps (De Sousa et al. AAAI 2020, StarCaps (Ahmed et al.) etc\n-- Use the architecture without a ResNet backbone, but just a single CNN layer in the input\n-- SmallNorb could be used to see whether the novel viewpoint generalisation is affected and to what extent.\n--Along the same lines Affnist could also be used to see the effect on affine transformations (along with or instead of Cifar10 that was presented in the appendix)\n\nTypos:\n--Line 3 Section 4 \"(see Equation (??))\"", "extra_scores": {}, "extra_text": {}}, {"review_id": "_pYtdked5Ef", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper investigates robustness of Capsule Networks (CapsNets) under adversarial attacks and makes several interesting observations around the behaviour of CapsNets under attack regime which have been used later to design a new attack against these types of networks (vote attack).  Through several experiments, they show the proposed vote-attack can reduce the robust accuracy of CapsNets significantly across different methods. Authors analyze the effectiveness of this attack from different perspectives (i.e.  transferability of adversarial examples, the adversarial robustness on affine-transformed inputs).\n\n\n- Correctness and Clarity: The paper seems correct and concise and they experiment different aspects of the methods for effectiveness. The paper is also in general  well-written and easy to follow.\n- Reproducibility: They have also provided several tables and figures on the main paper and appendix to help with understanding of the results and method. They also provided detailed description of the methods, training and experimental setup.  It would be great if authors could also release the code to reproduce the results.\nRelation to prior works: I am curious to see if the proposed attack can still be effective for Self-Routing Capsule Networks (Hanh et al.)  and the other family of CapsNets in which the different routing mechanism has been used. \nAdditional Feedback and Suggestions: There are some editorial issues, like missing ref to equations (e.g. Equation ??) , and missing name tags for numbered items (e.g. Table 4.)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "YVog82YZQ4K", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a new adversarial attack targeting the votes derived from the primary capsules of a capsule network. It demonstrates that this is a stronger attack against capsules networks than an attack directly optimizing the output logits of a capsule model. This paper has some novel contributions to the literature but can be improved in a few areas.  \n\nThe paper spends a fair bit of time discussing the efficiency of their attack but does not meaningfully motivate this focus. They claim that the time it takes to compute adversarial attacks against capsule networks may have contributed to the claimed robustness, but all the papers they cite which discuss the adversarial robustness of capsule networks measures the success rate with respects to the number of attack optimization steps as opposed to optimization time. Furthermore as shown in table 3, their new attack does not significantly decrease the attack creation time for most optimization strategies. I believe the focus on attack optimization time should be removed from the paper entirely, and the space can be dedicated to some of the results relegated to the appendix such as black box attacks, or the class conditional reconstruction detection. \n\nBy creating an attack that specifically targets the votes of the primary capsules this paper is able to increase the success rate of adversarial attacks against capsule networks. It does this however, by effectively not attacking the capsule part of a capsule network. Rather they have created an attack which optimizes the features extracted by the CNN feature extractor used to derive primary capsules. This is an interesting finding, and illustrates that capsules networks are more vulnerable to adversarial attack than perhaps previously believed due to their reliance on CNN feature extraction, but it is important to note that it does so by optimizing for activations of non-capsule components of a capsule network. This is not a major flaw of the paper, but i believe it warrants some discussion, space providing.\n\nThis paper also presents the success rate and undetected rate of their new attack against the class conditional reconstruction attack presented by Yao et al. This section improves the paper, but there are some omissions. Namely they do not in the main text nor in the appendix visualize the resultant attacks. This is an issue as both Yao et al (2020) (Detecting and diagnosing adversarial images with class-conditional capsule reconstructions) and  Yao et al (2020) (Deflecting Adversarial Attacks) show that the undetected attacks often resemble the target class, even under small epsilon bounds. This paper also does not address the additional defense mechanisms presented in Deflecting Adversarial Attacks, which were shown to drastically increase the attack detection rate, specifically in colour datasets such as cifar10.  \n\nThis paper would be improved by addressing those 3 main issues.  \n\n------------------- \nsmaller issues, \n\nthe text describing figure 2 is confusing, and i am not entirely sure what the point of the figure it. perhaps more space could be dedicated the distinguishing the b and c plots and discussing why this motivates the work. \n\nthere is a missing equation link in section 4 \n\nthere is a missing table link in appendix E  \n\n\n\n----POST AUTHOR RESPONSE --------\n\n1.) Efficiency of our Vote-Attack: \n\nIt is clear that the vote attack is more time efficient than other attacks, but there is no clear motivation for this improvement. To my knowledge the attack creation time has never been a barrier to adversarial research, nor has it prevented real world adversarial attacks. As a result this focus of the paper simply confuses the reader, be spending time addressing an issue that is not important in research or practical settings.  \n\n2.) Optimizing for activations of non-capsule components of a capsule network: \n\nIn the authors response they discuss the semantic meaning of the votes of capsules. This too is a bit of a red herring. Although when discussing the motivation behind capsules, the potential for semantically meaningful capsule votes is invoked, there is nothing in the training procedure that ensures that the activations of the capsules correspond directly to features that humans would find semantically meaningful. In my original review i mentioned that by not attacking the output of the capsules after the routing procedure, this attack was simply optimizing for representations extracted from a standard neural network. In this way this work is similar to the representation attacks first presented by [1] which showed the success of representation attacks on standard neural networks.  \n\n3.) The undetected attacks often resemble the target class, under the class-conditional capsule reconstructions detection:\n\nTh addition of the attack visualizations is an improvement but the authors do not specify which attacks are successful and undetected and a few of the visualized attacks do indeed resemble the target class.   \n\n4.) The additional defense mechanisms presented in Deflecting Adversarial Attacks:\n\nThe authors are right to point out the scope of the paper, and it is perhaps unreasonable to expect this paper to address these defence mechanisms, but their inclusion would greatly strengthen the paper. \n\n[1] Sara Sabour, Yanshuai Cao, Fartash Faghri, and David J Fleet. Adversarial manipulation of deep\nrepresentations. In ICLR, 2016.", "extra_scores": {}, "extra_text": {}}, {"review_id": "TeJp--8mLI2", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Authors argue that in order to attack Capsule network more effectively one should consider their inner workings, i.e. iterative routing. They propose two reasons behind the relative robustness of capsnets vs cnns: 1) gradient obfuscating 2) being more computationally intensive. Therefore, they propose a new attack which attacks a CapsNet running only 1 routing iteration. \nThe attacks are shown to be more effective than targeting the 3-iteration capsnets directly. Which suggests the gradients for 1-iteration CapsNets are close enough to a 3-iteration version that the adversarial images fool a 3-iteration capsnet too. \n\nThey also show that their attacks stay undetected by Qin et al method (class conditional reconstruction). However, the point of Qin et al was to some degree that although adversaries exist that Qin et al method will not detect, such adversaries have semantical resemblance to the target class and they are not undetected to human eye. i.e. the change is not scattered noise and is not imperceptible anymore. Therefore, to fully claim that their method fools Qin et al, a set of advarsarial images should be visualized to showcase they being imperceptible changes while going undetected.\n\nAuthors also provide a study on how attacking a capsule network changes the voting/routing mechanism. They find that attackers make the votes orthogonal to the target class parameters and therefore, remove votes from contributing. \nI was not able to see the connection of why this finding leads to their advarsarial attack. So maybe more clarification is required. \n\nBut given gradient obfuscating as a reason for capsnet robustness their attack is intuitive (rather than attacking the 3 iterations attack a 1 iteration version). \n\nThe study by itself is interesting and can lead to further robustness of CapsuleNetworks.\n\nOne caveat of the vote attack is that it seems it is most effective where there is only one Capsule layer. It seems that adding more and more capsule layers would make the model further robust to adversarial attacks. Hinton et al 2018 specifically had 3 capsule layers. An study with several capsule layers would enhance this papers argument and showcase its effectiveness better.\n\nPros: The paper is relatively well written. Authors explain their attack and motivate their research well. Also the experimental setup covers several attacks. There is novelty both in the idea and in the voting study.\n\nCons: Not covering multi capsule layer architectures, weak argument against Qin et al (not visualizing their detection aware advarsaries). Also, given that Michels et al 2019 has shown CapsNets are not robust to all attacks and already other attacks exist that CapsNets fail on, the relative interest domain of this paper is limited to researchers working on Capsule Networks.\n\n--------------------------------------------------POST AUTHOR RESPONSE\n1) Thank you for adding Fig. 3. My concern about the attacks being noticeable is resolved. \n3 & 4) The scope of this work is still limited to Specifically dynamic routing, which we already know are susceptible to black box attack (the community that already exists on architecture un-aware attacks) and some other attacks. As author's mention it is not trivial to apply their attack to other capsule architectures either. \nHowever, I find their various analysis informative. \nTherefore, I am increasing my score to 6.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the robustness of CapsNets under adversarial attacks. It is found that the votes from primary capsules in CapsNets are manipulated by adversarial examples and that the computationally expensive routing mechanism in CapsNets incurs high computational cost. As such, a new adversarial attack is specially designed by attacking the votes of CapsNets without having to involve the routing mechanism, making the method both effective and efficient.

**Strengths:**
  * This is the first work which proposes an attack specifically designed for CapsNets by exploiting their special properties.
  * The proposed vote attack is more effective and efficient than the other attacks originally proposed for CNNs rather than CapsNets.
  * The paper is generally well written.
  * The experimental study is quite comprehensive.
  * The code will be made available to facilitate reproducibility.

**Weaknesses:**
  * The study is mostly for only one type of CapsNets. It is not clear whether the observations in this paper still hold generally for other types of CapsNets even after some additional experiments have been added.
  * The presentation of the paper has room for improvement.

The authors are recommended to proofread the references thoroughly to ensure style consistency such as the consistent use of capitalization, e.g.
  * “Star-caps” -> “STAR-Caps”
  * “ieee symposium on security and privacy (sp)” -> “IEEE Symposium on Security and Privacy (SP)”

Despite its weaknesses especially those pointed out by Reviewer 2, this paper would be of interest to other researchers as it is the first paper that studies adversarial attacks on CapsNets.

Decision: Accept (Poster)