# An Evasion Attack against Stacked Capsule Autoencoder
## 5 Experimental Evaluation  
### Experimental Setup  
The original paper of the SCAE [8] showed that the SCAE can achieve 98.7% accuracy on MNIST, while only achieve 55.33% and 25.01% accuracy on SVHN and CIFAR10, which is not high enough to meet the needs of our experiments. ImageNet, which is similar to CIFAR10, is unsuitable for our experiments as well. Based on the performance of the SCAE on various datasets, we finally choose three datasets for the experiments: MNIST, Fashion-MNIST, and German Traffic Sign Recognition Benchmark (GTSRB), in order to make our experimental results more convincing. We train three SCAE models [8; 34] using the main parameter set as shown in Table 2:  
Most of the parameters are equivalent to those given in the supplemental document of the original paper of the SCAE [8]. For part CNN, \(2\times(128\):2) means two convolutional layers with 128 channels and a stride of two. For the set transformer, \(3\times(1-16)-256\) means three layers, one attention head, 16 hidden units and 256 output units. All three models share the same optimizer shown in Table 3:  
\begin{table}
\begin{tabular}{c c c c} \hline
**Algorithm** & **GDU** & **PSC** & **OPT** \\ \hline Num of iterations & \(n_{iter}\) & Not greater than \(n_{iter}\) & \(n_{o\_iter}\times n_{i\_iter}\) \\ Gradient calculation & \(\
abla_{x+p}\sum\limits_{i\in S}E(x+p)_{i}\) & \(\
abla_{x+p}\sum\limits_{i\
otin S}E(x+p)_{i}\) & \(\
abla_{p}\bigg{(}\sum\limits_{i\in S}E(x+p)_{i}+\left\lVert p\right\rVert_{ 2}\bigg{)}\) \\ Update of adversarial sample & Update along the sign & Update along fixed & Update via the optimizer \\ per iteration & of the gradient & direction & \\ Num of pixels modified & All pixels & 2 & All pixels \\ per iteration & & & \\ \hline \end{tabular}
\end{table}
Table 1: The complexity of the three algorithms: GDU, PSC, and OPT.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Dataset** & **MNIST** & **Fashion-MNIST** & **GTSRB** \\ \hline Canvas size & 40 & 40 & 40 \\ Num of part capsules & 24 & 24 & 40 \\ Num of object capsules & 24 & 24 & 64 \\ Num of channels & 1 & 1 & 3 \\ Template size & 11 & 11 & 14 \\ Part capsule noise scale & 4.0 & 4.0 & 0.0 \\ Object capsule noise scale & 4.0 & 4.0 & 0.0 \\ Part CNN & \(2\times(128\):\(2)-\(2\times(128\):\(1)\) & \(2\times(128\):\(2)-\(2\times(128\):\(1)\) & \(2\times(128\):\(1)-\(2\times(128\):\(2)\) \\ Set transformer & \(3\times(1-16)-256\) & \(3\times(1-16)-256\) & \(3\times(2-64)-256\) \\ \hline \end{tabular}
\end{table}
Table 2: SCAEs’ main parameters.  
According to [8], the value of \(k\) of the k-means classifiers is set as 10, which equals the number of the categories.  
In particular, when training the SCAE on GTSRB, the performance is poor when all 43 categories are used; thus, we choose only 10 categories to ensure that the classification accuracy meets the requirement of our experiments. Furthermore, we use min-max normalization (shown Formula (8)) to reduce the luminance difference between images. The chosen categories are shown in Figure 2.  
\[x_{i}^{\prime}=\frac{x_{i}-\min(x)}{\max(x)-\min(x)} \tag{8}\]  
The classification accuracy of all three models is given in Table 4: