# An Evasion Attack against Stacked Capsule Autoencoder
## 5 Experimental Evaluation
### Results and Discussion  
We perform the experiment on each dataset and each classifier with the above three algorithms, and the results are shown in Tables 6-8 respectively. The attack success rate is the ratio of the number of samples that are misclassified by the classifier to the total number of all test samples. The average \(L_{2}\) norm of \(p\) represents the average amount of the perturbations, and a lower value corresponds to an attack with better stealthiness. The standard deviation of \(p\) is used to estimate the stability of the attack. A higher standard deviation means that the performance of the attack fluctuates according to different samples. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Classifier**} & **Algorithm for** & **Attack** & **Average \(L_{2}\)** & **Standard** \\  & **Computing** & **Success Rate** & **Norm of \(p\)** & **Deviation of \(p\)** \\ \hline \multirow{2}{*}{Prior k-means classifier} & GDU & 0.9790 & 2.6292 & 1.0797 \\  & PSC & **1.0000** & 3.7964 & 1.5333 \\  & OPT & **1.0000** & **1.0839** & **0.6015** \\ \hline \multirow{2}{*}{Posterior k-means classifier} & GDU & 0.8994 & 3.1017 & 1.3118 \\  & PSC & 0.9994 & 4.4982 & 1.6447 \\ \cline{1-1}  & OPT & **1.0000** & **1.1838** & **0.6222** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Experimental results for the MNIST dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Optimizer Parameter** & **Value** \\ \hline Algorithm & Adam \\ Learning rate & 1.0 \\ \(\beta_{1}\) & 0.9 \\ \(\beta_{2}\) & 0.999 \\ \(\epsilon\) & \(1\times 10^{-8}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Settings of the optimizer used to attack the SCAEs. The above results show that our proposed evasion attack has high attack success rates with all three algorithms. When using GDU or PSC, the average \(L_{2}\) norm of \(p\) is larger than that obtained using OPT. Additionally, the perturbations generated by OPT have the smallest standard deviation of \(p\) since GDU and PSC focus on misclassification rather than invisibility, and their performance is relatively worse. As OPT takes the amount of perturbation into consideration and is more time-consuming, perturbations generated by it have better stealthiness and stability. We randomly select one sample from each dataset to perform the attack as examples. The visualization of the perturbed images is shown in Figure 3:  
Figure 3 shows that for the MNIST and Fashion-MNIST samples, the perturbations are confined to the objects and their surrounding areas, meaning that the adversarial images \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Classifier**} & **Algorithm for** & \multirow{2}{*}{**Attack**} & **Average \(L_{2}\)** & **Standard** \\  & **Computing** & **Success Rate** & **Norm of \(p\)** & **Deviation of \(p\)** \\ \hline \multirow{2}{*}{Prior k-means classifier} & GDU & 0.9836 & 3.0482 & 1.5615 \\  & PSC & 0.9960 & 4.0945 & 1.8384 \\  & OPT & **1.0000** & **1.4160** & **0.8982** \\ \hline \multirow{2}{*}{Posterior k-means classifier} & GDU & 0.9686 & 3.0768 & 1.6260 \\  & PSC & 0.9622 & 4.0196 & 1.9138 \\ \multirow{2}{*}{Posterior k-means classifier} & OPT & **1.0000** & **1.3598** & **0.9856** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Experimental results for the Fashion-MNIST dataset. Figure 3: **(I)** Original images; **(II)** adversarial samples generated by GDU; **(III)** adversarial samples generated by PSC; and **(IV)** adversarial samples generated by OPT.