## 3 Effect of Adversarial Training on Network Performance
### Performance of an Adversarially Trained LeNet5 Network  
Figure 3 shows the accuracy of the test set while the LeNet5 network is being trained. The network in Figure 3a is trained with only the clean images, while the networks in Figure 3b-d are trained with clean images plus adversarial images generated from FGSM, JSMA and C&W, respectively. The lines in each graph indicate the test sets. 'Clean' means the test set consist of original images from the MNIST test set. 'FGSM,' JSMA and 'C&W' means the test set consists of adversarial examples created from the original test set samples using FGSM, JSMA and C&W attack, respectively. In Figure 3a, we can see that the test accuracy of FGSM, JSMA and C&W set is much lower than the clean set, achieving between 40% and 60% accuracy. This means that the attack is successful roughly half of the time, even when the attacker does not know the parameters of the target network. Although the exact accuracy will depend on what parameters the attacker uses to create adversarial examples, this result shows that it is quite easy to break down the performance of a neural network even with limited information. Figure 3b shows the test accuracy when the network is trained with FGSM samples along with their clean pairs. We can see that the network accurately classifies most of FGSM adversarial samples, achieving an accuracy comparable to the clean samples. This means that the network has successfully learned the distribution of images created by FGSM. On the other hand, the accuracy of JSMA has not improved, suggesting that the JSMA images have a different distribution with original and FGSM images. For C&W, adversarial training with FGSM was beneficial, showing a 20% improvement in the accuracy. This result suggests the features of FGSM images have some correlation with features of C&W images, while there is little correlation between FGSM and JSMA images. In other words, perturbing samples using \(L_{\infty}\) distance generates images closer to those generated using \(L_{2}\) distance, compared to those generated using \(L_{0}\) distance. A similar observation can be made in Figure 3c as well, which shows performance of the network trained using JSMA samples along with their original pairs. Training with JSMA is most helpful in classifying JSMA adversarial examples, while the accuracy improvements are limited for FGSM and C&W samples. Figure 3d shows \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Attack Algorithm**} & \multirow{2}{*}{**Parameters**} & \multicolumn{2}{c}{**Time for Generating 60,000 Adversarial Samples (s)**} \\ \cline{3-4}  & & **LeNet5** & **ResNet** \\ \hline FGSM (\(L_{\infty}\)) & \(\epsilon=0.2\) & 48 & 66 \\ \hline JSMA (\(L_{0}\)) & \(\theta=1.0\), \(\gamma=0.2\) & 2226 & 9180 \\ \hline C\&W (\(L_{2}\)) & confidence: 5 & 8142 & 49,542 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameters for the attack algorithms and time for generating adversarial examples. Figure 2: Random translation, random rotation and random erasing applied to a sample in the training set. performance of the network trained using C&W samples. The network successfully classifies C&W adversarial samples, whereas achieves 60-70% accuracy on FGSM and JSMA samples. Table 2 shows the test accuracy of LeNet5 networks trained using different combinations of clean and adversarial examples. For each set of test samples, the medium and maximum accuracy obtained between 20 and 50 epochs is shown in the table. For example, for a network trained using only the clean images, the maximum accuracy achieved on clean test images is 99.42%, while the maximum accuracy of FGSM, JSMA and C&W images is 50.23%, 64.07% and 55.28%, respectively. We can see that, without adversarial training, the network can be easily attacked by a variety of attack methods, with its accuracy considerably lowered.