## 4 Impact of Data Augmentation on Robustness against Adversarial Examples
### Performance of a Network Trained without Data Augmentation  
We conducted an experiment where both the defender and the attacker train LeNet5 networks without data augmentation and create adversarial examples from the networks. Once generating adversarial examples from the training set, the defender trains LeNet5 again with original samples and the adversarial samples, also without data augmentation. The result is shown in Table 4. In the perspective of effectiveness of adversarial training, similar patterns can be observed with other experiments: adversarial training is effective in classifying adversarial samples created from the same method as samples used in training. Similar results can be observed if the defender and the attacker use the same strategy, regardless of whether they use or do not use data augmentation.  
Another interesting observation when comparing Tables 2 and 4 is the trade-off between accuracy of the clean test set and accuracy of the adversarial examples. Without data augmentation, a LeNet5 network achieves 99.16% accuracy of the clean test set. When data augmentation is used, the accuracy can be improved to 99.42%. However, an undefended network trained without data augmentation achieves much higher accuracy for adversarial examples for all three methods.  
\begin{table}
\begin{tabular}{l l c c c c c c c c} \hline \hline  & & \multicolumn{4}{c}{**Maximum**} & \multicolumn{4}{c}{**Median**} \\ \cline{3-10}  & & **Clean** & **FGSM** & **JSMA** & **C\&W** & **Clean** & **FGSM** & **JSMA** & **C\&W** \\ \hline \multirow{6}{*}{
\begin{tabular}{l} Trained \\ model \\ \end{tabular} } & Clean & 99.16 & 66.58 & 70.03 & 88.22 & 99.02 & 63.27 & 66.99 & 83.02 \\ \cline{2-10}  & FGSM & 99.11 & 99.11 & 74.41 & 95.66 & 99.00 & 99.01 & 73.05 & 94.19 \\ \cline{2-10}  & JSMA & 99.15 & 72.97 & 98.24 & 95.80 & 99.03 & 69.45 & 98.09 & 94.06 \\ \cline{2-10}  & C\&W & 99.19 & 85.58 & 75.11 & 99.38 & 99.08 & 79.18 & 72.93 & 99.21 \\ \cline{2-10}  & FGSM + JSMA & 99.16 & 99.00 & 98.15 & 98.02 & 99.05 & 98.89 & 97.88 & 97.52 \\ \cline{2-10}  & FGSM + C\&W & 99.14 & 99.12 & 78.55 & 99.37 & 99.01 & 99.03 & 77.06 & 99.16 \\ \cline{2-10}  & JSMA + C\&W & 99.25 & 86.68 & 98.36 & 99.23 & 99.15 & 81.08 & 98.12 & 99.03 \\ \cline{2-10}  & FGSM + JSMA + C\&W & 99.02 & 99.03 & 98.17 & 99.22 & 99.10 & 98.94 & 97.82 & 99.09 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Test accuracy of LeNet5 measured between 20 and 50 epochs. The network is trained without data augmentation techniques such as random rotation and random translation.  
Performance comparison of the LeNet5 network with and without data augmentation is shown in Figure 4. In Figure 4a, we can see that the accuracy is slightly improved for the clean test set, when data augmentation is used. However, for Figure 4b-d, the performance of LeNet network trained with data augmentation achieves significantly lower accuracy on adversarial examples compared to a network trained without data augmentation. It is well known that data augmentation improves classification performance of a network by increasing generalization capacity [36], and thus data augmentation is often heavily used for most of the tasks. However, it is possible that data augmentation strategies such as rotation and translation could increase vulnerability of the network against adversarial examples. Although not covered in this paper, we believe this trade-off is an important problem that needs to be looked into in the future.