## 4 Impact of Data Augmentation on Robustness against Adversarial Examples
### Robustness of Adversarially Trained Network with Mismatching Data Augmentation Strategy  
In the previous experiments, the same augmentation strategy was used by the attacker and the defender. If the defender obtained adversarial samples for training from a network trained with data augmentation, the attacker also created adversarial examples using a network trained with the same data augmentation strategy. On the other hand, if the defender trained a network without data augmentation, the attacker also used a network trained without data augmentation as the target network for the black-box attack. In such cases, the adversarial training was successful defending against a particular attack algorithm. However, what if the attacker and the defender used a different  
Figure 4: Test accuracy of LeNet5 model on each adversarial example attack. Each model is trained with: (**a**) Clean; (**b**) FGSM; (**c**) JSMA; and (**d**) C&W. augmentation strategy? Consider the case where the defender and the attacker both train a LeNet5 model and also use the same attack algorithm such as FGSM to create adversarial examples. However, the defender's network is trained with data augmentation while the attacker's network is not. Will the adversarial training still be successful against the attack? To answer the question, we conducted an experiment for cases where the defender uses data augmentation and the attacker does not, and vice versa. The result is shown in Table 5. In the table, \(M\) denotes a network model trained using data augmentation and \(M^{\prime}\) is a network trained without data augmentation. For the attack, \(A\) is the set of adversarial examples created from a target network trained with data augmentation and \(A^{\prime}\) is the set created from a network trained without data augmentation. For an undefended model, the attack is more successful if the augmentation strategy matches between the defender and the attacker. The accuracy of the FGSM samples is 41.74% when both the defender and the attacker uses data augmentation when training their networks. If both do not use data augmentation, the accuracy of the FGSM samples is 62.67%. If the augmentation strategy is different between the defender and the attacker, the accuracy of the FGSM samples is approximately 70%. However, when the defender is using adversarial training, the robustness is higher when the defender and the attacker use the same augmentation strategy. In the table, if the defender uses FGSM, JSMA and C&W samples for adversarial training, the accuracy of FGSM samples is above 99% when the augmentation strategy matches between the defender and the attacker. However, the accuracy drops down to 85% if they use different augmentation strategies. This result shows that the effectiveness of adversarial training depends not only on what algorithm is used for creating adversarial samples, but other factors such as data augmentation strategies for training the target network. It means if the network model needs to be robust against all possible attacks, the number of adversarial examples needed for adversarial training could become very large in order to cover all possible combinations. Overall, training a robust network that guarantees a certain level of robustness against all types of adversarial examples is very difficult, even for black-box attacks.