## 1 Introduction  
Deep neural networks (DNN) have recently been showing high performance for various applications such as computer vision [1, 2], natural language processing [3] and speech recognition [4]. However, a major vulnerability of existing neural networks has been pointed out, which is called vulnerability against adversarial examples. For an image classification task, an adversarial example is an image that is slightly modified from the original image, with the purpose of fooling a classifier network [5]. While the distorted image looks very similar to the original image and looks natural to the human eye, the classifier network misclassifies the image and outputs a wrong result. There are many different methods proposed for creating an adversarial image from an original image, such as Fast Gradient Sign Method (FGSM) [6], Carlini and Wagner Attack (C&W) [7], Deepfool [8] and Jacobian Saliency Map Attack (JSMA) [9]. Since the vulnerability has been found, researchers have put a large effort to make the neural networks robust to adversarial example attacks. Various solutions have been proposed, but many of them were found to be less powerful because new attack methods or small modifications to existing attack methods could circumvent the defense mechanisms and succeed in fooling the networks [7]. Among defense methods, _adversarial training_ is a method that has shown some good results in terms of robustness. Specifically, [PERSON] et al. used adversarial training to train a network that is mostly robust to adversarial images where the perturbations are \(L_{\infty}\) norm bounded by \(\epsilon=0.3\)[10]. However, to increase robustness by adversarial training, we need a large number of training data [11]. In addition, the network is still vulnerable to other attacks that uses a different distance metric such as \(L_{2}\) norm or \(L_{0}\) norm [12]. Since there are plenty of methods that create adversarial examples, a natural question is whether we can use adversarial training to train a network so that it is robust to multiple types of adversarial examples, such as ones created using \(L_{0}\)-, \(L_{1}\)-, \(L_{2}\)- and \(L_{\infty}\)-perturbation. [PERSON] et al. [13] tried to answer this question in the context of a white-box attack, where the attacker has complete access to the parameters of the trained model. They found that different types of adversarial examples, such as examples created using \(L_{\infty}\)- and \(L_{1}\)-perturbation, are mutually exclusive, which means adversarially training with one type of adversarial examples does not help defending against the other types of adversarial examples. Moreover, adversarially training the model with one type of adversarial examples may increase vulnerability to the other type of adversarial examples. In this paper, we try to answer the question in the context of a black-box attack, in which the attacker does not have access to the parameters of the trained model. In contrast to a white-box attack, an attacker performing a black-box attack cannot generate optimized adversarial examples to fool the target network. We also assume that the attacker cannot reverse engineer the model by repeatedly testing the model with his own inputs [14]. In many real-world settings, a black-box attack is the only choice for the attacker, because the service provider does not open the model parameters to the public. In addition, repeatedly testing the model would incur time and monetary cost [15]. Under these assumptions, the attacker trains his own target network and uses the target network to apply an attack algorithm and create adversarial examples. These examples are fed into the defender's network, hoping that they are misclassified by the target network. To find out if adversarial training is effective in defending against multiple types of adversarial examples, we train a neural network model using adversarial examples created from various well-known attack methods.