## 2 Preliminaries  
### Creating Adversarial Examples  
Although it is possible to create an adversarial example from scratch, a typical way is to take an original sample and modify the sample so that it could be misclassified by the target network. Figure 1 shows images from the MNIST dataset plus adversarial examples created by three different methods. We can observe from the images that the three methods, while they have the same goal, creates different images. A \"good\" adversarial example of an image is the one that looks very similar to the original image while being misclassified by the network. While there are many different metrics to measure image similarity, norm-based distance metrics are frequently used. For example, FGSM [6] uses the \(L_{\infty}\) distance by limiting the amount of noise added to each input pixel. On the other hand, JSMA [9] uses the \(L_{0}\) distance by limiting the number of pixels that can be modified. [PERSON] et al. [7] generalized the problem of creating adversarial examples, stating that the attacker tries to minimize the \(L_{p}\) distance between the original image \(x\) and the adversarial image \(x^{\prime}\), while \(x^{\prime}\) is misclassified by the target network. The \(L_{p}\) distance can be defined as Equation (1), where \(p\) is a non-negative number and \(n\) is the dimension of vector \(v\). \[\|v\|_{p}=\left(\sum_{i=1}^{n}|v_{i}|^{p}\right)^{\frac{1}{p}} \tag{1}\]  
When using \(L_{p}\) distance, we can define \(p\) to be zero or infinity. The \(L_{0}\) distance is the same as number of dimensions where the value is different between \(x\) and \(x^{\prime}\). The \(L_{\infty}\) distance, on the other hand, is the maximum of distances between each dimension of \(x\) and \(x^{\prime}\). For MNIST images, each dimension of input corresponds to a pixel. Thus, \(L_{0}\) distance between two images is the number of pixels where the color is different, whereas \(L_{\infty}\) distance between two images is the maximum of color differences among pixels of \(x\) and \(x^{\prime}\). For other \(p\) values, we can calculate the \(L_{p}\) distance by calculating \(\|x-x^{\prime}\|_{p}\). Among different values of \(p\), the \(L_{2}\) distance which corresponds to the Euclidean distance is the most widely used distance metric. In the paper, we ch0 ose three representative methods for creating adversarial examples: FGSM, JSMA and C&W \(L_{2}\). These methods use \(L_{\infty}\), \(L_{0}\) and \(L_{2}\) distance when creating adversarial examples, respectively. While many other attack methods also use these distance metrics, it should be noted that there are methods that use other \(p\) values such as \(L_{1}\) distance [17]. Suppose we are allowed to modify an original image \(x\) to an adversarial image \(x^{\prime}\) so that the distance between the two is less than some threshold under a particular distance metric. Among the infinite possible ways of modifying \(x\) under the given condition, we should choose the direction that will give us the best chance that \(x^{\prime}\) will be misclassified. If the attacker knows the architecture of the target network and has access to its parameters, he can calculate the gradient of a loss function  
Figure 1: Image of original MNIST and adversarial attacked examples generated by FGSM, JSMA and C&W \(L_{2}\) attacks. (such as cross-entropy between the prediction and the true class) with respect to the input. Using the gradient, the attacker can modify \(x\) in the direction that will increase the loss function. This is called a _white-box_ attack, meaning that the information of the target network is open to the attacker. However, often times the attacker does not have information on the architecture of the neural network or its parameters. Attacking the network without this information is called a _black-box_ attack. One can also define a _gray-box_ attack when the attacker has partial information on the target network. For example, the attacker may know the architecture of the target network, but not its parameters.