# Adversarial Training Methods for Deep Learning: A Systematic Review
## 2 Background
### Adversarial Samples  
The concept of adversarial samples appeared in 2014 when [PERSON] et al. [4] proposed the use of the box-constrained optimizer L-BFGS to generate misclassified data samples. The algorithm could find indistinguishable perturbations at L\({}_{2}\) distance to cause machine learning models to fail to produce a correct result. This was one of the earliest indications of the existence of adversarial samples.  
Later, in 2015, [PERSON] et al. [3] provided a more detailed explanation of the phenomenon of adversarial examples. They found that, when an image is perturbed in a direction, the logit output of the classifier model normally changes in a linear way. They hypothesized that this is caused by the partial linear properties of the activation function of the neurons. With this hypothesis, they proposed the fast gradient sign method (FGSM) to generate adversarial samples of clean images with only the gradient of the loss function. Later, [PERSON] et al. [6] enhanced their previous approach by proposing a more advanced algorithm based on the FGSM to apply perturbations to the input image iteratively, called the basic iterative method (BIM). This algorithm greatly improved the success rate of the generated adversarial samples that attacked the network models. Another improvement was proposed by [PERSON] et al. [13] in 2018 that involved combining the BIM with random initialization. This method is widely used as a benchmark algorithm to test and train a robust model [5]. These methods primarily generate adversarial samples with L\(\infty\) norms. The L\(\infty\) norm is one of the commonly used distance metrics in adversarial machine learning. The other commonly used metrics include L\({}_{0}\) and L\({}_{2}\) distance metrics. In this case, the L\(\infty\) metric represents the max perturbation across all the input values for the adversarial sample. Based on these findings, the adversarial samples were described as the error caused by the linear properties of the classifier models, which is different from the normal bias and boundary error. The existence of adversarial samples is related to the fundamental properties of the current machine learning models.  
In addition to these adversarial attack methods, there are other attack methods that focus on different distance metrics and vulnerabilities of the models. The DeepFool algorithm was proposed by [PERSON] et al. (2016) to produce adversarial samples of inputs with reduced perturbation distances compared to the FGSM. The algorithm considered the L\({}_{2}\) norm to be the closest decision boundary to generate the adversarial samples. [PERSON] and [PERSON] attacks (C&W attacks), proposed by [PERSON] et al. (2015), considered all three distance metrics (L\({}_{0}\), L\({}_{2}\), L\({}_{0}\)) to construct a strong attack. This attack method was proven to be highly effective with high transferability between models. The one-pixel attack proposed by [PERSON] et al. (2016) constrained the L\({}_{0}\) norm to perturb a fixed number of pixels of original inputs by utilizing an evolutionary algorithm. More adversarial attack methods have been developed to produce misclassification samples for machine learning models ([PERSON] et al., 2016). Based on these newly developed algorithms, a more general description of adversarial samples could be any malicious data input specifically created for a machine learning model to produce a misclassification result. In this study, we consider this more general description our definition of adversarial samples.