## 3 The Rise of Adversarial Machine Learning (AML)
### Understanding Adversarial Examples
#### 3.1.3 Adversarial Examples Crafting Methods  
The crafting methods of adversarial examples have been widely investigated and a framework might be proposed for further clarification. It is worth noting that the efficiency of crafting adversarial examples comes from minimizing the total perturbation as much as possible to avoid being easily detected. Accordingly, there are two possible steps that are repeated iteratively where datapoint \(X\) is replaced with \(X+\delta X\) until the adversarial goal is achieved and the perturbation \(\delta X\) is applied. An explanation of all equations notations is presented in Table 1. The following is the main steps of crafting methods of adversarial examples [28]:  
* Direction Sensitivity Estimation. In this step, the sensitivity of applying changes to inputs features is measured by analyzing the data distributions around specific datapoint where the model decision boundary is possibly affected and changed. Accordingly, there are several techniques for performing direction sensitivity estimation such as the following:  
* **Limited-memory Broy-den, Fletcher, Goldforb, Shanno (L-BFGS). ** This method has been proposed by [PERSON] et al. [6] for crafting adversarial examples through a minimization problem. In such a scenario, the adversary constructs with L2-norm an image \(X^{\prime}\) similar to the original image \(X\) where \(X^{\prime}\) can be labeled as a different class. This is considered a complex problem to be solved due to the use of nonlinear and non-convex functions. They tried to search for an adversarial sample by finding the minimum loss function additions to L2-norm according to the following formula [28]: \[\text{min.c.}\parallel X-X^{\prime}\parallel_{2}+loss_{F,J}(X^{\prime})\] (1) where \(c\) is a hyper-parameter that is randomly initialized by linear search; \(\parallel X-X^{\prime}\parallel_{2}\) is L2-norm and \(loss_{F,J}(*)\) is the loss function. Thus, the problem is transformed into a convex optimization process but with complicated and expensive calculations [28,35]. * **Fast Gradient Sign Method (FGSM). ** This method has been proposed by [PERSON] et al. [26] for crafting adversarial examples where the cost function is calculated with regard to the gradient direction. FGSM is different from LBFGS since it uses the L1-norm and does not perform iterative processes which makes it an excellent choice when it comes to computational cost and time. In such a scenario, misclassification can occur by adding perturbations according to the following formula [28]: \[X^{adv}=X+esign(\
abla_{X}J(X,\mathcal{Y}_{true}))\] (2) where \(X^{adv}\) is the adversarial example of \(X\), \(X\) is the original sample and \(\epsilon\) is a hyper-parameter that is randomly initialized to control the amplitude of the disturbance. Additionally, \(sign\) (_) is a sign function, and \(J\) (_) is the cost function with respect to the original sample with the correct label \(\mathcal{Y}_{true}\) and \(\
abla_{X}J\) is the gradient of \(X\). However, this method is subjected to label leaking where other researchers suggest replacing the correct label \(\mathcal{Y}_{true}\) with the predicted label [28,35,36]. * **Iterative Gradient Sign Method (IGSM). ** This method has been proposed by [PERSON] et al. [36] for crafting adversarial examples by optimizing the FGSM method. Per turbations are iteratively applied into several smaller steps followed by clipping the results which guarantees that these perturbations are close to the original samples. It is worth noting that the non-linearity of IGSM is in the gradient direction where multiple iterations are required. This reflects the simplicity of this method compared to L-BFGS and its higher success rate of the resulting adversarial samples compared to FGSM.