## 3 The Rise of Adversarial Machine Learning (AML)
### Modelling the Adversary
#### 3.2.1 Adversarial Capabilities  
Adversarial capabilities represent the potential impact of an adversary when attacking the ML models which can be grouped into two categories: Influence, and Specificity [7; 29; 39]. Influence:  
This category focuses on the adversary's influence on certain classification elements such as changing the dataset or the algorithms when running attacks on the target model. Such attacks include causative, evasion, or exploratory attacks where there is an influence over either the training dataset or the testing dataset, or both. Consideration of training and testing phases is used to clarify more about the adversary's influence according to his/her capabilities in those phases, as follows [29; 39]:  
* Training Phase Influence: In this phase, attacks take place by influencing or corrupting the model performance in which the datasets alteration is performed, and can be summarized as follows: \begin{table}
\begin{tabular}{c c} \hline
**Symbol** & **Definition** \\ \hline min & Minimum distance \\ C & Random hyperparameter \\ X & Original image \\ X\({}^{\prime}\), X\({}^{\text{adv}}\) & Adversarial image \\ \(\mid\mid\) X \(-\) X\({}^{\prime}\mid\mid_{2}\) & L\({}_{2}\)-norm \\ Loss\({}_{\text{E}}\)(X\({}^{\prime}\)) & The loss function of (X\({}^{\prime}\)) \\ \({}_{\text{E}}\) & Random hyperparameter \\ Sign (*) & Sign function \\ J(*) & Cost function \\ \(\mathcal{Y}_{\text{true}}\) & Correct label of X \\ \(\
abla_{\text{X}}\)J(*) & The gradient of X \\ ClipX\({}_{\text{c},\epsilon}\) (*) & Denotes [X \(-\) \(\epsilon\), X \(+\) \(\epsilon\)] \\ \(\mathcal{Y}_{\text{LL}}\) & Least likely (the lowest probability) target class \\ \(\
abla\)F(x) & Forward derivative \\ \hline \end{tabular}
\end{table} 
Table 1: Notations of equations. Data Injection: The adversary can affect the target model by injecting adversarial samples and inserting them into the training dataset. This can happen with some control over the training dataset but not over the learning algorithm. 2. Label manipulation: The adversary can modify the training labels only and gain the most vulnerable label to degrade the model performance. The label perturbations can happen with some control over the training dataset and can be applied in a random manner to the distribution of training data. An experiment indicates that a random perturbation of the training labels can degrade the performance of shallow ML models significantly [40]. 3. Data Manipulation: The adversary can poison the training dataset before it has been used for training the target model. The adversary can modify both the labels and input features of the training data and affect the decision boundary. The training data can be accessed but without the need to access the learning algorithm. 4. Logic Manipulation: The adversary can manipulate the learning algorithm and affect its workflow logic which thereby makes the ML model under his/her control. 5. Testing Phase Influence: In this phase, attacks take place to force the target model to produce incorrect outputs without influencing it. These types of attacks use other techniques to extract useful information rather than influencing the training phases, and can be summarized as follows: 1. Model Evasion: The adversary can evade the target model by crafting adversarial samples during the testing phase. 2. Model Exploratory: The adversary can gain various levels of knowledge about the target model in terms of the learning algorithm and training dataset distribution pattern, as follows: 1. Data Distribution Access: The adversary can access the training dataset distribution of the target models. The substitute local model is built to imitate the target model in classifying a set of distribution samples. This helps in generating adversarial samples where they are used on the target model for misclassification purposes. 2. Model Oracle: The adversary can only query the target model by inputting a set of samples and checking the related output labels.