# Adversarial Training Methods for Deep Learning: A Systematic Review
## 4 Findings
### Summary of the Results  
The 78 included papers are listed below. Each paper includes a unique solution to the adversarial training schema or proposes an optimization procedure utilizing adversary generation. The distribution of the papers by year of publication is shown below. From Figure 5, we can see an increasing interest in the research field related to this topic. We think this presents a great opportunity to conduct this research to produce a future roadmap. Based on the adversary generation approach used in adversarial training methods, we categorize the adversarial training methods as shown in Table 3. Furthermore, we discuss the details of the adversary generation methods and categorize them into further sub-categories whenever applicable. Adversarial training can be formulated as the following optimization problem:  
\[min\sum_{i}maxL(f(x_{i}+\delta),\ y_{i}), \tag{1}\]  
The inner part of the formula is intended to maximize the loss \(L\) of the model \(f\) regarding the output label \(y\) by adding perturbation \(\delta\) into the input \(x\). The optimization goal is to minimize the maximum loss of the model. In the following sections, we want to use the inner approximation function to lead the discussion. In general, inner approximation functions are constructed using different attack methods. This approximation is important to determine the upper limit of the adversarial optimization. Generally, the adversarial training method with a similar inner approximation function will share similar benefits and limitations, which are shown in Table 3. However, there might be several improvements built upon these basic methods. The detailed discovery of the outer optimization related to different improvements will be analyzed in the sub-categories to focus on the varieties of adversarial training. Overall, adversarial attacks could be fitted into two more extensive categories: white-box and black-box attacks [2]. The white-box or black-box adversarial generation methods usually share some common properties. The overall structural relationship of the common adversary generation methods studied in this paper is shown in Figure 6. The commonly used acronyms are also shown in the same figure. Figure 5: Number of publications in each year. White-box attacks refer to the adversarial attacks that require the parameter information of the targeted model [2]. This category of methods is the most used method in adversarial training or robustness optimization, as can be seen in Figure 4. The conventional white-box attacks used by adversarial training are referred to as FGSM, IFGSM, PGD, and their variants [2]. These methods utilize the model's gradient properties to generate adversaries on the Lo distance metric [2]. Therefore, the algorithm will consider the maximum perturbed input values to measure the significance of the adversarial input compared to the clean input counterpart. Since the perturbation noises are generated using the inner parameters of the victim model, the adversaries generated should be consistent if the model stays the same. The general benefit is that these methods use the internal information of the models to produce relatively efficient and consistent adversaries [2]. However, with the growing interest in this research area, more adversarial attacks could generate adversaries based on different distance metrics [2], which might be unreachable via this type of adversary generation [19]. On the other hand, black-box attack methods are beneficial for adversarial training. The biggest advantage of black-box attacks is that they generally require less information to generate the adversaries; therefore, they are more flexible and model-independent [2]. Furthermore, black-box adversarial attacks usually consider more distance metrics in their perturbation computations [15, 16]. Hence, adversarial training based on these methods could provide model robustness against adversarial samples based on these metrics.