# Adversarial Training Methods for Deep Learning: A Systematic Review
## 1 Introduction  
Neural networks are one of the most popular machine learning models, and they have been used in many applications, such as image classification, natural language processing, and other real-time applications [1]. Hence, ensuring the security of neural network models is critical and of substantial importance [2]. Adversarial attacks [2] pose significant threats to existing neural network models, and these attacks could perturb the model's inputs and cause the model to produce unexpected output results [3]. The perturbed inputs are usually called adversarial samples. Adversarial samples are the input data computed by an adversarial attack algorithm to make a classifier model misclassify the sample [4]. Normally, an adversarial input sample \(x\)' is computed from a clean input \(x\) with a restriction value, called epsilon \(\varepsilon\). \(\varepsilon\) is a variable that controls the strength of the perturbation [5]. The adversarial attack algorithm is responsible for finding the adversarial sample from a clean sample with \(\varepsilon\) perturbation. Adversarial training [5] is one of the most promising defensive methods to improve the robustness of a model by reducing the malicious effect caused by adversarial attacks. In general, a training model is able to generalize based on the adversarial sample it was trained against [3, 5, 6]. Hence, the quality of the adversarial sample provided during the training is important. In recent years, more adversarial training variations have been developedto tackle different shortcomings associated with adversarial training defense methods, such as reducing overfitting, improving generalization, and improving the efficiency of training. However, these problems still exist and there is no guarantee of robustness against adversarial attacks [1], especially when new attack methods are currently being proposed. In this work, we aim to understand the performance and effectiveness of adversarial training methods. We are particularly interested in the adversarial sample generation components of adversarial training methods. This is important because it is necessary to understand what kind of adversarial samples a model can be trained and generalized on and what the extent of the robustness of the current adversarial training methods is. Normally, adversarial sample generation methods are modified versions of adversarial attacks that are made to fit training schemas [5]. There are some other types of proposals that use non-conventional attack methods in their training schemas, such as generative models or other algorithms [7]. We aim to gather information from existing studies and to analyze the robustness and efficiency improvements and other potential benefits that were brought by those adversary generation components. In addition, we intend to identify the shortcomings, including overfitting, efficiency problems, unachievable robustness, and other research gaps of the adversarial training schema under study. The end goal of this work is to provide the research community with information about the status of the current development of adversarial training, and the adaption of the newly developed adversarial attacks or other newly developed adversary generation methods. Hence, this work can be viewed as a roadmap to guide other developers and researchers to develop more advanced adversarial sample generation methods to benefit adversarial training. In the literature, there are few surveys related to the topic of adversarial training. [PERSON] et al. [1] conducted a literature survey of the current advances in adversarial attacks and adversarial defense methods. They described the defensive method in three different categories: gradient masking/obfuscation, robust optimization, and adversarial example detection. They analyzed and discussed the advantages and limitations of each category. The review paper from [PERSON] et al.