# A Mask-Based Adversarial Defense Scheme
## 2 Related Work
### Adversarial Defense  
Adversarial defense aims to counter the adversarial effect and make DNNs achieve a performance on adversarial samples close to results on clean samples. Existing approaches can be roughly classified as (1) use of additional denoising structures before the DNN, and (2) enforcing the DNN to become more robust against adversarial attacks.  
Additional denoising structures. The reactive approach focuses on the detection of adversarial inputs (e.g., [5; 6; 7]) and tries to \"correct\" the adversarial inputs. MagNet [8] is a framework consisting of a detector that rejects adversarial samples that are far away from the normal input manifolds, and a reformer that finds the closest normal input if the adversarial input is not far from the manifolds. A similar approach is HGD [14], which applies a high-level representation-guided denoiser, so that it can be designed as a defense model that transforms adversarial inputs to easy-to-classify inputs. The defense-GAN [15] tries to model the distribution of clean images, and it can find a close output to an adversarial image which does not contain the adversarial changes. Denoised smoothing [16] prepends a custom-trained denoiser to a DNN and uses randomized smoothing for training the combination which enforces a nonlinear Lipschitz property. PixelDefend [17] approximates the training distribution by using a PixelCNN model and purifies images toward higher probability areas of the training distribution. Compared with our method, all these approaches introduce additional network structures that help to remove adversarial noise from inputs at the cost of increasing the number of parameters of the working model. In our experiment, we also empirically show that MAD outperforms MagNet and denoised smoothing in most of the test cases.  
DNN model enhancement. The proactive approach, sometimes known as adversarial defense, takes steps to strengthen DNNs (e.g., [8; 9; 10]), making them more robust to withstand perturbations on inputs. In this category, adversarial training [9; 18] and defensive distillation [4] are among the early successful approaches. In particular, adversarial training introduces adversarial samples at the training phase to enhance resistance against attacks. Defensive distillation transfers a DNN to another with the same functionality but is less sensitive to input perturbations. However, both adversarial training and defensive distillation require substantially more training cost. Introducing randomized variation to inputs and network parameters at the time of training as been discussed in [19; 20]. Parseval networks [21] limit the Lipschitz constant of linear, convolutional, and aggregation layers in the network to be smaller than 1, making these layers tight frames. PCL [10] separates the hidden feature of different categories by designing a new loss function and bypass network, so as to increase the difficulty of adversarial attacks and achieve the effect of adversarial defense. Because the existing DNN models are already being optimized with regard to performance (e.g., neural network search (NAS) [22] technology is widely used to find neural networks structure with better performance), the modification of DNN structures may introduce human bias, resulting in unpredictable performance degradation. In contrast, our approach MAD does not require any change to network structure or loss function design.