# Defense Against Adversarial Attacks in Deep Learning
## 2 Noise Reconstruction Algorithm Based on GAN: GNR
### Network Structure  
For defending the adversarial attack, we use the generative adversarial network to model the noise. We first combine the original image and the attack image into a one-to-one sample pair. Then, based on the sample pair, we subtract the original image from the original image to get true noise data. Finally, by training GAN with true noise dataset, we obtain the distribution of the noise samples. As a framework of the generated model, GAN has the ability to learn complex distribution. More importantly, the GAN can generate noise by forward propagation in the generator network without involving another component. In our algorithm, the specific network structure of GAN is shown in Figure 1.  
In the generator network, the size of the second to fourth filters are set to 5 \(\times\) 5, which are 256, 128, and 64. The number of filters is equal to the number of channel outputs. The generator network consists of three DenconvUnit modules, each containing one deconvolution block and three convolution blocks. Each convolution or deconvolution layer is followed by a batch normalization layer and a ReLU layer. The connection in the generator network prevents the problem of gradient dispersion and degradation during network training. The generator's input is a random noise that satisfies the distribution. The output of the generator will be passed to the authentication network as input. In the discriminator network, the first to fourth filters are 64, 128, 256, and 512, respectively. The discriminator network consists of three ConvUnits, each of which includes a convolutional layer, followed with a batch normalization layer and a Leaky ReLU layer. The output of the discriminator is  
Figure 1: Generative adversarial network (GAN) structure. The whole structure consists of a generator network and a discriminator network. The generator consists of three DeconvUnits and the discriminator has three ConvUnits. The output of the generator is the input of the discriminator.  
the probability that the noise is true. The Wasserstein distance is used to define the loss function of the model, which is shown in Equation (3).  
\[L_{GAN}=\underset{\widetilde{x}\sim P_{g}}{E}[F(\widetilde{x})]-\underset{x\sim P _{r}}{E}[F(x)]+\lambda\underset{\widetilde{x}\sim P_{r}}{E}[(\parallel\
abla_{ \overline{x}}F(\overline{x})\parallel_{2}-1)^{2}] \tag{3}\]  
In Equation (3), \(\widetilde{x}\) represents the generated noise, \(x\) represents the real noise, and \(\overline{x}\) represents the noise randomly selected on the line between the generated noise and the real noise. \(F(\cdot)\) represents the output of the discriminator. \(P_{g}\) represents the distribution of the generator, \(P_{r}\) represents the distribution of the real noise samples, and \(P_{\overline{x}}\) represents the distribution of the samples of the random sampling between the real noise and the generated noise. The added penalty can help generate noise that tends to be true to noise and helps the generator generate more diverse noise. As the training progresses, the noise distribution generated by the generator will continue to approach the true noise distribution. We can directly generate noise through the trained generator network and realize the reconstructing of the noise in the adversarial sample.