Adversarial Attack and Defense on Neural Networks in PyTorch | by Ta-Ying Cheng | Towards Data Science

The rise of deep learning and neural networks brought various opportunities and applications such as object detection and text-to-speech into the modern society. Yet, despite the seemingly high accuracy, neural networks (and almost all machine learning models) could actually suffer from data, namely adversarial examples, that are manipulated very slightly from original training samples. In fact, past researches have indicated that as long as you know the “correct” method to change your data, you can force your network to perform poorly on data which may not seem to be visually different through human eyes! These deliberate manipulations of the data to lower model accuracies are called adversarial attacks, and the war of attack and defense is an ongoing popular research topic in the machine learning domain.
This article will provide an overview on one of the easiest yet effective attacks — Fast Gradient Signed Method attack — along with its implementation in and defense through adversarial training in PyTorch.
Side Note: This article assumes prior knowledge in building simple neural networks and training them in PyTorch. If you are not familiar with them it is recommended to first checkout tutorials on PyTorch first.
Adversarial examples can be defined as inputs or data that are perturbed in order to fool a machine learning network. This idea was formulated by Ian et al. in his paper “Explaining and Harnessing Adversarial Examples” from ICLR 2015 conference. While publications before this paper claimed that these adversarial examples were caused by nonlinearity and overfitting of machine models, Ian et al. argued that neural networks are in fact vulnerable to these examples due to the high linearity of the architecture. Models such as LSTMs and activation functions such as ReLU still often behave in a very linear way, and hence these models would be very easily fooled by linear perturbations. He then followed up by providing a simple and fast one-step method of generating adversarial examples: Fast Gradient Sign Method.
The Fast Gradient Sign Method (FGSM) is a white-box attack, meaning the attack is generated based on a given network architecture. FGSM is based on the idea that normal networks follows a gradient descent to find the lowest point of loss, and hence if we follow the sign of the gradient (going the opposite direction from the gradient descent), we can maximise the loss by just adding a small amount of perturbation.
FGSM can hence be described as the following mathematical expression:
where x’ is the perturbed x that is generated by adding a small constant ε with the sign equal to the direction of the gradient of loss J with respect to x. Figure 1 is the classic illustration of a FGSM attack in the computer vision domain. With a less than 1% change in the image that isn’t visually recognisable by us, the image went from correctly classified with a mediocre confidence to falsely classified with a high confidence.
The fact that these simple methods can actually fool a deep neural network is a further evidence that adversarial examples exist because of neural network’s linearity.
To build the FGSM attack in PyTorch, we can use the CleverHans library provided and carefully maintained by Ian Goodfellow and Nicolas Papernot. The library provides multiple attacks and defenses and is widely used today for benchmarking. Although the majority of attacks were implemented in Tensorflow, they recently released the codes for FGSM in PyTorch as well.
The library can be downloaded and installed with the following command:
pip install git+https://github.com/tensorflow/cleverhans.git#egg=cleverhans
We will use the simple MNIST dataset to demonstrate how to build the attack.
Firstly, we have to create an ordinary PyTorch model and data loader for the MNIST dataset.
For demonstration, we will build a simple convolutional network as the following:
and the data loader as the following:
Afterwards, we implement a normal forward method to train the network on normal data:
By setting the batch size to 128, number of epochs to 4, and learning rate to 0.001, the network successfully achieves an accuracy of around 98% on the MNIST dataset after training.
After training the network, we can then apply the FGSM attack given the network architecture.
To do so, we have to first import the required functions from CleverHans:
from cleverhans.future.torch.attacks.fast_gradient_method import fast_gradient_method
This allows us to call the fast_gradient_method() function, which is simple and straightforward: Given the model, an input x, an ε, and a norm (norm=np.inf, 1, or 2), the function outputs a perturbed x.
We can then slightly change the original forward function by feeding the perturbed x instead of the original x to measure the results as the following:
The above attack, after testing, can actually force the accuracy to drop drastically from 98% to around 4%, proving that small perturbations, if on the correct direction, will actually lead to the network performing very poorly.
In the same paper by Ian et al, they proposed the adversarial training method to combat these samples. In simple words, the adversarial samples generated from the training set were also included in the training.
This concept can be easily implemented into the code by feeding both the original and the perturbed training set into the architecture at the same time. Note that both types of data should be used for adversarial training to prevent the loss in accuracy on the original set of data. The code below is my implementation of adversarial training:
Note that the network starts from the checkpoint where it is already trained on clean data. Both the clean and adversarial examples are fed into the network during adversarial training to prevent an accuracy decrease on clean data during further training.
With the same batch size, epochs, and learning rate settings, we could actually increase the accuracy back to approximately 90% for adversarial examples while maintaining the accuracy on clean data.
Although the aforementioned example illustrates how adversarial training could be adopted to generalise the model architecture, one main issue is that they will only be effective on a specific type of attack that the model is trained on. With different attacks generating different adversarial examples, the adversarial training method needs to be further investigated and evaluated for better adversarial defense.
FGSM and adversarial training are one of the earliest attacks and defenses. Recent attacks such as the C&W attack and DeepFool and defenses such as distillation have opened up new opportunities for future research and investigation. This article serves as an introduction to the field of adversarial attacks and hopefully sparks your interest to dig deeper into this field! To learn more, here is another article that I think is wonderful for a short read for better understanding on the fast gradient sign method.
The full code of my implementation is also posted in my Github:
github.com
Thank you for making it this far 🙏! I will be posting more on different areas of computer vision/deep learning, make sure to check out my other articles and articles by Chuan En Lin too!