Saliency Map Using PyTorch | Towards Data Science

When we use machine learning models such as Logistic Regression or Decision Tree, we can interpret which variables contribute to the prediction result. But, this step will be difficult when we are using deep learning model.
Deep Learning model is a black-box model. It means that we cannot analyze how the model can predict the result based on the data. As the model gets more complex, the interpretability of the model will reduce.
But this doesn’t mean that the model cannot be interpreted. We can infer the deep learning model, but we have to work on its complexity.
In this article, I will show you how to visualize deep learning model result based on gradients. We call this method as saliency map. We will use a framework called PyTorch to implement this method.
Without further ado, let’s get started!
Before we get into the saliency map, let’s talk about the image classification. Given this equation,
Where,
Sc(I) is the score of the class c
I corresponds to the image (on one-dimensional vector).
w and b correspond to the weight and bias for the class c.
As you can see on the equation, we multiply the image vector (I) with the weight vector (w). We can infer that the weight (w) can define the importance for each pixel on the image to the class (c).
So, how can we get the weight (w) that correspond to score (S) given to the image (I)? We can get the relationship between those variables by looking at the gradients!
But wait, what does basically a gradient? I don’t want to explain so many calculus on it, but I will tell you this.
The gradient describes how much the effect of a variable, let’s say x, can affect another variable result, in this case, y.
When we use that analogy, we can say that the gradient describes on how strong for each pixel of the image (I) can contribute to the prediction result (S). The equation for looking the weights (w) looks like this,
By knowing the weights (w) for each pixel, we can visualize it as a saliency map, where each pixel of it describes the power of that pixel affects the prediction result.
Now, let’s get into the implementation!
In this section, we will implement the saliency map using PyTorch. The deep learning model that we will use has trained for a Kaggle competition called Plant Pathology 2020 — FGVC7. To download the dataset, you access on the link here.
Here are the steps that we have to do,
Set up the deep learning model
Open the image
Preprocess the image
Retrieve the gradient
Visualize the result
Now, the first thing that we have to do is to set up the model. In this case, I will use my pretrained model weight using ResNet-18 architecture. Also, I’ve set up the model so we can use the GPU to get the result. The code looks like this,
Right after we set up the model, now we can set up the image. To do this, we will use PIL and torchvision libraries for transform that image. The code looks like this,
After we transform the image, we have to reshape it because our model reads the tensor on 4-dimensional shape (batch size, channel, width, height). The code looks like this,
# Reshape the image (because the model use # 4-dimensional tensor (batch_size, channel, width, height))image = image.reshape(1, 3, 224, 224)
After we reshape the image, now we set our image to run on GPU. The code looks like this,
# Set the device for the imageimage = image.to(device)
Then, we have to set the image to catch gradient when we do backpropagation to it. The code looks like this,
# Set the requires_grad_ to the image for retrieving gradientsimage.requires_grad_()
After that, we can catch the gradient by put the image on the model and do the backpropagation. The code looks like this,
Now, we can visualize the gradient using matplotlib. But there is one task that we have to do. The image has three channels to it. Therefore, we have to take the maximum value from those channels on each pixel position. Finally, we can visualize the result using the code looks like this,
Here is the visualization looks like,
As you can see from the image above, the left side is the image, and the right size is the saliency map. Recall from its definition the saliency map will show the strength for each pixel contribution to the final output.
In this case, the leaf on this image has a disease called rust as you can see on the yellow spot on it. And if you look carefully, some pixels has a brighter color than any other images. It indicates that those pixels have a huge contribution to the final result, which is the rust itself.
Therefore, we can confidently say that the model has predicted the result by looking at the right information of it.
Well done! Now you have to implement your own saliency map for interpreting your deep learning model. I hope you can implement the saliency map for your own cases, and don’t forget to follow me on Medium!
If you want to have a discussion with me about data science or machine learning, you can contact me on LinkedIn. Thank you for reading my article!
[1] Simonyan K, Vedaldi A, Zisserman A. 2014. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv:1312.6034v2 [cs.CV] [2] Rastogi A. 2020. Visualizing Neural Networks using Saliency Maps in PyTorch. Data Driven Investor. [3] https://github.com/sijoonlee/deep_learning/blob/master/cs231n/NetworkVisualization-PyTorch.ipynb