How to Visually Explain any CNN based Models | by Renu Khandelwal | Towards Data Science

CNN Deep Learning Models — Why it interpreted what it interpreted?
Deep Learning models are now able to give very high accuracy. The most critical piece for adopting computer vision algorithms at scale for Image Classification, Object Detection, Semantic Segmentation, Image Captioning, or Visual Question-Answer is understanding why the CNN model interpreted what they interpreted.
Explainability or Interpretability of a CNN model is the key to build the trust and its adoption
Only if we understand why the model failed to identify a class or an object, then we can concentrate our efforts to address the failure, of the model. Better explainable or interpretable deep learning models will help humans build trust and lead to higher adoption rates.
A good explainable or interpretable model should highlight fine-grained details in the image to visually explain why a class was predicted by the model.
Several methods explain the CNN models like
Guided backpropagation visualizes fine-grained details in the image. Its premise is: neurons act like detectors of particular image features, so when backpropagation, the gradient, negative gradients are set to zero to highlight the pixels that are important in the image.
Class Activation Maps(CAM) are class discriminative, where it localizes the category or class of the image. CAM requires feature maps to directly precede the prediction layer. CAM is thus applicable to CNN architectures performing global average pooling over convolutional maps immediately prior to the prediction layer and hence not generic to other computer vision algorithms.
Grad CAM visualizations are class-discriminative and localize relevant image regions but do not highlight the fine-grained pixel importance like guided backpropagation; however, unlike CAM, Grad CAM applies to any CNN architecture.
Guided Grad CAM combines the best of Grad CAM, which is class-discriminative and localizes relevant image regions, and Guided Backpropagation, which visualizes gradients with respect to the image where negative gradients set to zero to highlight import pixel in the image when backpropagating through ReLU layers.
Guided Gradient Weighted Class Activation Map(Guided Grad CAM) is class discriminative as well as highlights fine-grained important regions of an image for prediction in high resolution for any CNN Architecture.
An image is forward propagated through the CNN network generating feature maps, and then a task-specific network is applied to obtain the score for the category of the class.
Different types of tasks can be Image Classification, Image Captioning, Object Detection, Visual Question-Answer, etc.
The gradient for the desired class is set to 1, and the gradients for all the other classes are set to zero. This is done to obtain the class discriminative localization map.
The gradients with respect to the activations are backpropagated till the final convolutional layer; this captures the important features. A weighted combination of the forward activation map and backpropagated gradients is followed by a ReLU to obtain the Grad CAM Localization (blue heatmap) that explains the feature the model looks at to make a particular decision.
Apply guided backpropagation to compute gradients and then zero out negatives gradients and backpropagate.
As a final step, apply pointwise multiplication of the Grad CAM heatmap with guided backpropagation to get Guided Grad-CAM visualizations. These visualizations are now class discriminative as well as localizes relevant image regions with high-resolution.
In the code below, we will forward propagate the image through the Conv Layers of the VGG19 model.
Non-negative gradients are backpropagated to the rectified convolutional feature maps of interest. These gradients are then combined to compute the coarse Grad-CAM localization, representing the features in the image where the model looks to make the particular decision.
Finally, we apply pointwise multiplication of the Grad CAM heatmap with guided backpropagation to get Guided Grad-CAM visualizations, which are both high-resolution and concept-specific or class discriminant.
# Import the required libraraiesimport cv2import tensorflow as tfimport tensorflow.keras.backend as Kimport numpy as npfrom skimage.transform import resizeimport matplotlib.pyplot as plt%matplotlib inline#Setting the image apth and the last conv layer for VGG19IMAGE_PATH = 'cat_and_dog1.jpg'LAYER_NAME='block5_conv4'#Load the imageimg = tf.keras.preprocessing.image.load_img(IMAGE_PATH, target_size=(224, 224))# Displaying the original imageplt.axis("off")plt.imshow(img)plt.show()# Preprocess the image using vgg19 preprocess functionimg =  tf.keras.preprocessing.image.img_to_array(img)x = np.expand_dims(img, axis=0)preprocessed_input = tf.keras.applications.vgg19.preprocess_input(x)
We are using the VGG19 transfer learned model on the Imagenet dataset.
#Create the transfer learned modelmodel = tf.keras.applications.vgg19.VGG19(weights='imagenet', include_top=True)
We create a model from the VGG19 transfer learned model until the last convolutional layer. Conv layers retain the spatial information which is lost when we apply the fully connected layer in the deep learning CNN model.
'''create a model till  last convolutional layers to have the best compromise between high-level semantics and detailed spatialinformation'''gb_model = tf.keras.models.Model(    inputs = [model.inputs],        outputs = [model.get_layer(LAYER_NAME).output])layer_dict = [layer for layer in gb_model.layers[1:] if hasattr(layer,'activation')]
Gradients of ReLU are overridden by applying @tf.custom_gradient which allows fine-grained control over the gradients for backpropagating non-negative gradients to have a more efficient or numerically stable gradient.
@tf.custom_gradientdef guidedRelu(x):  def grad(dy):    return tf.cast(dy>0,"float32") * tf.cast(x>0, "float32") * dy  return tf.nn.relu(x), grad
Apply guided ReLU to all the Conv layers where activation was ReLU
for layer in layer_dict:  if layer.activation == tf.keras.activations.relu:    layer.activation = guidedRelu
Use Gradient Tape to record the preprocessed input image for the forward pass which will help calculate the gradients for the backward pass. Use the GradientTape object to capture the gradients on the last Conv layer.
Here we find the gradients of the target class score with respect to the feature maps of the last convolutional layer
with tf.GradientTape() as tape: inputs = tf.cast(preprocessed_input, tf.float32) tape.watch(inputs) outputs = gb_model(inputs)[0]grads = tape.gradient(outputs,inputs)[0]
Average the gradients spatially where each entry is the mean intensity of the gradient over a specific feature map channel. Build a ponderated map of filters according to gradient importance. The result is the final class discriminative saliency map.
weights = tf.reduce_mean(grads, axis=(0, 1))grad_cam = np.ones(outputs.shape[0: 2], dtype = np.float32)for i, w in enumerate(weights):    grad_cam += w * outputs[:, :, i]
Divide each intensity value of the heatmap with the maximum intensity value to normalize the heatmap such that all values fall between 0 and 1 and plot the Gradient Class Activation map on the image
grad_cam_img = cv2.resize(grad_cam.numpy(), (224, 224))grad_cam_img = np.maximum(grad_cam_img, 0)heatmap = (grad_cam_img - grad_cam_img.min()) / (grad_cam_img.max() - grad_cam_img.min())grad_cam_img = cv2.applyColorMap(np.uint8(255*heatmap), cv2.COLORMAP_JET)output_image = cv2.addWeighted(cv2.cvtColor(img.astype('uint8'), cv2.COLOR_RGB2BGR), 0.5, grad_cam_img, 1, 0)plt.imshow(output_image)plt.axis("off");plt.show()
Using Guided backpropagation to capture pixels detected by neurons, not the ones that suppress neurons. Guided backpropagation computes the gradient of the target output with respect to the input where negative gradients are suppressed when backpropagating through ReLU layers.
#Visualizing the guided back propguided_back_prop =gradsgb_viz = np.dstack((            guided_back_prop[:, :, 0],            guided_back_prop[:, :, 1],            guided_back_prop[:, :, 2],        ))       gb_viz -= np.min(gb_viz)gb_viz /= gb_viz.max()    imgplot = plt.imshow(gb_viz)plt.axis("off")plt.show()
Guided Grad CAM combines the best aspects of Grad CAM and Guided Backpropagation via element-wise multiplication. The visualization for Guided Grad CAM is both high-resolution and class-discriminative
guided_cam = np.maximum(grad_cam, 0)guided_cam = guided_cam / np.max(guided_cam) # scale 0 to 1.0guided_cam = resize(guided_cam, (224,224), preserve_range=True)#pointwise multiplcation of guided backprop and grad CAM gd_gb = np.dstack((        guided_back_prop[:, :, 0] * guided_cam,        guided_back_prop[:, :, 1] * guided_cam,        guided_back_prop[:, :, 2] * guided_cam,    ))imgplot = plt.imshow(gd_gb)plt.axis("off")plt.show()
The visualization highlights the pixels for the ears, the nose, the body of the dog, and its paws that the model looks to make the prediction for the dog and the cat features are not highlighted.
Guided Grad CAM is class discriminative, localizes relevant image regions while highlighting fine-grained pixel details. Visualization using Guided Grad CAM helps explain what the CNN model looks at for making the prediction thus explaining why the model interpreted what it interpreted and thus clearly building the trust in deep learning models.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization