Package: grad-cam
Advanced AI explainability for PyTorchpip install grad-camDocumentation with advanced tutorials:https://jacobgil.github.io/pytorch-gradcam-bookThis is a package with state of the art methods for Explainable AI for computer vision.
This can be used for diagnosing model predictions, either in production or while
developing models.
The aim is also to serve as a benchmark of algorithms and metrics for research of new explainability methods.⭐ Comprehensive collection of Pixel Attribution methods for Computer Vision.⭐ Tested on many Common CNN Networks and Vision Transformers.⭐ Advanced use cases: Works with Classification, Object Detection, Semantic Segmentation, Embedding-similarity and more.⭐ Includes smoothing methods to make the CAMs look nice.⭐ High performance: full support for batches of images in all methods.⭐ Includes metrics for checking if you can trust the explanations, and tuning them for best performance.MethodWhat it doesGradCAMWeight the 2D activations by the average gradientHiResCAMLike GradCAM but element-wise multiply the activations with the gradients; provably guaranteed faithfulness for certain modelsGradCAMElementWiseLike GradCAM but element-wise multiply the activations with the gradients then apply a ReLU operation before summingGradCAM++Like GradCAM but uses second order gradientsXGradCAMLike GradCAM but scale the gradients by the normalized activationsAblationCAMZero out activations and measure how the output drops (this repository includes a fast batched implementation)ScoreCAMPerbutate the image by the scaled activations and measure how the output dropsEigenCAMTakes the first principle component of the 2D Activations (no class discrimination, but seems to give great results)EigenGradCAMLike EigenCAM but with class discrimination: First principle component of Activations*Grad. Looks like GradCAM, but cleanerLayerCAMSpatially weight the activations by positive gradients. Works better especially in lower layersFullGradComputes the gradients of the biases from all over the network, and then sums themDeep Feature FactorizationsNon Negative Matrix Factorization on the 2D activationsVisual ExamplesWhat makes the network think the image label is 'pug, pug-dog'What makes the network think the image label is 'tabby, tabby cat'Combining Grad-CAM with Guided Backpropagation for the 'pug, pug-dog' classObject Detection and Semantic SegmentationObject DetectionSemantic SegmentationExplaining similarity to other images / embeddingsDeep Feature FactorizationClassificationResnet50:CategoryImageGradCAMAblationCAMScoreCAMDogCatVision Transfomer (Deit Tiny):CategoryImageGradCAMAblationCAMScoreCAMDogCatSwin Transfomer (Tiny window:7 patch:4 input-size:224):CategoryImageGradCAMAblationCAMScoreCAMDogCatMetrics and Evaluation for XAIChoosing the Target LayerYou need to choose the target layer to compute CAM for.
Some common choices are:FasterRCNN: model.backboneResnet18 and 50: model.layer4[-1]VGG and densenet161: model.features[-1]mnasnet1_0: model.layers[-1]ViT: model.blocks[-1].norm1SwinT: model.layers[-1].blocks[-1].norm1If you pass a list with several layers, the CAM will be averaged accross them.
This can be useful if you're not sure what layer will perform best.Using from code as a libraryfrompytorch_grad_camimportGradCAM,HiResCAM,ScoreCAM,GradCAMPlusPlus,AblationCAM,XGradCAM,EigenCAM,FullGradfrompytorch_grad_cam.utils.model_targetsimportClassifierOutputTargetfrompytorch_grad_cam.utils.imageimportshow_cam_on_imagefromtorchvision.modelsimportresnet50model=resnet50(pretrained=True)target_layers=[model.layer4[-1]]input_tensor=# Create an input tensor image for your model..# Note: input_tensor can be a batch tensor with several images!# Construct the CAM object once, and then re-use it on many images:cam=GradCAM(model=model,target_layers=target_layers)# You can also use it within a with statement, to make sure it is freed,# In case you need to re-create it inside an outer loop:# with GradCAM(model=model, target_layers=target_layers) as cam:#   ...# We have to specify the target we want to generate# the Class Activation Maps for.# If targets is None, the highest scoring category# will be used for every image in the batch.# Here we use ClassifierOutputTarget, but you can define your own custom targets# That are, for example, combinations of categories, or specific outputs in a non standard model.targets=[ClassifierOutputTarget(281)]# You can also pass aug_smooth=True and eigen_smooth=True, to apply smoothing.grayscale_cam=cam(input_tensor=input_tensor,targets=targets)# In this example grayscale_cam has only one image in the batch:grayscale_cam=grayscale_cam[0,:]visualization=show_cam_on_image(rgb_img,grayscale_cam,use_rgb=True)# You can also get the model outputs without having to re-inferencemodel_outputs=cam.outputsMetrics and evaluating the explanationsfrompytorch_grad_cam.utils.model_targetsimportClassifierOutputSoftmaxTargetfrompytorch_grad_cam.metrics.cam_mult_imageimportCamMultImageConfidenceChange# Create the metric target, often the confidence drop in a score of some categorymetric_target=ClassifierOutputSoftmaxTarget(281)scores,batch_visualizations=CamMultImageConfidenceChange()(input_tensor,inverse_cams,targets,model,return_visualization=True)visualization=deprocess_image(batch_visualizations[0,:])# State of the art metric: Remove and Debiasfrompytorch_grad_cam.metrics.roadimportROADMostRelevantFirst,ROADLeastRelevantFirstcam_metric=ROADMostRelevantFirst(percentile=75)scores,perturbation_visualizations=cam_metric(input_tensor,grayscale_cams,targets,model,return_visualization=True)# You can also average accross different percentiles, and combine# (LeastRelevantFirst - MostRelevantFirst) / 2frompytorch_grad_cam.metrics.roadimportROADMostRelevantFirstAverage,ROADLeastRelevantFirstAverage,ROADCombinedcam_metric=ROADCombined(percentiles=[20,40,60,80])scores=cam_metric(input_tensor,grayscale_cams,targets,model)Advanced use cases and tutorials:You can use this package for "custom" deep learning models, for example Object Detection or Semantic Segmentation.You will have to define objects that you can then pass to the CAM algorithms:A reshape_transform, that aggregates the layer outputs into 2D tensors that will be displayed.Model Targets, that define what target do you want to compute the visualizations for, for example a specific category, or a list of bounding boxes.Here you can find detailed examples of how to use this for various custom use cases like object detection:These point to the new documentation jupter-book for fast rendering.
The jupyter notebooks themselves can be found under the tutorials folder in the git repository.Notebook tutorial: XAI Recipes for the HuggingFace 🤗 Image Classification ModelsNotebook tutorial: Deep Feature Factorizations for better model explainabilityNotebook tutorial: Class Activation Maps for Object Detection with Faster-RCNNNotebook tutorial: Class Activation Maps for YOLO5Notebook tutorial: Class Activation Maps for Semantic SegmentationNotebook tutorial: Adapting pixel attribution methods for embedding outputs from modelsNotebook tutorial: May the best explanation win. CAM Metrics and TuningHow it works with Vision/SwinT transformersSmoothing to get nice looking CAMsTo reduce noise in the CAMs, and make it fit better on the objects,
two smoothing methods are supported:aug_smooth=TrueTest time augmentation: increases the run time by x6.Applies a combination of horizontal flips, and mutiplying the image
by [1.0, 1.1, 0.9].This has the effect of better centering the CAM around the objects.eigen_smooth=TrueFirst principle component ofactivations*weightsThis has the effect of removing a lot of noise.AblationCAMaug smootheigen smoothaug+eigen smoothRunning the example script:Usage:python cam.py --image-path <path_to_image> --method <method> --output-dir <output_dir_path>To use with a specific device, like cpu, cuda, cuda:0 or mps:python cam.py --image-path <path_to_image> --device cuda  --output-dir <output_dir_path>You can choose between:GradCAM,HiResCAM,ScoreCAM,GradCAMPlusPlus,AblationCAM,XGradCAM,LayerCAM,FullGradandEigenCAM.Some methods like ScoreCAM and AblationCAM require a large number of forward passes,
and have a batched implementation.You can control the batch size withcam.batch_size =CitationIf you use this for research, please cite. Here is an example BibTeX entry:@misc{jacobgilpytorchcam,
  title={PyTorch library for CAM methods},
  author={Jacob Gildenblat and contributors},
  year={2021},
  publisher={GitHub},
  howpublished={\url{https://github.com/jacobgil/pytorch-grad-cam}},
}Referenceshttps://arxiv.org/abs/1610.02391Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batrahttps://arxiv.org/abs/2011.08891Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networks Rachel L. Draelos, Lawrence Carinhttps://arxiv.org/abs/1710.11063Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks Aditya Chattopadhyay, Anirban Sarkar, Prantik Howlader, Vineeth N Balasubramanianhttps://arxiv.org/abs/1910.01279Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, Xia Huhttps://ieeexplore.ieee.org/abstract/document/9093360/Ablation-cam: Visual explanations for deep convolutional network via gradient-free localization. Saurabh Desai and Harish G Ramaswamy. In WACV, pages 972–980, 2020https://arxiv.org/abs/2008.02312Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, Biao Lihttps://arxiv.org/abs/2008.00299Eigen-CAM: Class Activation Map using Principal Components Mohammed Bany Muhammad, Mohammed Yeasinhttp://mftp.mmcheng.net/Papers/21TIP_LayerCAM.pdfLayerCAM: Exploring Hierarchical Class Activation Maps for Localization Peng-Tao Jiang; Chang-Bin Zhang; Qibin Hou; Ming-Ming Cheng; Yunchao Weihttps://arxiv.org/abs/1905.00780Full-Gradient Representation for Neural Network Visualization Suraj Srinivas, Francois Fleurethttps://arxiv.org/abs/1806.10206Deep Feature Factorization For Concept Discovery Edo Collins, Radhakrishna Achanta, Sabine Süsstrunk