Package: torchcam
TorchCAM: class activation explorerSimple way to leverage the class-specific activation of convolutional layers in PyTorch.Source: image fromwoopets(activation maps created with a pretrainedResnet-18)Quick TourSetting your CAMTorchCAM leveragesPyTorch hooking mechanismsto seamlessly retrieve all required information to produce the class activation without additional efforts from the user. Each CAM object acts as a wrapper around your model.You can find the exhaustive list of supported CAM methods in thedocumentation, then use it as follows:# Define your modelfromtorchvision.modelsimportresnet18model=resnet18(pretrained=True).eval()# Set your CAM extractorfromtorchcam.methodsimportSmoothGradCAMppcam_extractor=SmoothGradCAMpp(model)Please note that by default, the layer at which the CAM is retrieved is set to the last non-reduced convolutional layer. If you wish to investigate a specific layer, use thetarget_layerargument in the constructor.Retrieving the class activation mapOnce your CAM extractor is set, you only need to use your model to infer on your data as usual. If any additional information is required, the extractor will get it for you automatically.fromtorchvision.io.imageimportread_imagefromtorchvision.transforms.functionalimportnormalize,resize,to_pil_imagefromtorchvision.modelsimportresnet18fromtorchcam.methodsimportSmoothGradCAMppmodel=resnet18(pretrained=True).eval()# Get your inputimg=read_image("path/to/your/image.png")# Preprocess it for your chosen modelinput_tensor=normalize(resize(img,(224,224))/255.,[0.485,0.456,0.406],[0.229,0.224,0.225])withSmoothGradCAMpp(model)ascam_extractor:# Preprocess your data and feed it to the modelout=model(input_tensor.unsqueeze(0))# Retrieve the CAM by passing the class index and the model outputactivation_map=cam_extractor(out.squeeze(0).argmax().item(),out)If you want to visualize your heatmap, you only need to cast the CAM to a numpy ndarray:importmatplotlib.pyplotasplt# Visualize the raw CAMplt.imshow(activation_map[0].squeeze(0).numpy());plt.axis('off');plt.tight_layout();plt.show()Or if you wish to overlay it on your input image:importmatplotlib.pyplotaspltfromtorchcam.utilsimportoverlay_mask# Resize the CAM and overlay itresult=overlay_mask(to_pil_image(img),to_pil_image(activation_map[0].squeeze(0),mode='F'),alpha=0.5)# Display itplt.imshow(result);plt.axis('off');plt.tight_layout();plt.show()SetupPython 3.8 (or higher) andpip/condaare required to install TorchCAM.Stable releaseYou can install the last stable release of the package usingpypias follows:pipinstalltorchcamor usingconda:condainstall-cfrgfmtorchcamDeveloper installationAlternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source:gitclonehttps://github.com/frgfm/torch-cam.git
pipinstall-etorch-cam/.CAM ZooThis project is developed and maintained by the repo owner, but the implementation was based on the following research papers:Learning Deep Features for Discriminative Localization: the original CAM paperGrad-CAM: GradCAM paper, generalizing CAM to models without global average pooling.Grad-CAM++: improvement of GradCAM++ for more accurate pixel-level contribution to the activation.Smooth Grad-CAM++: SmoothGrad mechanism coupled with GradCAM.Score-CAM: score-weighting of class activation for better interpretability.SS-CAM: SmoothGrad mechanism coupled with Score-CAM.IS-CAM: integration-based variant of Score-CAM.XGrad-CAM: improved version of Grad-CAM in terms of sensitivity and conservation.Layer-CAM: Grad-CAM alternative leveraging pixel-wise contribution of the gradient to the activation.Source:YouTube video(activation maps created byLayer-CAMwith a pretrainedResNet-18)What elseDocumentationThe full package documentation is availableherefor detailed specifications.Demo appA minimal demo app is provided for you to play with the supported CAM methods! Feel free to check out the live demo onIf you prefer running the demo by yourself, you will need an extra dependency (Streamlit) for the app to run:pip install -e ".[demo]"You can then easily run your app in your default browser by running:streamlit run demo/app.pyExample scriptAn example script is provided for you to benchmark the heatmaps produced by multiple CAM approaches on the same image:pythonscripts/cam_example.py--archresnet18--class-idx232--rows2All script arguments can be checked usingpython scripts/cam_example.py --helpLatency benchmarkYou crave for beautiful activation maps, but you don't know whether it fits your needs in terms of latency?In the table below, you will find a latency benchmark (forward pass not included) for all CAM methods:CAM methodArchGPU mean (std)CPU mean (std)CAMresnet180.11ms (0.02ms)0.14ms (0.03ms)GradCAMresnet183.71ms (1.11ms)40.66ms (1.82ms)GradCAMppresnet185.21ms (1.22ms)41.61ms (3.24ms)SmoothGradCAMppresnet1833.67ms (2.51ms)239.27ms (7.85ms)ScoreCAMresnet18304.74ms (11.54ms)6796.89ms (415.14ms)SSCAMresnet18ISCAMresnet18XGradCAMresnet183.78ms (0.96ms)40.63ms (2.03ms)LayerCAMresnet183.65ms (1.04ms)40.91ms (1.79ms)CAMmobilenet_v3_largeN/A*N/A*GradCAMmobilenet_v3_large8.61ms (1.04ms)26.64ms (3.46ms)GradCAMppmobilenet_v3_large8.83ms (1.29ms)25.50ms (3.10ms)SmoothGradCAMppmobilenet_v3_large77.38ms (3.83ms)156.25ms (4.89ms)ScoreCAMmobilenet_v3_large35.19ms (2.11ms)679.16ms (55.04ms)SSCAMmobilenet_v3_largeISCAMmobilenet_v3_largeXGradCAMmobilenet_v3_large8.41ms (0.98ms)24.21ms (2.94ms)LayerCAMmobilenet_v3_large8.02ms (0.95ms)25.14ms (3.17ms)*The base CAM method cannot work with architectures that have multiple fully-connected layersThis benchmark was performed over 100 iterations on (224, 224) inputs, on a laptop to better reflect performances that can be expected by common users. The hardware setup includes anIntel(R) Core(TM) i7-10750Hfor the CPU, and aNVIDIA GeForce RTX 2070 with Max-Q Designfor the GPU.You can run this latency benchmark for any CAM method  on your hardware as follows:pythonscripts/eval_latency.pySmoothGradCAMppAll script arguments can be checked usingpython scripts/eval_latency.py --helpExample notebooksLooking for more illustrations of TorchCAM features?
You might want to check theJupyter notebooksdesigned to give you a broader overview.CitationIf you wish to cite this project, feel free to use thisBibTeXreference:@misc{torcham2020,title={TorchCAM: class activation explorer},author={François-Guillaume Fernandez},year={2020},month={March},publisher={GitHub},howpublished={\url{https://github.com/frgfm/torch-cam}}}ContributingFeeling like extending the range of possibilities of CAM? Or perhaps submitting a paper implementation? Any sort of contribution is greatly appreciated!You can find a short guide inCONTRIBUTINGto help grow this project!LicenseDistributed under the Apache 2.0 License. SeeLICENSEfor more information.