Visualizing attention when using transfer learning method for images

Mona_Jalal: <p>I came across this paper but I wonder if I am using the PyTorch transfer learning tutorial, a code like</p><aside class="onebox githubblob" data-onebox-src="https://github.com/monajalal/Inception_V3-cats_and_dogs/blob/master/inception_v3_test.ipynb">
  <header class="source">

      <a href="https://github.com/monajalal/Inception_V3-cats_and_dogs/blob/master/inception_v3_test.ipynb" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/monajalal/Inception_V3-cats_and_dogs/blob/master/inception_v3_test.ipynb" target="_blank" rel="noopener nofollow ugc">monajalal/Inception_V3-cats_and_dogs/blob/master/inception_v3_test.ipynb</a></h4>


      <pre><code class="lang-ipynb">{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
</code></pre>



  This file has been truncated. <a href="https://github.com/monajalal/Inception_V3-cats_and_dogs/blob/master/inception_v3_test.ipynb" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
How can I apply this kinds of attention methodology to it since I don’t have access to the full network and I am using a model pretrained on ImageNet.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/e/7e13ef20ddf7d1286a5f3c7b5085cf5f39f1af11.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/7e13ef20ddf7d1286a5f3c7b5085cf5f39f1af11" title="Screenshot from 2022-03-29 17-50-33"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7e13ef20ddf7d1286a5f3c7b5085cf5f39f1af11_2_690x336.jpeg" alt="Screenshot from 2022-03-29 17-50-33" data-base62-sha1="hZkRVOs4Py6lZgmaWC3ahaaSKqt" width="690" height="336" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7e13ef20ddf7d1286a5f3c7b5085cf5f39f1af11_2_690x336.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7e13ef20ddf7d1286a5f3c7b5085cf5f39f1af11_2_1035x504.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7e13ef20ddf7d1286a5f3c7b5085cf5f39f1af11_2_1380x672.jpeg 2x" data-dominant-color="DADEE6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-03-29 17-50-33</span><span class="informations">1612×786 260 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>^ this is from this paper: Park, Jongchan, Sanghyun Woo, Joon-Young Lee, and In So Kweon. <a href="https://joonyoung-cv.github.io/assets/paper/20_ijcv_a_simple.pdf" rel="noopener nofollow ugc">“A simple and light-weight attention module for convolutional neural networks.”</a> <em>International Journal of Computer Vision</em> 128, no. 4 (2020): 783-798.</p>
<p>Also, Li, Xingjian, Haoyi Xiong, Zeyu Chen, Jun Huan, Ji Liu, Cheng-Zhong Xu, and Dejing Dou. “<a href="https://dl.acm.org/doi/pdf/10.1145/3473912" rel="noopener nofollow ugc">Knowledge Distillation with Attention for Deep Transfer Learning of Convolutional Networks.</a>” <em>ACM Transactions on Knowledge Discovery from Data (TKDD)</em> 16, no. 3 (2021): 1-20.<br>
is another example of what I am interested in for transfer learning using a pre-trained model.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/0/e0029dd6353111c7db87022b220a73c00e4bf476.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e0029dd6353111c7db87022b220a73c00e4bf476" title="Screenshot from 2022-03-29 18-19-45"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/0/e0029dd6353111c7db87022b220a73c00e4bf476_2_690x295.jpeg" alt="Screenshot from 2022-03-29 18-19-45" data-base62-sha1="vXGyFZS8aogmR6YiEl730qEr4mG" width="690" height="295" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/0/e0029dd6353111c7db87022b220a73c00e4bf476_2_690x295.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/0/e0029dd6353111c7db87022b220a73c00e4bf476_2_1035x442.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/0/e0029dd6353111c7db87022b220a73c00e4bf476_2_1380x590.jpeg 2x" data-dominant-color="8DA4AC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-03-29 18-19-45</span><span class="informations">1638×702 113 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I am open to any method such as GradCAM or RISE that could be applicable to visualizing the saliency/attention map in the transfer learning paradigm. If there are such existing notebooks for a starter you could point me to would be really great. (Fig 1.d)</p>

thecho7: <p>You are finding how to draw a heatmap from the resulting attention.<br>
I give you an useful notebook from GradCAM (<a href="https://github.com/jacobgil/pytorch-grad-cam/blob/master/tutorials/Class%20Activation%20Maps%20for%20Object%20Detection%20With%20Faster%20RCNN.ipynb" class="inline-onebox" rel="noopener nofollow ugc">pytorch-grad-cam/Class Activation Maps for Object Detection With Faster RCNN.ipynb at master · jacobgil/pytorch-grad-cam · GitHub</a>)</p>