KeyError with get_activation

bramsurewaard: <p>Hey,</p>
<p>Already got a lot of helpful things form this form so many thanks!</p>
<p>I found the following function from ptrblck to visualize a feature map:</p>
<pre><code class="lang-auto">activation = {}
def get_activation(name):
    def hook(model, input, output):
        activation[name] = output.detach()
    return hook
</code></pre>
<p>This worked perfectly for my ResNet50, and now i wanted to try this on the discriminator of a GAN.<br>
This is model is made up like this:</p>
<pre><code class="lang-auto">class Discriminator128(nn.Module):
    def __init__(self, ngpu):
        super(Discriminator128, self).__init__()
        self.ngpu = ngpu
        self.main = nn.Sequential(
            # input is (nc) x 128 x 128
            nn.Conv2d(nc, ndf, 4, 2, 1, bias=False),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*2) x 64 x 64
            nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 2),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*4) x 32 x 32
            nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 4),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*8) x 16 x 16
            nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 8),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*16) x 8 x 8
            nn.Conv2d(ndf * 8, ndf * 16, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 16),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*8) x 4 x 4
            nn.Conv2d(ndf * 16, 1, 4, 1, 0, bias=False),
            nn.Sigmoid()
        )

    def forward(self, input):
        return self.main(input)
</code></pre>
<p>But when I try to get the activation in the following way:</p>
<pre><code class="lang-auto">act = activation['main[0]'].squeeze()
</code></pre>
<p>I get:</p>
<pre><code class="lang-auto">KeyError: 'main[14]'
</code></pre>
<p>Anybody who can help me out?</p>
<p>Thanks!</p>

ptrblck: <p>Could you post the code showing, how you’ve registered the hooks?<br>
Also, it seems you are passing <code>'main[14]'</code> as the key.</p>

bramsurewaard: <p>Like this:</p>
<pre><code class="lang-auto">netD.main[14].register_forward_hook(get_activation('main[14]'))
output = model(img)
torch.Size([1, 3, 128, 128])
print("main[14]") 
act = activation['main[0]'].squeeze()
print(act.type)
num_plot = 4
fig, axarr = plt.subplots(min(act.size(0), num_plot))
for idx in range(min(act.size(0), num_plot)):
    print(idx)
    axarr[idx].imshow(act[idx],cmap="gray")
</code></pre>

ptrblck: <p>The code seems to work:</p>
<pre><code class="lang-python">nc = 3
ndf = 3
model = Discriminator128(0)
model.main[14].register_forward_hook(get_activation('main[14]'))
output = model(torch.randn(1, 3, 128, 128))
act = activation['main[14]']
</code></pre>
<p>Note that <code>act</code> will have the shape <code>[1, 1, 1, 1]</code>, so that <code>imshow</code> won’t make much sense (plotting a single pixel).</p>

bramsurewaard: <p>Thanks for the reply!</p>
<p>I no longer get the KeyError but I know was wondering if you could help me visualize the feature map. Like you said the imshow doesn’t work.</p>
<p>The code now looks similar to something you wrote on a different toppic: <a href="https://discuss.pytorch.org/t/visualize-feature-map/29597" class="inline-onebox">Visualize feature map</a></p>
<p>So what would you think would be the best way to visualize the feature map for a given image?</p>

ptrblck: <p>The linked code should work.<br>
However, the current layer (<code>main[14]</code>) only has one output pixel, so while the code will “visualize” this value, there might not be much information in this “image”.<br>
I would suggest to select other layers with activation maps which are larger in their spatial size, so that you can visualize them accordingly. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>