Weird results with IntermediateLayerGetter to get intermediate features of Alexnet

username: nikiguo93
cooked: <p>I use IntermediateLayerGetter from torch_intermediate_layer_getter to get intermediate features of alexnet. But something weird happened.<br>
When I remove the layer after fc6 layer and let it output the fc6 features directly, the results I obtained is quite different from the results using IntermediateLayerGetter to get fc6 features from pretrained alexnet.<br>
the code looks like the following:</p>
<h1>
<a name="using-intermediatelayergetter-1" class="anchor" href="#using-intermediatelayergetter-1"></a>using IntermediateLayerGetter</h1>
<p>alexnet = models.alexnet(pretrained=True)<br>
alexnet.cuda()</p>
<p>return_layers = {  ‘classifier.1’: ‘classifier.1’}<br>
mid_getter = MidGetter(alexnet, return_layers=return_layers, keep_output=True)</p>
<p>Testlr_list =[]<br>
Test_batchsize = 1<br>
alexnet.eval()<br>
with torch.no_grad():<br>
for i in range(TestImageData.size()[0]//Test_batchsize):<br>
local_X = TestImageData[i*Test_batchsize:(i+1)*Test_batchsize,:,:,:].float().cuda()<br>
mid_outputs, model_output = mid_getter(local_X)<br>
print(mid_outputs[‘classifier.1’].size())<br>
Testlr_list.append(mid_outputs[‘classifier.1’].detach().cpu())<br>
Testlr_data = torch.cat(Testlr_list)<br>
print(Testlr_data.shape)</p>
<p><span class="hashtag">#remove</span> layers after fc6 and output fc6 features directly<br>
m =  models.alexnet(pretrained=True)<br>
m.classifier = nn.Sequential(*list(m.classifier.children())[:-5])<br>
m.cuda()<br>
Testlr_list =[]<br>
Test_batchsize = 1<br>
m.eval()<br>
with torch.no_grad():<br>
for i in range(TestImageData.size()[0]//Test_batchsize):<br>
local_X = TestImageData[i*Test_batchsize:(i+1)*Test_batchsize,:,:,:].float().cuda()<br>
out = m(local_X)<br>
Testlr_list.append(out.detach().cpu())<br>
Testlr_data = torch.cat(Testlr_list)<br>
print(Testlr_data)</p>
<p>the results are quite different. Is there anything wrong with my code?</p>

username: ptrblck
cooked: <p>I don’t know how <code>MidGetter</code> is defined, but <code>IntermediateLayerGetter</code> doesn’t accept the <code>keep_output</code> argument. In any case, I think you are seeing a difference, since <code>AlexNet</code> is using inplace <code>nn.ReLU</code> modules, which would manipulate the output of the previous layer inplace.</p>

username: nikiguo93
cooked: <p>Hi, thanks for your information.<br>
MidGetter is defined as follows:<br>
from torch_intermediate_layer_getter import IntermediateLayerGetter as MidGetter</p>
<p>So IntermediateLayerGetter is not good to extract intermediate features of alexnet? Or how to fix the problem? Do you have any other suggestions?</p>
<p>looking forward to hearing from you!</p>

username: nikiguo93
cooked: <p>hi, ptrblck. Can I use forward hook function to extract the intermediate features of alexnet? But something weird happened again. This result is quite different from the other two. My code looks like this:</p>
<p>model = torchvision.models.alexnet(pretrained=True)<br>
model.type(dtype)</p>
<p>activation = {}<br>
def get_activation(name):<br>
def hook(model, input, output):<br>
activation[name] = output.detach()<br>
return hook</p>
<p>model.classifier[1].register_forward_hook(get_activation(‘fc6’))</p>
<p>Testcomp_list =[]<br>
Test_batchsize = 1<br>
model.eval()<br>
with torch.no_grad():<br>
for i in range(TestImageData.size()[0]//Test_batchsize):<br>
local_X = TestImageData[i*Test_batchsize:(i+1)*Test_batchsize,:,:,:].type(dtype)<br>
out = model(local_X)<br>
Testcomp_list.append(activation[‘fc6’].detach().cpu())<br>
<span class="hashtag">#h</span>.remove()<br>
print(len(Testcomp_list))<br>
print(Testcomp_list[1].size())<br>
Testcomp_data = torch.cat(Testcomp_list)<br>
print(Testcomp_data.size())</p>
<p>Is there anything wrong again?</p>