How to extract features of an image from a trained model

big_tree: Hi all,

apaszke: <SCODE>torchvision.models.resnet18(pretrained=True)
<ECODE>

fmassa: You can either reconstruct the classifier once the model was instantiated, as in the following example: <SCODE>import torch
import torch.nn as nn
from torchvision import models

model = models.alexnet(pretrained=True)

# remove last fully-connected layer
new_classifier = nn.Sequential(*list(model.classifier.children())[:-1])
model.classifier = new_classifier
<ECODE> Or, if instead you want to extract other parts of the model, you might need to recreate the model structure, and reusing the parts of the pre-trained model in the new model. <SCODE>import torch
import torch.nn as nn
from torchvision import models

original_model = models.alexnet(pretrained=True)

class AlexNetConv4(nn.Module):
            def __init__(self):
                super(AlexNetConv4, self).__init__()
                self.features = nn.Sequential(
                    # stop at conv4
                    *list(original_model.features.children())[:-3]
                )
            def forward(self, x):
                x = self.features(x)
                return x

model = AlexNetConv4()
<ECODE>

big_tree: 

kpar: Is there a way to get values from multiple layers with one forward pass (for neural style transfer etc.)? In Tensorflow it would be along the lines of <SCODE>features = sess.run([net.conv1, net.conv2, net.conv3])
<ECODE>

fmassa: <SCODE>class Net(nn.Module):
    def __init__(self):
        self.conv1 = nn.Conv2d(1, 1, 3)
        self.conv2 = nn.Conv2d(1, 1, 3)
        self.conv3 = nn.Conv2d(1, 1, 3)

    def forward(self, x):
        out1 = F.relu(self.conv1(x))
        out2 = F.relu(self.conv2(out1))
        out3 = F.relu(self.conv3(out2))
        return out1, out2, out3
<ECODE>

kpar: That looks like it’ll do the trick. Is there a convenient way to fetch the intermediate values when the forward behavior is defined by nn.Sequential()? It seems like right now the only way to compose multiple responses would be to split off all the individual layers and forward the values manually in forward(). Essentially what I want to do is take an existing network (e.g. VGG) and just pick some responses of some layers (conv1_1, pool1, pool2, etc.) and concatenate them into a feature vector.

fmassa: You could write your own sequential version that keeps track of all intermediate results in a list. Something like <SCODE>class SelectiveSequential(nn.Module):
    def __init__(self, to_select, modules_dict):
        super(SelectiveSequential, self).__init__()
        for key, module in modules_dict.items():
            self.add_module(key, module)
        self._to_select = to_select
    
    def forward(x):
        list = []
        for name, module in self._modules.iteritems():
            x = module(x)
            if name in self._to_select:
                list.append(x)
        return list
<ECODE> And then you could use it like <SCODE>class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.features = SelectiveSequential(
            ['conv1', 'conv3'],
            {'conv1': nn.Conv2d(1, 1, 3),
             'conv2': nn.Conv2d(1, 1, 3),
             'conv3': nn.Conv2d(1, 1, 3)}
        )

    def forward(self, x):
        return self.features(x)
<ECODE>

lcelona: Thank you for your help

alexis-jacq: <SCODE>net = models.alexnet(pretrained=True).features`

class Feature_extractor(nn.module):
    def forward(self, input):
        self.feature = input.clone()
        return input

new_net = nn.Sequential().cuda() # the new network

target_layers = [conv_1, conv_2, conv_4] # layers you want to extract`

i = 1
for layer in list(cnn):
    if isinstance(layer,nn.Conv2d):
        name = "conv_"+str(i)
        art_net.add_module(name,layer)

        if name in target_layers:
            new_net.add_module("extractor_"+str(i),Feature_extractor())

        i+=1

    if isinstance(layer,nn.ReLU):
        name = "relu_"+str(i)
        new_net.add_module(name,layer)

    if isinstance(layer,nn.MaxPool2d):
        name = "pool_"+str(i)
        new_net.add_module(name,layer)

new_net.forward(your_image)
print new_net.extractor_3.feature<ECODE>

apaszke: If you really want to do something like that, I’d recommend this: <SCODE>class FeatureExtractor(nn.Module):
    def __init__(self, submodule, extracted_layers):
        self.submodule = submodule

    def forward(self, x):
        outputs = []
        for name, module in self.submodule._modules.items():
            x = module(x)
            if name in self.extracted_layers:
                outputs += [x]
        return outputs + [x]
<ECODE>

alexis-jacq: Wow! good to know that… Thanks! That would be really useful!

Atcold: <SCODE>x = Variable(torch.rand(1, 3, 224, 224))
h_x = resnet_18.forward(x)
last_view = h_x.creator.previous_functions[0][0]
last_pool = last_view.previous_functions[0][0]
embedding = last_pool.saved_tensors[0]
<ECODE>

apaszke: Why do you want to inspect the graph? A new instance is going to be created at every forward.

Atcold: <SCODE>for name, module in self.submodule._modules.items():
    x = module(x)
<ECODE>

apaszke: 

alexis-jacq: 

Atcold: 

edran: We managed to quickly write what we currently have because getting TensorBoard to take 2d values / histograms was relatively straightforward, however adding integration for more complex data types is not a one-day job. After the ICML deadline we’ll give it a look.