Resnet.features not work

yunjey: <p>Hi,</p>
<p>I want to extract features from the pretrained CNN.<br>
Until recently, the following code worked, but now I get an error.</p>
<pre><code>import torchvision.models as models

resnet = models.resnet152(pretrained=True)

resnet.features    # Error " 'ResNet' object has no attribute 'features' " occurred.
</code></pre>
<p>How can i fix this problem?</p>
<p>Thanks.</p>

SelvamArul: <p>Hi,</p>
<p>resnet has no module named features. (I guess you followed examples of using pretrained VGG network.)</p>
<p>Are you trying to use only a few layers from resnet? If you explain what you are trying to do, I will try yo help.</p>

yunjey: <p><a class="mention" href="/u/selvamarul">@SelvamArul</a></p>
<p>I want to extract feature vectors from the last hidden layer (before the softmax layer).</p>
<p>I don’t want replace the fc layer like <code>resnet.fc = nn.Linear(resnet.fc.in_features, embed_size)</code>. I don’t want train the resnet fc layer,  just want to extract the feature vectors from it.</p>
<p>Thanks</p>

yunjey: <p>Currently I am solving the problem like this.</p>
<pre><code>resnet = models.resnet152(pretrained=True)
modules = list(resnet.children())[:-1]      # delete the last fc layer.
resnet = nn.Sequential(*modules)</code></pre>

SelvamArul: <p>Just as a side note, if you just want to extract the features do not  forget to set <code>parameters.requires_grad = False</code></p>

Alpha: <p>Hi,<br>
Can you explained it more in detail, where should I   set <strong>parameters.requires_grad = False</strong>, i.e<br>
what is the full expression？<br>
thank you</p>

SelvamArul: <p>When you use pretrained models for finetuning, you don’t want to backpropagate though the pretrained model. i.e. You only update the weights of the new layers added on top the pretrained model. To a achieve this, you should iterate through the parameters of the pretrained model and set <code>requires_grad = False</code>. For example,</p>
<blockquote>
<p>resnet = models.resnet152(pretrained=True)<br>
modules = list(resnet.children())[:-1]      # delete the last fc layer.<br>
resnet = nn.Sequential(*modules)<br>
<code>### Now set requires_grad to false</code><br>
for param in resnet.parameters():<br>
param.requires_grad = False</p>
</blockquote>

syedumaramin: <p>Did you solve this problem, how did you extract features</p>

garyb: <p>I’m a newbie myself. I love PyTorch’s architecture, but I’m tempted to go back to TensorFlow because the documentation and examples are few and fragmentary. For more fragments below, first we create a mean and standard deviation of correct dimenstions</p>
<pre><code>resnet_mean = torch.from_numpy(
    np.array([0.485, 0.456, 0.406], dtype=np.float32))
resnet_std = torch.from_numpy(
    np.array([0.229, 0.224, 0.225], dtype=np.float32))
# Oh, for below, we need to match the dimensions. If you aren't unsqueezing, you're not pytorching
mean = resnet_mean.unsqueeze(0).unsqueeze(-1) #add a dimenstion for batch and (width*height)
std = resnet_std.unsqueeze(0).unsqueeze(-1) #add a dimenstion for batch and (width*height)
</code></pre>
<p>OK, now it gets slightly more complex. Images in pytorch tensors are <span class="hashtag">#channels</span> x height x width but mostly they come in collections or “batches” which means batch x channels x height x width. Let’s assume, via your wrestling with your datalaoder that you have the later. Then, lets reshap that tensor (OK, some programmer wants to call reshape that we all know from numpy as “view” oi vey): Assume we have a tensor of batch x chan x height x width and again, this would presumably be in some data loader normalization function<br>
… tensor enters normalization function …<br>
h, w = tensor.shape[2:]<br>
norm_tensor = tensor.view(tensor.shape[0], tensor.shape[1], -1) <span class="hashtag">#batch</span> x channel x (height*width)<br>
norm_tensor = norm_tensor - mean     # Make image mean zero<br>
norm_tensor = norm_tensor / std         # Make std = 1<br>
norm_tensor = norm_tensor.view(tensor.shape[0], tensor.shape[1], h, w) <span class="hashtag">#back</span> to batch x chan x w x h</p>
<p>All set, the tensors are now normalized</p>
<p>Now, joining after Arul’s code … presumably this would be the data loader loop that wraps the incoming tensors normalized as above:<br>
output = resnet(norm_tensor)</p>
<p>OK, you have output features from your headless resnet.  I think what you really wanted is not the features, but some other trainable head you put on top of the headless resnet … currently grinding through that. gist is to create a model, “foo” then.<br>
Right after Arul’s code, do</p>
<pre><code>foo_resnet = nn.Sequential(resnet(), foo())
</code></pre>
<p>And of course, my line above this then becomes:</p>
<p>output = foo_resnet(norm_tensor)  <span class="hashtag">#your</span> mileage may vary unchecked code</p>

Shivender_Khajuria: <p>What is alternative to renset50.features?</p>

logankilpatrick: <p>I ran into this issue while going through: <a href="https://towardsdatascience.com/visualizing-convolution-neural-networks-using-pytorch-3dfa8443e74e" rel="noopener nofollow ugc">https://towardsdatascience.com/visualizing-convolution-neural-networks-using-pytorch-3dfa8443e74e</a> since I was using ResNet50 instead of the model from the article.</p>
<p>The solution I came up with was:</p>
<pre><code class="lang-auto">features = []
# model._modules came from: https://github.com/utkuozbulak/pytorch-cnn-visualizations/issues/50#issuecomment-531757757
for key,value in model._modules.items():
    features.append(value)
</code></pre>
<p>where model is the model I created and then instead of doing <code>model.features</code> I just reference it as <code>features</code> since it’s a separate object.</p>