The model.parameter() feed into optimizer when backpropagation

lisyuan: <p>I want to frozen the weight of pretrained vgg19, and just update the last fc layer when backpropagation<br>
The model is shown in following block. I have already set  <code>with torch.no_grad():</code> when I input the images into vgg19.</p>
<pre><code class="lang-auto">class ImgEncoder(nn.Module):

    def __init__(self, embed_dim):

        super(ImgEncoder, self).__init__()
        self.model = models.vgg19(pretrained=True)
        in_features = self.model.classifier[-1].in_features
        self.model.classifier = nn.Sequential(*list(self.model.classifier.children())[:-1]) # remove vgg19 last layer
        self.fc = nn.Linear(in_features, embed_dim)

    def forward(self, image):

        with torch.no_grad():
            img_feature = self.model(image) # (batch, channel, height, width)
        img_feature = self.fc(img_feature)

        l2_norm = F.normalize(img_feature, p=2, dim=1).detach()
        return l2_norm

model = ImgEncoder(embed_dim=300)
</code></pre>
<p>the question is when I using the <code>optimizer</code>, how should I feed the parameter</p>
<pre><code class="lang-auto">optimizer = optim.Adam(model.parameters(), lr=0.001)
</code></pre>
<p>or</p>
<pre><code class="lang-auto">optimizer = optim.Adam(model.fc.parameters(), lr=0.001)
</code></pre>
<p>This question is similar to the <a href="https://pytorch.org/docs/master/notes/autograd.html#excluding-subgraphs" rel="noopener nofollow ugc">tutorial</a> in official website like following:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/3/f38fa50452c88675dd89de310ceb8a5e3fb6ac68.png" data-download-href="https://discuss.pytorch.org/uploads/default/f38fa50452c88675dd89de310ceb8a5e3fb6ac68" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/3/f38fa50452c88675dd89de310ceb8a5e3fb6ac68_2_690x221.png" alt="image" data-base62-sha1="yKDMvJzMLsFxBL0ayxZhBOexnIc" width="690" height="221" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/3/f38fa50452c88675dd89de310ceb8a5e3fb6ac68_2_690x221.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/3/f38fa50452c88675dd89de310ceb8a5e3fb6ac68_2_1035x331.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/3/f38fa50452c88675dd89de310ceb8a5e3fb6ac68.png 2x" data-dominant-color="ECEEF1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1064×342 28.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Why we can’t simply put the <code>model.parameters()</code> but <code>model.fc.parameters()</code> instead ?</p>

Nikronic: <p>Hi,</p>
<p>The answer is yes, you can and there won’t be any differences in result.<br>
Please see this thread for more details:</p><aside class="quote" data-post="1" data-topic="83358">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/valerio_biscione/48/22759_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/passing-to-the-optimizers-frozen-parameters/83358">Passing to the optimizers frozen parameters</a> 
  </div>
  <blockquote>
    When I want to do transfer learning, I set the require_grad = False and only pass the other parameters to the optimizer. But what happens if I only do one of these two steps? 

What happens if I set several modules’ reguire_grad = False but then I pass all net.parameters() to the optimize.
What happens if I keep the require_grad = True but only pass a subset of the parameters to the optimizer (the subset I want to fine tune, for example).
  </blockquote>
</aside>

<p>Bests</p>

lisyuan: <p>thanks for your sharing!</p>