Adding Layers to Transfer Learnig

username: ganariya
cooked: <p>Hello!<br>
I’m currently wondering if I can add a layer before and after the transfer learning model.</p>
<h2><a name="in-many-cases-1" class="anchor" href="#in-many-cases-1"></a>In many cases</h2>
<p>In many cases, for example,<br>
<a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" rel="noopener nofollow ugc">TRANSFER LEARNING FOR COMPUTER VISION TUTORIAL</a>,</p>
<p>only one layer is replaced with a new layer.</p>
<p><a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html</a></p>
<pre data-code-wrap="python"><code class="lang-python">model_ft = models.resnet18(pretrained=True)
num_ftrs = model_ft.fc.in_features
# Here the size of each output sample is set to 2.
# Alternatively, it can be generalized to nn.Linear(num_ftrs, len(class_names)).
model_ft.fc = nn.Linear(num_ftrs, 2)
</code></pre>
<h2><a name="what-want-to-do-2" class="anchor" href="#what-want-to-do-2"></a>What want to do</h2>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6cb6b13b3d25857047d17ee84581d6c63244e189.png" data-download-href="https://discuss.pytorch.org/uploads/default/6cb6b13b3d25857047d17ee84581d6c63244e189" title="無題のプレゼンテーション"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6cb6b13b3d25857047d17ee84581d6c63244e189.png" alt="無題のプレゼンテーション" data-base62-sha1="fvIYXIDA3vAFw89QDGuZ6ohYi8x" width="690" height="388" data-dominant-color="E2F3F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">無題のプレゼンテーション</span><span class="informations">960×540 7.89 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I would like to add layers to the transferred model, for example, VGG 16.</p>
<p>I can add layers to the transferred model. (I’m not sure this is working😢.)</p>
<pre data-code-wrap="python"><code class="lang-python">net2 = models.vgg16(pretrained=use_pretrained)

net2.classifier[6] = nn.Linear(in_features=4096, out_features=2)

# add layers after
net2.classifier.add_module('addition', nn.Linear(2, 2))
</code></pre>
<p>However, I cannot know how to insert layers before the transferred model.<br>
And, I am not sure <code>method of adding layers to the transferred model after</code>, the above code.</p>
<p>How do we insert layers to transferred model?</p>
<p>Thank you very much!</p>

username: Caruso
cooked: <p>Hi,<br>
you could create a new class for your model, which contains your layers before the model + the transfer model with a changed last layer (how you did int line 2 of your example) + all your additional layers at the end. In your forward method you now call your ‘before layers’, then your transferred model and last your ‘after layers’.</p>
<pre><code class="lang-auto">class MyModel(nn.Model):
  def __init__(self, out_features, use_pretrained):
    self.pre_layers = ...
    self.backbone = models.vgg16(pretrained=use_pretrained)
    self.backbone[6] = nn.Linear(in_features=4096, out_features=out_features)
    self.after_layers = ...

  def forward(self, x):
    x = self.pre_layers(x)
    x = self.backbone(x)
    x = self.after_layers(x)
    return x
</code></pre>

username: ganariya
cooked: <p>Thank you! <a class="mention" href="/u/caruso">@Caruso</a></p>
<pre><code class="lang-python">class MyModel(nn.Module):
    
    def __init__(self):
        super(MyModel, self).__init__()
        self.backbone = models.vgg16(pretrained=use_pretrained)
        self.backbone.classifier[6] = nn.Linear(in_features=4096, out_features=2)
        self.fv1 = nn.Linear(in_features=2, out_features=2)
        self.fv2 = nn.Linear(in_features=2, out_features=2)
    
    def forward(self, x):
        x = self.backbone(x)
        x = self.fv1(x)
        x = self.fv2(x)
        return x
</code></pre>
<p>This code goes well! Thank you very much <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>