Transfer weights of pretrained model to a different one

batoul_zr: <p>I have a model that is a concatination of pretraind vgg16 and a similar one with different kernel size. Is there a way to transfer the weights of the pretrained network to the other one so that network 2 don’t start training from 0?  I want to use this network as feature extraction in transfer learning</p>
<p>This is the structure of the whole network:</p>
<h1>
<a name="model-structure-1" class="anchor" href="#model-structure-1"></a>Model Structure:</h1>
<pre><code>class Detector(nn.Module):
    def __init__(self):
        super(Detector, self).__init__()
</code></pre>
<h1>
<a name="pretrained-vgg16-kernel-size-33-2" class="anchor" href="#pretrained-vgg16-kernel-size-33-2"></a>Pretrained VGG16: Kernel Size = 3*3</h1>
<pre><code>    self.base = torchvision.models.vgg16(pretrained=True).features
</code></pre>
<h1>
<a name="new-network-kernel-size-55-3" class="anchor" href="#new-network-kernel-size-55-3"></a>New Network: Kernel Size = 5*5</h1>
<pre><code>    self.conv_1a = nn.Conv2d(3, 64, kernel_size=5, stride=1, padding=1)
    self.conv_1b = nn.Conv2d(64, 64, kernel_size=5, stride=1, padding=1)

    self.conv_2a = nn.Conv2d(64, 128, kernel_size=5, stride=1, padding=1)
    self.conv_2b = nn.Conv2d(128, 128, kernel_size=5, stride=1, padding=1)
    self.conv_2c = nn.Conv2d(128, 128, kernel_size=5, stride=1, padding=1)

    self.conv_3a = nn.Conv2d(128, 256, kernel_size=5, stride=1, padding=1)
    self.conv_3b = nn.Conv2d(256, 256, kernel_size=5, stride=1, padding=1)
    self.conv_3c = nn.Conv2d(256, 256, kernel_size=5, stride=1, padding=1)
    self.conv_3d = nn.Conv2d(256, 256, kernel_size=5, stride=1, padding=1)

    self.conv_4a = nn.Conv2d(256, 512, kernel_size=5, stride=1, padding=1)
    self.conv_4b = nn.Conv2d(512, 512, kernel_size=5, stride=1, padding=1)
    self.conv_4c = nn.Conv2d(512, 512, kernel_size=5, stride=1, padding=1)
    self.conv_4d = nn.Conv2d(512, 512, kernel_size=5, stride=1, padding=1)
</code></pre>
<h1>
<a name="merging-feature-maps-4" class="anchor" href="#merging-feature-maps-4"></a>Merging Feature maps</h1>
<pre><code>    self.featuremap = nn.Conv2d(1024, 512, kernel_size=1, stride=1, padding=0)

def forward(self, image_tensor: Tensor) -&gt; Tensor:
    
    k3 = image_tensor
    k5 = image_tensor
</code></pre>
<h1>
<a name="kernel-33-5" class="anchor" href="#kernel-33-5"></a>kernel 3*3:</h1>
<pre><code>    for layer in self.base:
        k3 = layer(k3)
</code></pre>
<h1>
<a name="kernel-55-6" class="anchor" href="#kernel-55-6"></a>kernel 5*5:</h1>
<pre><code>    k5 = F.relu(self.conv_1a(k5))
    k5 = F.relu(self.conv_1b(k5))
    k5 = F.max_pool2d(k5, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)

    k5 = F.relu(self.conv_2a(k5))
    k5 = F.relu(self.conv_2b(k5))
    k5 = F.relu(self.conv_2c(k5))
    k5 = F.max_pool2d(k5, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)

    k5 = F.relu(self.conv_3a(k5))
    k5 = F.relu(self.conv_3b(k5))
    k5 = F.relu(self.conv_3c(k5))
    k5 = F.relu(self.conv_3d(k5))
    k5 = F.max_pool2d(k5, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)

    k5 = F.relu(self.conv_4a(k5))
    k5 = F.relu(self.conv_4b(k5))
    k5 = F.relu(self.conv_4c(k5))
    k5 = F.relu(self.conv_4d(k5))
    k5 = F.max_pool2d(k5, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
</code></pre>
<h1>
<a name="concat-the-2-networks-7" class="anchor" href="#concat-the-2-networks-7"></a>Concat the 2 Networks:</h1>
<pre><code>    conc_featuremap = torch.cat((k3, k5),dim=1)
</code></pre>
<h1>
<a name="merge-concatinated-feature-8" class="anchor" href="#merge-concatinated-feature-8"></a>Merge Concatinated Feature</h1>
<pre><code>    featuremap = self.featuremap(conc_featuremap)
    
    return featuremap
</code></pre>