How to modify the first and last layer of pretrained network

1235: <p>Hello. I have VGG19 network and i want modify the the first and last layer.</p>
<p>I want the modified network have 6 channel input channel. and the weight for first three channel is that of VGG 19.</p>
<p>In addition, I want to modify the last layer of  VGG (FC layer) with output channel 1</p>
<p>How can i do this?</p>

krishnavishalv: <p>You can write your code for the modified VGG network and then manually load weights into each layer.</p>

muzi: <p>hello, did you solve that issu？i have the same experience as you.</p>

jmandivarapu1: <aside class="quote no-group" data-username="1235" data-post="1" data-topic="22597">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/1235/48/1360_2.png" class="avatar"> 1235:</div>
<blockquote>
<p>6 channel input channe</p>
</blockquote>
</aside>
<pre><code class="lang-auto">class VGG(nn.Module):

    def __init__(self, features, num_classes=1000, init_weights=True):
        super(VGG, self).__init__()
        self.features = features
        self.classifier = nn.Sequential(
            nn.Linear(512 * 7 * 7, 4096),
            nn.ReLU(True),
            nn.Dropout(),
            nn.Linear(4096, 4096),
            nn.ReLU(True),
            nn.Dropout(),
            nn.Linear(4096, num_classes),
        )
        if init_weights:
            self._initialize_weights()

    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        return x
    def _initialize_weights(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
                if m.bias is not None:
                    nn.init.constant_(m.bias, 0)
            elif isinstance(m, nn.BatchNorm2d):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)
            elif isinstance(m, nn.Linear):
                nn.init.normal_(m.weight, 0, 0.01)
                nn.init.constant_(m.bias, 0)

cfg = {
    'A': [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
    'B': [64, 64, 'M', 128, 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
    'D': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512, 'M', 512, 512, 512, 'M'],
    'E': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512, 'M'],
}
def vgg19(pretrained=False, **kwargs):
    """VGG 19-layer model (configuration "E")
    Args:
        pretrained (bool): If True, returns a model pre-trained on ImageNet
    """
    if pretrained:
        kwargs['init_weights'] = False
    model = VGG(make_layers(cfg['E']), **kwargs)
    if pretrained:
        model.load_state_dict(model_zoo.load_url(model_urls['vgg19']))
    return model
</code></pre>
<pre><code class="lang-auto">def make_layers(cfg, batch_norm=False):
    layers = []
    in_channels = 6 #Define the number of channels
    for v in cfg:
        if v == 'M':
            layers += [nn.MaxPool2d(kernel_size=2, stride=2)]
        else:
            conv2d = nn.Conv2d(in_channels, v, kernel_size=3, padding=1)
            if batch_norm:
                layers += [conv2d, nn.BatchNorm2d(v), nn.ReLU(inplace=True)]
            else:
                layers += [conv2d, nn.ReLU(inplace=True)]
            in_channels = v
    return nn.Sequential(*layers)
vgg19(pretrained=False)
</code></pre>
<p>The Above is the VGG19 with 6 input channels.<br>
For the weight you need to load it manually for every layer through iterating.</p>
<p>One solution would be us vgg19(pretrained=True) with input channels=3 then you will get net weights as ‘vgg19-dcbb9e9d.pth’ file then see my other reply</p><aside class="quote quote-modified" data-post="2" data-topic="22887">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jmandivarapu1/48/3544_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/convert-image-tensor-to-numpy-image-array/22887/2">Convert image tensor to numpy image array</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    I have below implementation which can flatten and unflatten both the net itself.Hopefully you can use some of that. 
import numpy as np

#############################################################################
# Flattening the NET
#############################################################################
def flattenNetwork(net):
    flatNet = []
    shapes = []
    for param in net.parameters():
        #if its WEIGHTS
        curr_shape = param.cpu().data.numpy().shape
        shapes.ap…
  </blockquote>
</aside>

<p>use flat_weights,shapes=flattenNetwork(vgg19_3channels)<br>
x=unFlattenNetwork(flat_weights,shapes) --this will give you the numpy array for each layer.</p>
<p>then you modify the first x[0] which is 3 channel in the above to the 6 channels just by adding your weights or dummy weights.<br>
then iterate through all layers and bias and modify the vgg19_6channel model with modified x you got above.</p>
<pre><code class="lang-auto"> vgg19_6channel.conv1.weight.data = torch.from_numpy(x[0])
 vgg19_6channel.conv1.bias.data =   torch.from_numpy(x[1]) ..etc
``</code></pre>

Divya_B: <p>This worked for me: I was trying to use greyscale image.</p>
<p>vgg16 = models.vgg16_bn(pretrained=True)<br>
for param in vgg16.parameters():<br>
param.requires_grad = False<br>
layers = list(vgg16.features.children())[:-1]<br>
layers[0] = nn.Conv2d(1, 64, kernel_size=3, stride=1, padding=1)<br>
features = nn.Sequential(*layers).cuda()</p>

I_am_you: <p>Hi <a class="mention" href="/u/divya_b">@Divya_B</a>  could explain how it is modified in the code snippet which you have written? I am trying to change the number of input channels to 4 in vgg19. If you could explain a little it would help me with my task.</p>