Use pretrained resnet18 for segmentation

username: Cverlpeng
cooked: <p>hi,everyone<br>
I rebuild resnet18 and use pretrained of pytorch for segmentation task,   I trained this model,but the network has not learned anything.Is this written correctly? Is there a more concise way of writing? I not use average pool and fc layers of resnet18 ,I want use pretrained weight of pytorch !</p>
<p>thanks !!!</p>
<pre><code class="lang-python">class ResNet18(nn.Module):
    def __init__(self, out_dim, *args, **kwargs):
        super(ResNet18, self).__init__(*args, **kwargs)
        resenet18 = torchvision.models.resnet18(pretrained=True)
        self.conv1 = resenet18.conv1 
        self.bn1 = resenet18.bn1
        self.relu = resenet18.relu
        self.maxpool = resenet18.maxpool
        self.c1 = resenet18.layer1
        self.c2 = resenet18.layer2
        self.c3 = resenet18.layer3
        self.c4 = resenet18.layer4
        self.deconv = nn.ConvTranspose2d(512, out_dim, kernel_size=32, stride=32, padding=0, output_padding=0)

    def forward(self, x):
        x =self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)
        x = self.c1(x)
        x = self.c2(x)
        x = self.c3(x)
        x = self.c4(x)        
        x = self.deconv(x)
        x = [x]
        return tuple(x)
</code></pre>

username: Cverlpeng
cooked: <p>who can help me ? thanks</p>

username: ptrblck
cooked: <p>The model looks generally alright. I’m wondering a bit why you are wrapping <code>x</code> in a list and return a <code>tuple</code>?<br>
Could you try to overfit the model on a very small sample of your dataset, e.g. just 10 examples?<br>
If that’s not working you might have some other bugs in your code.</p>

username: Cverlpeng
cooked: <p>I am imitating this file <a href="https://github.com/junfu1115/DANet/blob/master/encoding/models/fcn.py" rel="noopener nofollow ugc">DANet</a><br>
I return tuple due to</p>
<ul>
<li>Current PyTorch DataParallel Table is not supporting mutl-gpu loss calculation, which makes the gpu memory usage very in-balance. We address this issue here by doing DataParallel for Model &amp; Criterion. <a href="https://hangzhang.org/PyTorch-Encoding/parallel.html" rel="noopener nofollow ugc">Parallel</a></li>
</ul>
<p>I trained this model on lane segmentation dataset , it has train set 1319，I get this resulit :<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/2/239314f62bcac9fbc801e5ba80c13094566ecca7.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/239314f62bcac9fbc801e5ba80c13094566ecca7" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/2/239314f62bcac9fbc801e5ba80c13094566ecca7_2_690x421.jpeg" alt="image" data-base62-sha1="54HPDPDU6UqSYy2vFGDYX6vfTlZ" width="690" height="421" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/2/239314f62bcac9fbc801e5ba80c13094566ecca7_2_690x421.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/2/239314f62bcac9fbc801e5ba80c13094566ecca7_2_1035x631.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/2/239314f62bcac9fbc801e5ba80c13094566ecca7.jpeg 2x" data-dominant-color="0C0C0C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1275×779 160 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>this image have four images,The bottom right corner of the picture is network input lane label image ,other three images is softmax result of resnet18 model output 。( result_softmax * 255 )</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/5a6ae64891303f5d780c064abfc80fde81191f81.png" data-download-href="https://discuss.pytorch.org/uploads/default/5a6ae64891303f5d780c064abfc80fde81191f81" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/5a6ae64891303f5d780c064abfc80fde81191f81_2_690x257.png" alt="image" data-base62-sha1="cTS0dGaBR6UaABpNBNiNkCHlQVr" width="690" height="257" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/5a6ae64891303f5d780c064abfc80fde81191f81_2_690x257.png, https://discuss.pytorch.org/uploads/default/optimized/2X/5/5a6ae64891303f5d780c064abfc80fde81191f81_2_1035x385.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/5/5a6ae64891303f5d780c064abfc80fde81191f81_2_1380x514.png 2x" data-dominant-color="6C5164"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1840×686 102 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>very faster when i trained model. print training log some abnormalities, some times print two epoch log,some times print one epoch informention。</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/533ada31de74cb0452f0b79e95fd28af6da46be7.png" data-download-href="https://discuss.pytorch.org/uploads/default/533ada31de74cb0452f0b79e95fd28af6da46be7" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/533ada31de74cb0452f0b79e95fd28af6da46be7_2_690x291.png" alt="image" data-base62-sha1="bShIFRHuL5DqXcXnk4CdDN1yG4D" width="690" height="291" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/533ada31de74cb0452f0b79e95fd28af6da46be7_2_690x291.png, https://discuss.pytorch.org/uploads/default/optimized/2X/5/533ada31de74cb0452f0b79e95fd28af6da46be7_2_1035x436.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/5/533ada31de74cb0452f0b79e95fd28af6da46be7_2_1380x582.png 2x" data-dominant-color="FDFDFD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1546×654 35.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
this train model learning rate and loss</p>
<p>I only change network, used rebuild resnet18,but when I use deeplabv1 model,it can segmentaion lane image</p>
<p>This problem has been bothering me for a few days. I hope you can help me, thank you very much!</p>

username: ptrblck
cooked: <p>Did you try to overfit a small data sample?<br>
I’m not familiar with deeplabv1, but did you only change the model or something else?</p>

username: Cverlpeng
cooked: <p>I not try to overfit a small data sample, yes ,I only change network Network structure, The rest of the steps are not changed, when i use deeplabv1 ( Code reproduced by others ) to training, it can segmentation image. but I use resnet18 (Code reproduced by me) ,it not work, so I think reproduced code have error !</p>

username: Cverlpeng
cooked: <p>I am pytorch novice, I don’t quite understand why, many open split jobs will rewrite DataParallel,such as <a href="https://hangzhang.org/PyTorch-Encoding/parallel.html" rel="nofollow noopener">parallel</a>, but don’t use official this function.</p>

username: ptrblck
cooked: <p>Ok, but if your model succesfully overfits on a small sample, this would be at least some signal that your code is working, so I would suggest to try it first.</p>
<p>I’m not familiar with the package you’ve linked, but apparently it’s used to avoid an imbalances GPU memory usage in <code>nn.DataParallel</code>.</p>

username: Cverlpeng
cooked: <p>Thank you for your prompt reply<br>
I rebuild vgg16 ,it can work(pretrained = True or False) , I rebulit resnet18, it can’t work, but I don know why?</p>
<pre data-code-wrap="python"><code class="lang-python">class VGG16(nn.Module):
    def __init__(self, out_dim, *args, **kwargs):
        super(VGG16, self).__init__(*args, **kwargs)
        vgg16 = torchvision.models.vgg16(pretrained=True)
        self._up_kwargs = {'mode': 'bilinear', 'align_corners': True}
        self.features = vgg16.features
        self.conv1x1 = nn.Conv2d(512, out_dim, 1,1,0)
        self.deconv = nn.ConvTranspose2d(512, out_dim, kernel_size=32, stride=32, padding=0, output_padding=0)
    def forward(self, x):
        imsize = x.size()[2:]
        x =self.features(x)
        x = self.conv1x1(x)
        x = F.upsample(x, imsize, **self._up_kwargs)
        x = [x]
        return tuple(x)
</code></pre>
<pre data-code-wrap="python"><code class="lang-python">class ResNet18(nn.Module):
    def __init__(self, out_dim, *args, **kwargs):
        super(ResNet18, self).__init__(*args, **kwargs)
        resenet18 = torchvision.models.resnet18(pretrained=True)
        self.conv1 = resenet18.conv1 
        self.bn1 = resenet18.bn1
        self.relu = resenet18.relu
        self.maxpool = resenet18.maxpool
        self.c1 = resenet18.layer1
        self.c2 = resenet18.layer2
        self.c3 = resenet18.layer3
        self.c4 = resenet18.layer4
        self.deconv = nn.ConvTranspose2d(512, out_dim, kernel_size=32, stride=32, padding=0, output_padding=0)

    def forward(self, x):
        x =self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)
        x = self.c1(x)
        x = self.c2(x)
        x = self.c3(x)
        x = self.c4(x)        
        x = self.deconv(x)
        x = [x]
        return tuple(x)
</code></pre>
<p>I  find resulit of segmentation,The jagged phenomenon is obvious.  I try to use Multiple models , but get same result. A small square appears in the lane probability map.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/9/94411be67c28bb3a6b9536d1e010e0b6dee9cbeb.png" data-download-href="https://discuss.pytorch.org/uploads/default/94411be67c28bb3a6b9536d1e010e0b6dee9cbeb" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/9/94411be67c28bb3a6b9536d1e010e0b6dee9cbeb_2_690x396.png" alt="image" data-base62-sha1="l9w9YPddfPGzDYtDS7aqiMml8r9" width="690" height="396" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/9/94411be67c28bb3a6b9536d1e010e0b6dee9cbeb_2_690x396.png, https://discuss.pytorch.org/uploads/default/optimized/2X/9/94411be67c28bb3a6b9536d1e010e0b6dee9cbeb_2_1035x594.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/9/94411be67c28bb3a6b9536d1e010e0b6dee9cbeb.png 2x" data-dominant-color="060606"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1267×728 23.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
this image show result of Vgg16</p>

username: amk
cooked: <p>I encountered the same issue. Have you had any success meanwhile?</p>