Need help for first steps "Semantic Segmentation"

DL_Fan: <p>Hi,</p>
<p>I want to build my first neural network for semantic segmentation and I am looking for<br>
a good base to start, but all repos that I found on Github up to now were poorly explained<br>
and / or had a lot of issues. By the way, I would like to use a pretrained model.</p>
<p>Do you have any recommendations for me regarding a good repo or an advice for the<br>
first steps I should take? Thank you very much in advance!</p>
<p>BR</p>

onomatopoeia: <p>This has been discussed previously <a href="https://discuss.pytorch.org/t/finetune-semantic-segmentation-model-on-our-dataset/48133/3">here</a>; that may be of some help to you.</p>

DL_Fan: <p>Hi,</p>
<p>Thanks for your quick reply! Yes, this was helpful, but unfortunately it didn’t help me to solve my problem. The pytorch tutorial for finetuning deals mainly with classification and I have some problems transfering this to my segmentation challenge.</p>
<p>It would be great, could you help me again.</p>
<p>I  want to use a pretrained DeepLabV3</p>
<p><em>dlv3 = models.segmentation.deeplabv3_resnet101(pretrained=True)</em></p>
<p>which is trained on 21 categories. 2 would be enough for me. In the first phase my NN shall only be able to segment streets on an image and the rest can be background.</p>
<p>I think I need to change the last layers of the model (the classifier part), but I am not sure how and in particular which:</p>
<hr>
<p>(classifier): DeepLabHead(<br>
(0): ASPP(<br>
(convs): ModuleList(<br>
(0): Sequential(<br>
(0): Conv2d(2048, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): ReLU()<br>
)<br>
(1): ASPPConv(<br>
(0): Conv2d(2048, 256, kernel_size=(3, 3), stride=(1, 1), padding=(12, 12), dilation=(12, 12), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): ReLU()<br>
)<br>
(2): ASPPConv(<br>
(0): Conv2d(2048, 256, kernel_size=(3, 3), stride=(1, 1), padding=(24, 24), dilation=(24, 24), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): ReLU()<br>
)<br>
(3): ASPPConv(<br>
(0): Conv2d(2048, 256, kernel_size=(3, 3), stride=(1, 1), padding=(36, 36), dilation=(36, 36), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): ReLU()<br>
)<br>
(4): ASPPPooling(<br>
(0): AdaptiveAvgPool2d(output_size=1)<br>
(1): Conv2d(2048, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(3): ReLU()<br>
)<br>
)<br>
(project): Sequential(<br>
(0): Conv2d(1280, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): ReLU()<br>
(3): Dropout(p=0.5, inplace=False)<br>
)<br>
)<br>
(1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(3): ReLU()<br>
(4): Conv2d(256, 21, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
(aux_classifier): FCNHead(<br>
(0): Conv2d(1024, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): ReLU()<br>
(3): Dropout(p=0.1, inplace=False)<br>
(4): Conv2d(256, 21, kernel_size=(1, 1), stride=(1, 1))<br>
)</p>
<hr>
<p>For each training image I have got a dense semantic label id map (s. below).</p>
<p>And then - is it enough to set the parameters to requires_grad = false (except the new classifier layers) and train the complete model with a 3 channel RGB and the dense label id map picture as a target? What else do I have to think of?</p>
<p>Unfortunately I am a little bit lost right now. All beginnings are difficult I have to realize… :-/</p>
<p>Thank you very much in advance!</p>
<p>BR</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/c/bc59a292bbaacd664bec0a3632a6772dff8ee6ac.png" data-download-href="https://discuss.pytorch.org/uploads/default/bc59a292bbaacd664bec0a3632a6772dff8ee6ac" title="rs00000"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc59a292bbaacd664bec0a3632a6772dff8ee6ac_2_690x388.png" alt="rs00000" data-base62-sha1="qSdO0MPyxuagggYMdfkOv12KoB6" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc59a292bbaacd664bec0a3632a6772dff8ee6ac_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc59a292bbaacd664bec0a3632a6772dff8ee6ac_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc59a292bbaacd664bec0a3632a6772dff8ee6ac_2_1380x776.png 2x" data-dominant-color="0B0B0B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">rs00000</span><span class="informations">1920×1080 32.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

onomatopoeia: <p>In this <a href="https://expoundai.wordpress.com/2019/08/30/transfer-learning-for-segmentation-using-deeplabv3-in-pytorch/" rel="nofollow noopener">tutorial</a> that was on the previously linked page, under the header " DeepLabv3 Model" they give a custom function on how to change the ‘head’ of the network to predict however many classes you want. Their example of having only 2 classes, background, and something else, is what you’re doing. So I would stick close to their tutorial.</p>
<p>Hope that helps!</p>

DL_Fan: <p>Hi</p>
<p>and again thanks for your quick response. I really appreciate it. Yes,you are right, this is indeed helpful, but unfortunately this is not really a kind of tutorial I am looking for. You can copy it, change a few parameters and let it run. But it doesn’t really help you to understand the whole thing better. Of course only my opinion.</p>
<p>Thanks again and I hope my next questions will be more concrete. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>
<p>Best regards</p>

Pratibha: <p>Hello,<br>
Are you able to change the number of output channels in the last layer? I am also trying to train deeplabv3 model with my dataset. I am following this link:<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://expoundai.files.wordpress.com/2019/03/cropped-expoundai.png?w=32" class="site-icon" width="16" height="16">
      <a href="https://expoundai.wordpress.com/2019/08/30/transfer-learning-for-segmentation-using-deeplabv3-in-pytorch/" target="_blank" rel="noopener nofollow ugc" title="10:07PM - 30 August 2019">My Journey with Deep Learning and Computer Vision – 30 Aug 19</a>
  </header>
  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:116/28;"><img src="https://expoundai.files.wordpress.com/2019/08/image.png?fit=440%2C330" class="thumbnail" width="116" height="28"></div>

<h3><a href="https://expoundai.wordpress.com/2019/08/30/transfer-learning-for-segmentation-using-deeplabv3-in-pytorch/" target="_blank" rel="noopener nofollow ugc">Transfer Learning for Segmentation Using DeepLabv3 in PyTorch</a></h3>

<p>In this post, I’ll be covering how to use a pre-trained semantic segmentation DeepLabv3 model for the task of road crack detection in PyTorch by using transfer learning. The same procedure ca…</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
But the result still shows 21 output channels. Can anyone please help me out with this? I have 3 class labels for my dataset.<br>
If possible suggest some other github repo or a link to perform semantic segmentation.