Transfer learning in semantic segmentation on Cityscapes dataset

Ves: <p>Hello there.<br>
I am totally new to Semantic Segmentation, and my current project is observing effects of domain shifting in Semantic Segmentation. I was thinking of using Cityscapes as my real data set and GTA alongside Synthia as my synthetic data. I have done some basic transfer learning with classification which was fairly straight-forward but I am struggling immensely with it in semantic segmentation. I have tried following several guides that I will link at the end of this post, they are all to some degree different approaches to the same thing. After none of them working for me I don’t know where to look for anymore.<br>
I have a basic understanding of how semantic segmentation works, I can also load my data with no problems. The only problem I have is how to adapt a pretrained semantic segmentation model such as DeepLabV3 to my dataset (in this case Cityscapes). The latest unsuccessful approach I tried was this:</p>
<p>def createDeepLabv3(outputchannels=1):<br>
“”“DeepLabv3 class with custom head<br>
Args:<br>
outputchannels (int, optional): The number of output channels<br>
in your dataset masks. Defaults to 1.<br>
Returns:<br>
model: Returns the DeepLabv3 model with the ResNet101 backbone.<br>
“””<br>
model = torchvision.models.segmentation.deeplabv3_resnet50(pretrained=True,<br>
progress=True)<br>
model.classifier = DeepLabHead(2048, outputchannels)<br>
# Set the model in training mode<br>
model.train()<br>
return model</p>
<p>But I am not sure what output channels represent. Are they the number of classes in my dataset? If so, Cityscapes has around 35 but only 20 or so are useful, so that also further confuses me as to how should I do this. I am aware that this stems from a lack of understanding of a DeepLabV3 architecture but I am kind of crunching so I am afraid trying to understand that wouldn’t be worth it at his point.<br>
Anyway, the closest I got was using this guide which implemented its own model using LightningModule library which I am not familiar with in the slightest:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/talhaanwarch/youtube-tutorials/blob/main/cityscape-tutorial.ipynb">
  <header class="source">

      <a href="https://github.com/talhaanwarch/youtube-tutorials/blob/main/cityscape-tutorial.ipynb" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/talhaanwarch/youtube-tutorials/blob/main/cityscape-tutorial.ipynb" target="_blank" rel="noopener nofollow ugc">talhaanwarch/youtube-tutorials/blob/main/cityscape-tutorial.ipynb</a></h4>


      <pre><code class="lang-ipynb">{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6c3a9222",
   "metadata": {},
   "source": [
    "# dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8f2560db",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
</code></pre>



  This file has been truncated. <a href="https://github.com/talhaanwarch/youtube-tutorials/blob/main/cityscape-tutorial.ipynb" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Another one that i tried following but was even less successful with was this one:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://towardsdatascience.com/transfer-learning-for-segmentation-using-deeplabv3-in-pytorch-f770863d6a42">
  <header class="source">
      <img src="https://miro.medium.com/v2/resize:fill:256:256/1*VzTUkfeGymHP4Bvav-T-lA.png" class="site-icon" width="207" height="206">

      <a href="https://towardsdatascience.com/transfer-learning-for-segmentation-using-deeplabv3-in-pytorch-f770863d6a42" target="_blank" rel="noopener nofollow ugc" title="05:02AM - 05 December 2020">Medium – 5 Dec 20</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/486;"><img src="https://miro.medium.com/v2/resize:fit:1021/0*P2ZsEEeUE4wwR9Db" class="thumbnail" width="690" height="486"></div>

<h3><a href="https://towardsdatascience.com/transfer-learning-for-segmentation-using-deeplabv3-in-pytorch-f770863d6a42" target="_blank" rel="noopener nofollow ugc">Transfer Learning for Segmentation Using DeepLabv3 in PyTorch</a></h3>

  <p>Learn how to perform road crack detection from just 118 images!</p>

  <p>
    <span class="label1">Reading time: 11 min read</span>
  </p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I’d appreciate a link to some guide that explains how to adjust a pretrained semantic segmentation model to a specific dataset, in this case Cityscapes.</p>