Controling the segmantation prediction better

username: blackbirdbarber
cooked: <p>I have the <code>segmentation.deeplabv3_resnet50</code> under my test. It’s working bad at the moment.</p>
<p>As you can see this is the original image normalized<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/7/b7197c9a59046fe8aa6114bdd75c90e993ff7e69.jpeg" alt="image" data-base62-sha1="q7LYTq9Q4e0Wx1BoTuBGTMBSPWN" width="320" height="194"><br>
And this is the ground truth with just 3 classes (background - red, vessel - black,  and filled - blue).</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53e20784f55251ae1d11df297146feed0b28b733.png" alt="image" data-base62-sha1="bY3TxeZTdFQdTOlRlwft5fPQ0Ej" width="253" height="188"></p>
<p>I just don’t have the experience training this model. Exactly:</p>
<ol>
<li>What should be the final prediction? Single channel image or 3 channel image? My ground truth is just a single channel image 1x800x800 and it has 3 values inside: background 0, vessel 1 and filled 2. So should I create 3 channel image from 1x800x800?</li>
</ol>
<p><em>Note:</em> I used <code>cmap="flag"</code> in matplotlib to show the ground truth.</p>
<ol start="2">
<li>Is it possible to prohibit the prediction images to have other values than 0, 1, and 2?</li>
</ol>
<p>Currently my prediction is 3 channels:</p>
<pre><code class="lang-auto">net.eval()
pred=net(imtest)['out']
pred.squeeze(0).shape
</code></pre>
<p><strong>torch.Size([3, 800, 800])</strong></p>
<p>When I created the model I had 2 options.</p>
<pre><code class="lang-auto">net = torchvision.models.segmentation.deeplabv3_resnet50(num_classes=3)
</code></pre>
<p>and 2nd:</p>
<pre><code class="lang-auto">net = torchvision.models.segmentation.deeplabv3_resnet50(pretrained=True) # Load net
net.classifier[4] = torch.nn.Conv2d(256, 3, kernel_size=(1, 1), stride=(1, 1)) # Change final layer to 3 classes
</code></pre>
<p>but I selected the 2nd option.</p>
<p>Also you may saw values other than 0, 1, 2 on the edges.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53e20784f55251ae1d11df297146feed0b28b733.png" alt="image" data-base62-sha1="bY3TxeZTdFQdTOlRlwft5fPQ0Ej" width="506" height="376"></p>
<p>The original target PNG images have just 0,1,2 values inside, after transforms resize I think the default resizer produces that edge values don’t have the 0,1 and 2 any more instead they are somewhere in between because of interpolation.</p>
<ol start="3">
<li>Any ideas I suppress that.</li>
</ol>

username: ptrblck
cooked: <ol>
<li>
<p>The output for a multi-class segmentation use case would usually have the shape <code>[batch_size, nb_classes, height, width]</code> where <code>num_classes=3</code> in your use case.</p>
</li>
<li>
<p>Each channel of the prediction contains the logit values for the corresponding class. Using <code>torch.argmax(output, dim=1)</code> will return the predicted class indices in the range <code>[0, 2]</code>.</p>
</li>
<li>
<p>Yes, I think you are correct and these values are created from e.g. a linear interpolation during resizing. Use the nearest neighbor interpolation technique for any <code>Resize</code> transformation and these values should disappear.</p>
</li>
</ol>