Multiple class semantic segmentation issue

username: krioux5
cooked: <p>Hello,</p>
<p>I got a network trained to do a binary semantic segmentation but now I am trying to train it to segment 12 classes. The issue is that my code always interprets my target mask as all zeros. The target mask is read correctly and only gets messed up at this section of code:</p>
<pre><code class="lang-auto">def get_transform(phase,mean,std):
    list_trans=[]
    if phase=='train':
        list_trans.extend([HorizontalFlip(p=0.5)])
    list_trans.extend([Normalize(mean=mean,std=std,p=1),ToTensor()]) # normalizing the data &amp; then converting to tensors
    list_trans=Compose(list_trans)
    return list_trans
</code></pre>
<p>I realized that my binary segmentation masks had a pixel depth of 1 whereas my 12 class masks are pixel depth 8 which may be getting messed up due to the following:</p>
<p>The version of ToTensor i’m utilizing is from albumentations:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d877eaf1016169a9efd58fc028ce4c231d0e9a7.png" data-download-href="https://discuss.pytorch.org/uploads/default/1d877eaf1016169a9efd58fc028ce4c231d0e9a7" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1d877eaf1016169a9efd58fc028ce4c231d0e9a7_2_690x264.png" alt="image" data-base62-sha1="4de8RQd8GVVH8lBUtHYtMGlHkFN" width="690" height="264" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/d/1d877eaf1016169a9efd58fc028ce4c231d0e9a7_2_690x264.png, https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d877eaf1016169a9efd58fc028ce4c231d0e9a7.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d877eaf1016169a9efd58fc028ce4c231d0e9a7.png 2x" data-dominant-color="EEF1F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">912×350 35.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So is the issue that because my masks are now uint8 so they’re being divided by 255 as mentioned in the documentation? I am not totally sure what the issue is, or if it’s what I suspect, not sure how to fix. Lots of people do multi class semantic seg so there has to be an explanation!</p>
<p>Any help appreciated.</p>
<p>Kyle</p>

username: Kushaj
cooked: <p>Convert the mask to tensor yourself, using <code>torch.tensor(mask, dtype=torch.LongTensor)</code> and then you can pass this to albumentation,</p>

username: krioux5
cooked: <p>But the ToTensor() I have now in the code is applying it to everything (image, mask). If I make the mask a tensor beforehand, it will apply twice… right? Also what different behaviour would that solution provide?</p>

username: Kushaj
cooked: <aside class="quote no-group" data-username="krioux5" data-post="3" data-topic="91278">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/848f3c/48.png" class="avatar"> krioux5:</div>
<blockquote>
<p>If I make the mask a tensor beforehand, it will apply twice… right?</p>
</blockquote>
</aside>
<p>You remove that transform from albumentation.</p>