Loss function for segmentation

amin: <p>hello everyone. I am trying to create a segmentation network (Unet) but I get confused a little.<br>
my network output dimension is y_pread &gt;&gt;(3,960,960) and also I have labels for my outputs with the same dimension y_real&gt;&gt;(3,960,960). I know that I can get one channel instead of three-channel and change the RGB image into a grayscale it means  y_pread &gt;&gt;(1,960,960)  and  y_real &gt;&gt;(1,960,960)<br>
but the problem is a crossentropy loss or the other loss such as BCEloss or the other could not support image or 2 dimensions in a loss.<br>
what should I do for my Unet loss function???</p>

JuliousHurtado: <p>I am not an expert, but just today I was reading a blog post that mentions something about that.<br>
They use MSELoss.</p>
<p>Here is the blog post:<br>
<a href="https://towardsdatascience.com/colorizing-black-white-images-with-u-net-and-conditional-gan-a-tutorial-81b2df111cd8" rel="noopener nofollow ugc">https://towardsdatascience.com/colorizing-black-white-images-with-u-net-and-conditional-gan-a-tutorial-81b2df111cd8</a></p>
<p>I hope it helps.</p>

KFrank: <p>Hi Amin!</p>
<p>Most typically for semantic segmentation you will classify each pixel in<br>
your image  into one of <code>nClass</code> classes (including what’s usually called<br>
a “background” class).</p>
<p>The input to your model will have shape <code>[nBatch, nChannel, H, W]</code><br>
(where the batch size, <code>nBatch</code>, is not built into the model, and the<br>
model is (usually) agnostic to it).  For an RGB image, <code>nChannel</code> would<br>
be 3.</p>
<p>The final layer of your U-Net model would typically be a <code>Conv2d</code> with<br>
<code>out_channels = nClass</code> and the output of the model would have<br>
shape <code>[nBatch, nClass, H, W]</code>.</p>
<p>In this situation you would typically use (or at least start with)<br>
<code>CrossEntropyLoss</code> as the loss function, and your ground-truth<br>
labels, the <code>target</code> passed to <code>CrossEntropyLoss</code>, will have shape<br>
<code>[nBatch, H, W]</code> (no <code>nClass</code> dimension), and consist of integer<br>
class labels that run from <code>0</code> to <code>nClass - 1</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

amin: <p>thanks a million  Mr.Hurtado</p>

amin: <p>Hi KFrank<br>
thanks for your quick response. if I understand correctly, I create y-label with 3 classes and also the output of my network.<br>
the batch size here is 1 and H = 4 and W = 3 in one channel, like the code below:</p>
<pre><code class="lang-auto">y_label = [[[1, 0, 2],
            [1, 1, 0],
            [2, 0, 0],
            [1, 0, 0]]]
y_label = torch.FloatTensor(y_label)
# print(y_label.shape) &gt;&gt; torch.Size([1, 4, 3])

y_target = [[[0.950, 0.210, 0.890],
             [0.875, 0.254, 0.248],
             [0.875, 0.012, 0.148],
             [0.542, 0.143, 0.002]]]
y_target = torch.FloatTensor(y_target)
# print(y_target.shape) &gt;&gt; torch.Size([1, 4, 3])

loss_func = nn.CrossEntropyLoss()
loss = loss_func(y_label, y_target)
</code></pre>
<p>but I get an error<br>
ValueError: Expected target size (1, 3), got torch.Size([1, 4, 3])</p>

KFrank: <p>Hi Amin!</p>
<aside class="quote no-group" data-username="amin" data-post="5" data-topic="129703" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/8e7dd6/48.png" class="avatar"> amin:</div>
<blockquote>
<p>I create y-label with 3 classes and also the output of my network.</p>
</blockquote>
</aside>
<p>No, this is not quite right.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">y_label = [[[1, 0, 2],
            [1, 1, 0],
            [2, 0, 0],
            [1, 0, 0]]]
y_label = torch.FloatTensor(y_label)
</code></pre>
</blockquote>
</aside>
<p>Yes, this is almost correct (assuming that the <code>y_label</code> are your<br>
ground-truth labels, provided externally, rather than predicted<br>
by your network).  The shape and values are fine, but this needs<br>
to be a <code>LongTensor</code> (rather than a <code>FloatTensor</code>).  Thus:</p>
<pre data-code-wrap="python"><code class="lang-python">y_label_list = [[[1, 0, 2],
            [1, 1, 0],
            [2, 0, 0],
            [1, 0, 0]]]
y_label = torch.LongTensor (y_label_list)
# or
y_label = torch.tensor (y_label_list, dtype = torch.int64)
# or just
y_label = torch.tensor (y_label_list)
</code></pre>
<p>(This will be the <em>second</em> argument passed to a <code>CrossEntropyLoss</code><br>
object and is referred to in the <code>CrossEntropyLoss</code> documentation<br>
as <code>target</code>.)</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">y_target = [[[0.950, 0.210, 0.890],
             [0.875, 0.254, 0.248],
             [0.875, 0.012, 0.148],
             [0.542, 0.143, 0.002]]]
y_target = torch.FloatTensor(y_target)
</code></pre>
</blockquote>
</aside>
<p>If this is meant to be the prediction made by your network (your<br>
network’s output), then it isn’t right.</p>
<p>(As a matter of nomenclature, I would <em>not</em> call the output of your<br>
network “target” as “target” is normally used to denote the<br>
ground-truth labels.  I like to call it “prediction.”  This becomes the<br>
first argument to your <code>CrossEntropyLoss</code> object and is referred<br>
to in the documentation as <code>input</code>.  Thus the <code>output</code> of your network<br>
is the “prediction” and is the <code>input</code> to <code>CrossEntropyLoss</code>.)</p>
<p>Let me use <code>y_pred</code> as the name for what you call <code>y_target</code>.</p>
<p>Your <code>y_pred</code> is missing its <code>nClass</code> dimension.  That is, it should<br>
have shape <code>[nBatch, nClass, H, W]</code>, so, using the values in your<br>
example, <code>[1, 3, 4, 3]</code>.</p>
<p>For each “pixel” in your predicted “image” you have <em>three</em> (<code>nClass</code>)<br>
<em>logits</em> that are the raw scores for that pixel being in each of your<br>
three classes.  Thus <code>y_pred[0, :, 2, 1]</code> is the 1d vector of three<br>
logits that predicts to which of your three classes your “2-1” pixel<br>
belongs.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">loss_func = nn.CrossEntropyLoss()
loss = loss_func(y_label, y_target)
</code></pre>
</blockquote>
</aside>
<p>Based on what I think your variables are supposed to be, you have<br>
your two arguments to <code>loss_func</code> passed in the wrong order.</p>
<p>Using my preferred <code>y_pred</code>) (instead of your <code>y_target</code>), this should<br>
be:</p>
<pre data-code-wrap="python"><code class="lang-python">loss_func = nn.CrossEntropyLoss()
loss = loss_func (y_pred, y_label)
</code></pre>
<p>Thus:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.9.0'
&gt;&gt;&gt; _ = torch.manual_seed (2021)
&gt;&gt;&gt; y_label = [[[1, 0, 2],
...             [1, 1, 0],
...             [2, 0, 0],
...             [1, 0, 0]]]
&gt;&gt;&gt; y_label = torch.tensor (y_label)
&gt;&gt;&gt; y_pred = torch.randn (1, 3, 4, 3)
&gt;&gt;&gt; loss_func = torch.nn.CrossEntropyLoss()
&gt;&gt;&gt; loss = loss_func (y_pred, y_label)
&gt;&gt;&gt; loss
tensor(1.5173)
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

amin: <p>thank a million KFrank,<br>
it works for me,<br>
send a million kisses  <img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/kissing_heart.png?v=10" title=":kissing_heart:" class="emoji" alt=":kissing_heart:"><img src="https://discuss.pytorch.org/images/emoji/apple/heart_eyes.png?v=10" title=":heart_eyes:" class="emoji" alt=":heart_eyes:"><img src="https://discuss.pytorch.org/images/emoji/apple/heart_eyes.png?v=10" title=":heart_eyes:" class="emoji" alt=":heart_eyes:"><img src="https://discuss.pytorch.org/images/emoji/apple/heart_eyes.png?v=10" title=":heart_eyes:" class="emoji" alt=":heart_eyes:"><img src="https://discuss.pytorch.org/images/emoji/apple/heart_eyes.png?v=10" title=":heart_eyes:" class="emoji" alt=":heart_eyes:"></p>