Custom Dice Loss Function

username: DoubleRedX
cooked: <p>Hi guys,<br>
met some problem with the Dice loss function:</p>
<p>class DiceLoss(nn.Module):<br>
def <strong>init</strong>(self, weight=None, size_average=True):<br>
super(DiceLoss, self).<strong>init</strong>()<br>
def forward(self, inputs, targets, smooth=1):<br>
inputs = functional.sigmoid(inputs)<br>
inputs = inputs.view(-1)<br>
targets = targets.view(-1)<br>
intersection = (inputs*targets).sum()<br>
dice = (2.*intersection + smooth)/(inputs.sum() + targets.sum() + smooth)</p>
<pre><code>    return 1-dice
</code></pre>
<p>def main():<br>
…<br>
loss_ = loss_fn(pred, label)<br>
…</p>
<p>loss_fn in main() is DiceLoss()<br>
and<br>
pred shape: torch.Size([16, 4, 240, 240])  （batchsize, num_classes, width, height）<br>
label shape: torch.Size([16, 240, 240])  (batchsize, width, height)(the element of tensor is in [0, 1, 2, 3], represent pixel class)<br>
After flatten op:<br>
pred  shape: torch.Size([3686400])<br>
label shape: torch.Size([921600])</p>
<p>THE BUG :RuntimeError: The size of tensor a (3686400) must match the size of tensor b (921600) at non-singleton dimension 0</p>
<p>what I am curious about is that there is no bug when I am using nn.CrossEntropyLoss() as my loss_fn,but when I used DiceLoss() , it turned to have a bug above.<br>
Please help me why this bug occurs and how to achieve DiceLoss to get it worked.<br>
Thank you so MUCH!!</p>

username: KFrank
cooked: <p>Hi Double!</p>
<aside class="quote no-group quote-modified" data-username="DoubleRedX" data-post="1" data-topic="136215" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/doubleredx/48/43752_2.png" class="avatar"> DoubleRedX:</div>
<blockquote>
<p>pred shape: torch.Size([16, 4, 240, 240])  （batchsize, num_classes, width, height）<br>
label shape: torch.Size([16, 240, 240])  (batchsize, width, height)(the element of tensor is in [0, 1, 2, 3], represent pixel class)<br>
…<br>
there is no bug when I am using nn.CrossEntropyLoss() as my loss_fn</p>
</blockquote>
</aside>
<p>At issue is that your <code>DiceLoss</code> function is expecting labels that<br>
are different in form than those used by <code>CrossEntropyLoss</code> (and<br>
there is no reason to have expected them to be the same).</p>
<p>For a given batch sample and given pixel, <code>CrossEntropyLoss</code><br>
compares a vector of <code>num_classes</code> <em>logits</em> (akin to probabilities)<br>
to a <em>single</em> integer class label.</p>
<p>In contrast, your <code>DiceLoss</code> compares a vector of predicted class<br>
probabilities with a vector of <code>targets</code> class probabilities.  (Either of<br>
these could happen to be – or be restricted to – probabilities that<br>
are all exactly either <code>0.0</code> or <code>1.0</code> (but if you convert your predictions<br>
to <code>0.0</code>–<code>1.0</code> probabilities, you will lose differentiability and not be<br>
able to train).</p>
<aside class="quote no-group">
<blockquote>
<p>class DiceLoss(nn.Module):<br>
def forward(self, inputs, targets, smooth=1):<br>
inputs = functional.sigmoid(inputs)</p>
</blockquote>
</aside>
<p>You will most likely prefer to use <code>softmax()</code> to convert your logit<br>
predictions to probabilities:</p>
<pre data-code-wrap="python"><code class="lang-python">        inputs = torch.nn.functional.softmax (inputs, dim = 1)
</code></pre>
<p>You will need to convert your integer-class-label <code>targets</code> to<br>
(<code>0.0</code>–<code>1.0</code>) class probabilities, most conveniently with <code>one_hot()</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">        targets = torch.nn.functional.one_hot (inputs, num_classes = num_classes).transpose (-1, 1)
</code></pre>
<p>(Please think about the shapes and structures of your tensors to be<br>
sure that you understand why the <code>.transpose()</code> is necessary.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: DoubleRedX
cooked: <p>Hi, Frank:<br>
Thank you so so much!<br>
Your answer has solved my problem. Appreciate that!!!<br>
In fact, I changed my code with your advice and it worked. Actually, I met some little problem like,</p>
<ol>
<li>Wrong Tensor Type: such as *(ops) indicated a bug:0 element has no grad_fn<br>
2.dot() was not implemented for Long()<br>
(These 2 bug I have solved)<br>
So I am thinking, Could you please recommend me some lessons or something that I can learn pytorch systematically ? There are always too many bugs, dealing with them one by one is not efficient…<br>
Thanks a lot !!!<br>
Best.</li>
</ol>

username: KFrank
cooked: <p>Hi Double!</p>
<aside class="quote no-group" data-username="DoubleRedX" data-post="3" data-topic="136215" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/doubleredx/48/43752_2.png" class="avatar"> DoubleRedX:</div>
<blockquote>
<p>Could you please recommend me some lessons or something that I can learn pytorch systematically ?</p>
</blockquote>
</aside>
<p>I would recommend the various <a href="https://pytorch.org/tutorials/" rel="noopener nofollow ugc">Pytorch Tutorials</a>.  Work through some<br>
of the basic tutorials and then maybe through some others that catch<br>
your eye or are relevant to the projects you are working on.</p>
<p>I haven’t seen nor read it, but Tom V, one of the pytorch developers<br>
and forum participants, has a book out, “Deep Learning with PyTorch,”<br>
and it’s bound to be good, based on Thomas’s knowledge and<br>
experience.</p>
<p>(And of course – read the documentation!)</p>
<p>The bad news is that there will always be too many bugs that you will<br>
have to deal with one by one.  But they do get easier to find and fix as<br>
you gain more systematic knowledge of pytorch.</p>
<p>Best.</p>
<p>K. Frank</p>

username: DoubleRedX
cooked: <p>Hi Frank!<br>
Thank you SO MUCH!! I will do the tutorials and also the “Deep Learning with PyTorch”.You are right about the bugs.Thanks again!!<br>
Best.</p>