CrossEntropyLoss for Image Segmentation Error

Lukas_Lansche: <p>Hi, I’m a little stuck with the CrossEntropyLoss,<br>
I have a dataset with 500 Images all pixelwise labeled for Semantic Segmentation. The Dataset contains 5 classes, now the problem is that one class covers about 84% of all pixels.<br>
That’s why I wanted to use the CrossEntropyLoss, to weight the other classes higher.</p>
<p>I’m stuck with this error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\segmentation_models_pytorch\utils\train.py", line 47, in run
    loss, y_pred = self.batch_update(x, y)
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\segmentation_models_pytorch\utils\train.py", line 104, in batch_update
    loss = self.loss(prediction, y.long())
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\torch\nn\modules\loss.py", line 916, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\torch\nn\functional.py", line 2021, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\torch\nn\functional.py", line 1840, in nll_loss
    ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: 1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [2, 5, 928, 928]

Process finished with exit code 1

</code></pre>
<p>both tensors, input and target have the same size of  [2, 5, 928, 928] with Batch_size = 2, number_classes = 5 and the image Size [928, 928]</p>
<p>any suggestions what I’m doing wrong ? I also tried the target as a not one-hot encoded tensor but ended in another error …</p>

KFrank: <p>Hi Lukas!</p>
<aside class="quote no-group quote-modified" data-username="Lukas_Lansche" data-post="1" data-topic="79141" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/lukas_lansche/48/23553_2.png" class="avatar"> Lukas_Lansche:</div>
<blockquote>
<p>Hi, I’m a little stuck with the CrossEntropyLoss,<br>
…<br>
both tensors, input and target have the same size of  [2, 5, 928, 928] with Batch_size = 2, number_classes = 5 and the image Size [928, 928]</p>
</blockquote>
</aside>
<p>This is your problem.  <code>CrossEntropyLoss</code> expects integer<br>
class labels for its <code>target</code>.  So <code>target</code> should have shape<br>
<code>[2, 928, 928]</code>, and the <code>target</code> values should be integers<br>
that run from <code>0</code> to <code>4</code>.</p>
<aside class="quote no-group">
<blockquote>
<p>I also tried the target as a not one-hot encoded tensor but ended in another error …</p>
</blockquote>
</aside>
<p>Try again not one-hotting your <code>target</code>.  If it still doesn’t work, please<br>
post the code where you instantiate <code>CrossEntropyLoss</code>, the code<br>
where you call your loss function, the shape of your <code>input</code> and<br>
<code>target</code>, and any error messages you get.</p>
<p>Good luck.</p>
<p>K. Frank</p>

Lukas_Lansche: <p>Hi Frank , thank you for your reply here is my function, where I call the loss function:</p>
<pre><code class="lang-auto">    def batch_update_cross(self, x, y):
        self.optimizer.zero_grad()
        prediction = self.model.forward(x)
        depth = y.shape[1]
        merged_mask = torch.zeros([2, 928 ,928])
        for j in range(depth):
            merged_mask[y[:, j, :, :] == 1] = int(j + 1)
        y = merged_mask.long().cuda()
        loss = self.loss(prediction, y)
        loss.backward()
        self.optimizer.step()
        return loss, prediction
</code></pre>
<p>I first tried it with :</p>
<pre><code class="lang-auto">y = merged_mask.int().cuda()
</code></pre>
<p>but then I got this error:</p>
<pre><code class="lang-auto">RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #2 'target' in call to _thnn_nll_loss2d_forward
</code></pre>
<p>So i changed .int() into .long() and this following error appears:</p>
<pre><code class="lang-auto">C:/w/1/s/tmp_conda_3.7_100118/conda/conda-bld/pytorch_1579082551706/work/aten/src/THCUNN/SpatialClassNLLCriterion.cu:104: block: [9,0,0], thread: [526,0,0] Assertion `t &gt;= 0 &amp;&amp; t &lt; n_classes` failed.
Traceback (most recent call last):
  File "C:/Users/lukas/Documents/SchneidbiegeRoboter/PyTorch/Pytroch_train_model.py", line 150, in &lt;module&gt;
    m, o = train(train_idx=training_idx, val_idx=validation_idx, retrain=settings.retrain)
  File "C:/Users/lukas/Documents/SchneidbiegeRoboter/PyTorch/Pytroch_train_model.py", line 91, in train
    train_logs = train_epoch.run(train_loader)
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\segmentation_models_pytorch\utils\train.py", line 47, in run
    loss, y_pred = self.batch_update_cross(x, y)
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\segmentation_models_pytorch\utils\train.py", line 94, in batch_update_cross
    loss.backward()
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\torch\tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "C:\Users\lukas\anaconda3\envs\Python\lib\site-packages\torch\autograd\__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED
</code></pre>
<p>now there could be one issue I’m thinking of, my class labels aren’t 0 to 4, they are 1 to 5 does this make any difference ?</p>
<p>Cheers Lukas</p>

KFrank: <p>Hello Lukas!</p>
<aside class="quote no-group" data-username="Lukas_Lansche" data-post="3" data-topic="79141" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/lukas_lansche/48/23553_2.png" class="avatar"> Lukas_Lansche:</div>
<blockquote>
<p>I first tried it with :</p>
<pre><code class="lang-auto">y = merged_mask.int().cuda()
</code></pre>
<p>but then I got this error:</p>
<pre><code class="lang-auto">RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #2 'target' in call to _thnn_nll_loss2d_forward
</code></pre>
</blockquote>
</aside>
<p>This makes sense.  <code>CrossEntropyLoss</code> requires <code>long</code> for its<br>
target.</p>
<aside class="quote no-group">
<blockquote>
<p>So i changed .int() into .long() and this following error appears:</p>
<pre><code class="lang-auto">RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED
</code></pre>
</blockquote>
</aside>
<p>This looks like you’re having a cuda / gpu error.</p>
<p>First try running your code entirely on your cpu.  (Get rid of any<br>
<code>.cuda()</code> calls so that all of your tensors – model, input, target,<br>
etc. – are on the cpu.)  See if you can get this to work.</p>
<p>Then try something simple on the gpu.  Create two <code>FloatTensor</code>s,<br>
move them to the gpu, and then try to add them together, or the<br>
like.</p>
<p>If this works, but your real code keeps giving <code>cuDNN error</code>, then<br>
there will be more debugging to do.</p>
<aside class="quote no-group">
<blockquote>
<p>now there could be one issue I’m thinking of, my class labels aren’t 0 to 4, they are 1 to 5 does this make any difference ?</p>
</blockquote>
</aside>
<p>Yes, this will also be a problem (but it doesn’t explain the cuda<br>
error).  The target for <code>CrossEntropyLoss</code> must be (long) integer<br>
class labels that run from <code>0</code> to <code>nClass - 1</code>.  If they include a<br>
value of <code>nClass</code> (or larger), you will get an error – “index out<br>
of range”, or something similar, I think.</p>
<p>Good luck.</p>
<p>K. Frank</p>

Lukas_Lansche: <p>Hi Frank,<br>
Thank you so much for your advice.  I got it running!<br>
I did exactly what you said, tried it with the <code>cpu</code> and got the following error:</p>
<pre><code class="lang-auto">IndexError: Target 5 is out of bounds.
</code></pre>
<p>So then I rewrote my class labels to a range of <code>0</code> to <code>nClass - 1</code> and tried again and it worked for <code>cpu</code> and for <code>cuda</code>.  Thank you so much !</p>