Cross Entropy Loss error on image segmentation

joekid: <p>I am a new user of Pytorch. I am adapting the Unet segmentation model, but I have an error in the evaluation of the Cross Entropy Loss  function during training.</p>
<p>I used  torch.utils.data.Dataset to build a specific dataset</p>
<p>train_data = DataLoaderSegmentation(train_path, mode=‘train’)<br>
train_loader = DataLoader(train_data, batch_size = 4, shuffle=True, num_workers=4)</p>
<p>which works fine and load both the image and its mask in the format:</p>
<p>[batch,channel,W,H] , like  [4,1,220,220], for both images and masks. I put channel to 1 since for background and foreground segmentation.</p>
<p>Parameters for train:</p>
<p>model = Unet(n_filters=32, n_class=1, input_channels=1)<br>
device = ‘cuda’<br>
LR = 0.001<br>
criterion = nn.CrossEntropyLoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr=LR)<br>
model = model.to(device)</p>
<p>During training I have an error on the marked line:</p>
<p>for data, target in train_loader:<br>
data = data.to(device)<br>
target = target.to(device)<br>
optimizer.zero_grad()<br>
output = model(data)<br>
loss = criterion(output, target) ############## ERROR</p>
<p>for data, target in train_loader:<br>
data = data.to(device)<br>
target = target.to(device)<br>
optimizer.zero_grad()<br>
output = model(data)<br>
loss = criterion(output, target) ######## ERROR</p>
<hr>
<p><strong>File “C:\Anaconda3\envs\envTorch\lib\site-packages\torch\nn\functional.py”, line 1840, in nll_loss</strong><br>
**    ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)**</p>
<p><strong>RuntimeError: Expected object of scalar type Long but got scalar type Float for argument <span class="hashtag">#2</span> ‘target’ in call to _thnn_nll_loss2d_forward</strong></p>
<hr>
<p>I see that  target and output have the same dim [4,1,220,220] but  the criterion(output, target) trow error.</p>
<p>Sorry, but after read some answer, I cannot resolve it yet.</p>

Megh_Bhalerao: <p>Do <code>output.float()</code> and <code> target.float()</code> and see if it works.</p>

joekid: <p><a class="mention" href="/u/megh_bhalerao">@Megh_Bhalerao</a>,  It gives the same error.</p>
<p>RuntimeError: Expected object of scalar type Long but got scalar type Float for argument <span class="hashtag">#2</span> ‘target’ in call to _thnn_nll_loss2d_forward</p>
<p>Would the error come from the model?</p>
<hr>
<p>loss = criterion(output.float(), target.float())<br>
Traceback (most recent call last):</p>
<p>File “”, line 1, in <br>
loss = criterion(output.float(), target.float())</p>
<p>File “C:\Anaconda3\envs\envTorch\lib\site-packages\torch\nn\modules\module.py”, line 541, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)</p>
<p>File “C:\Anaconda3\envs\envTorch\lib\site-packages\torch\nn\modules\loss.py”, line 916, in forward<br>
ignore_index=self.ignore_index, reduction=self.reduction)</p>
<p>File “C:\Anaconda3\envs\envTorch\lib\site-packages\torch\nn\functional.py”, line 2009, in cross_entropy<br>
return nll_loss(<strong>log_softmax</strong>(input, 1), target, weight, None, ignore_index, None, reduction)</p>
<p>File “C:\Anaconda3\envs\envTorch\lib\site-packages\torch\nn\functional.py”, line 1840, in nll_loss<br>
ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)</p>
<p>RuntimeError: Expected object of scalar type Long but got scalar type Float for argument <span class="hashtag">#2</span> ‘target’ in call to _thnn_nll_loss2d_forward</p>
<hr>
<p>log_softmax,</p>
<p>the last activation function in the model is a nn.Sigmoid(), no a nn.Softmax(), it could be?</p>

Megh_Bhalerao: <p>Sorry, I meant try <code>criterion(ouput.double(),target.double())</code><br>
Let me know if this works</p>

joekid: <p>sorry, it doesn’t work.</p>

joekid: <p>I only changed the loss function to</p>
<p><code>criterion = torch.nn.BCEWithLogitsLoss()</code></p>
<p>and the error disappeared. However during training the accuracy showed is always low.</p>

ptrblck: <p>If you are dealing with a binary classification use case, a single output channel and <code>nn.BCEWithLogitsLoss</code> should be working.<br>
Make sure to pass raw logits to the criterion (no sigmoid at the end).</p>
<p><code>nn.CrossEntropyLoss</code> is usually applied for multi class classification/segmentation use cases, where you are dealing with more than two classes.<br>
In this case, your target should be a <code>LongTensor</code>, should not have the channel dimension, and should contain the class indices in <code>[0, nb_classes-1]</code>.</p>
<p>Since your accuracy is low, could you try to overfit a small data sample and see, if your model and training routine can successfully overfit this sample?</p>

joekid: <p>Thank you for your comments. I can’t make it work (I no tested more <code>nn.BCEWithLogitsLoss</code>, I focused in N=5 multi class segmentation). Images are [1,220,220] and  their mask [5,220,220]. Each channel is a binary image with values 0 and 1, 1s for the object of interest on the respective channel, and 0s for the background. I don’t know if this mask form is correct.</p>
<p>The error occurs in the line</p>
<blockquote>
<p>loss = criterion(output, target) ,</p>
</blockquote>
<p>during the first pass. The error seems related to data type, but If I am not mistaken, internal tensors work with float numbers (all images and masks are floats).</p>
<p>Here is my code:</p>
<p>model = UnetModel(n_filters=32, n_class=5, input_channels=1)<br>
device = ‘cuda’<br>
model = model.to(device)</p>
<p>criterion = torch.nn.CrossEntropyLoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)</p>
<p>model.train()</p>
<p>epochs = 20<br>
itr = 1<br>
p_itr = 50<br>
total_loss = 0<br>
train_loss = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
train_acc = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<p>for epoch in range(epochs):<br>
for data, target in train_loader:</p>
<pre><code>    data = data.to(device)
    target = target.to(device)
    
    optimizer.zero_grad()
    output = model(data) 
    loss = criterion(output, target) ######## **Error**
    loss.backward()  
    optimizer.step() 
            
    total_loss += loss.item()        
    if itr%p_itr == 0:
        pred = torch.argmax(output, dim=1)
        correct = pred.eq(target)
        acc = torch.mean(correct.float())
        print('[Epoch {}/{}] Iteration {} -&gt; Train Loss: {:.4f}, Accuracy: {:.3f}'.format(epoch+1, epochs, itr, total_loss/p_itr, acc))
        train_loss.append(total_loss/p_itr)
        train_acc.append(acc)
        total_loss = 0            
    itr += 1
</code></pre>
<p><strong>The error is</strong><br>
File “C:\Anaconda3\envs\envTorch\lib\site-packages\torch\nn\functional.py”, line 1840, in nll_loss<br>
ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)</p>
<p><strong>RuntimeError</strong>: Expected object of scalar type <strong>Long</strong> but got scalar type <strong>Float</strong> for argument <span class="hashtag-raw">#2</span> ‘target’ in call to _thnn_nll_loss2d_forward</p>

KFrank: <p>Hi Joe!</p>
<aside class="quote no-group quote-modified" data-username="joekid" data-post="8" data-topic="60194" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/joekid/48/22490_2.png" class="avatar"> joekid:</div>
<blockquote>
<p>Thank you for your comments. I can’t make it work<br>
…</p>
<p>The error occurs in the line</p>
<blockquote>
<p>loss = criterion(output, target) ,</p>
</blockquote>
<p>Here is my code:</p>
<p>…</p>
<p>criterion = torch.nn.CrossEntropyLoss()<br>
…</p>
<p><strong>The error is</strong><br>
…<br>
<strong>RuntimeError</strong>: Expected object of scalar type <strong>Long</strong> but got scalar type <strong>Float</strong> for argument <span class="hashtag-raw">#2</span> ‘target’ in call to _thnn_nll_loss2d_forward</p>
</blockquote>
</aside>
<p>The error (“scalar type <strong>Long</strong>”) and the documentation for<br>
your loss function, <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc">CrossEntropyLoss</a>, should serve to sort<br>
things out.</p>
<p>As <a class="mention" href="/u/ptrblck">@ptrblck</a> mentioned in one of his posts, above,<br>
<code>CrossEntropyLoss</code> takes a <code>LongTensor</code> for its targets,<br>
not a <code>FloatTensor</code>, hence the error.</p>
<p><code>CrossEntropyLoss</code> (perhaps better called more explicitly<br>
“categorical cross-entropy loss”) requires integer class labels<br>
in <code>[0, nClass - 1]</code> for its targets.  Provided the <code>target</code> in<br>
your code is, indeed, class labels (as compared to probabilities,<br>
or some such), it should suffice to convert it to a <code>LongTensor</code>,<br>
e.g., <code>loss = criterion (output, target.long())</code>.</p>
<p><code>BCELossWithLogits</code> does take a <code>FloatTensor</code> for its targets.<br>
(It takes probabilities, not just <code>0,1</code> class labels.)  So that’s why<br>
you didn’t get the error when you tried <code>BCELossWithLogits</code>.</p>
<p>Good luck.</p>
<p>K. Frank</p>

joekid: <p>I am getting a new error.</p>

joekid: <p>Dear <a class="mention" href="/u/kfrank">@KFrank</a>.</p>
<ol>
<li>I remake the target as [batch, nClasses, H, W], being nClasses = 5. I split  the objects labels along layers as follows (but I don’t know if it is correct):</li>
</ol>
<p>Layer 0: image with all  pixels value to zero, label 0 = background<br>
Layer 1:  pixels value to 1 for object 1 (e.g. dog), and 0s for the background<br>
Layer 2:  pixels value to 2 for object 2 (e.g. cat), and 0s for the background<br>
and son on.</p>
<p><strong>is it a reasonable model for the layers in semantic segmentation?</strong></p>
<ol start="2">
<li>However a get a new error:</li>
</ol>
<p><code>RuntimeError: 1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [4, 5, 224, 224] ,</code></p>
<p>I use batchsize=5, nClasses=1.  The input data is [4,1,224,244] aiming to obtain [4,5,224,244],</p>

KFrank: <p>Hello Joe!</p>
<aside class="quote no-group" data-username="joekid" data-post="11" data-topic="60194" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/joekid/48/22490_2.png" class="avatar"> joekid:</div>
<blockquote>
<ol>
<li>I remake the target as [batch, nClasses, H, W], being nClasses = 5.</li>
</ol>
</blockquote>
</aside>
<p>No, this is not right.  Your <code>target</code> (“ground truth”) should be (a<br>
<code>LongTensor</code>) of shape <code>(batch, H, W)</code>.  For any given sample<br>
within the batch, the values of the  “pixels” in your “H x W” “image”<br>
should be the integer class labels in <code>[0, 5)</code>.</p>
<p>You don’t use multiple “Layers” to specify which class a given<br>
pixel is in – you just use an integer class label as the value of<br>
that pixel.</p>
<p>Note, the <code>input</code> to <code>CrossEntropyLoss</code> is <em>not</em> of the same shape<br>
as the <code>target</code>; the <code>input</code> is of shape<br>
<code>(batch, nClasses, H, W)</code>.  For any given sample in the batch,<br>
each pixel in the H x W input “image” has <code>nClasses</code> values.  These<br>
are the “logits” (“scores” in -infinity to +infinity) that tell you how likely<br>
the model thinks a given pixel is to be in each of the <code>nClasses</code><br>
classes.</p>
<aside class="quote no-group">
<blockquote>
<ol start="2">
<li>However a get a new error:</li>
</ol>
<p><code>RuntimeError: 1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [4, 5, 224, 224] ,</code></p>
<p>I use batchsize=5, nClasses=1.  The input data is [4,1,224,244] aiming to obtain [4,5,224,244],</p>
</blockquote>
</aside>
<p>This error message looks reasonable in light of what I said above<br>
(except that it looks like you are actually using <code>batchsize = 4</code>,<br>
not <code>batchsize = 5</code>).</p>
<p>Assuming <code>batchsize = 4</code>, <code>nClasses = 5</code>, <code>H = 224</code>, and<br>
<code>W = 224</code>, <code>CrossEntropyLoss</code> will be expecting the <code>input</code><br>
(prediction) you give it to be a <code>FloatTensor</code> of shape<br>
<code>(4, 5, 244, 244)</code>, and the <code>target</code> (ground truth) to be a<br>
<code>LongTensor</code> of shape (4, 244, 244).</p>
<p>Good luck.</p>
<p>K. Frank</p>

joekid: <aside class="quote no-group" data-username="KFrank" data-post="12" data-topic="60194">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Assuming <code>batchsize = 4</code> , <code>nClasses = 5</code> , <code>H = 224</code> , and<br>
<code>W = 224</code> , <code>CrossEntropyLoss</code> will be expecting the <code>input</code><br>
(prediction) you give it to be a <code>FloatTensor</code> of shape<br>
<code>(4, 5, 244, 244)</code> , and the <code>target</code> (ground truth) to be a<br>
<code>LongTensor</code> of shape (4, 244, 244).</p>
</blockquote>
</aside>
<p>Dear <a class="mention" href="/u/kfrank">@KFrank</a> you hit the nail, thank you. Thank you.</p>
<p>The target is a single image HxW, each pixel labeled as belonging to [0…nClasses-1]</p>
<p><strong>loss = CrossEntropyLoss(input, target)</strong>  in the specified  size computes correctly (I was too wrong).</p>
<p>Can you suggest me some place to learn the keys of DL, I am starting on it. Thank you again.</p>