Error : "host_softmax" not implemented for 'Long'

abhi8569: <p>I am trying to train a model for image segmentation. I have 3 different class to segment whihc is denoted by [0,1,2] in the ground truth image. These are the output from different steps:</p>
<pre><code class="lang-auto">Model Output : torch.Size([5, 3, 120, 160]) #batch,channel,height,width
Argmax Output : torch.Size([5, 120, 160]) #check maximum along the channel axis
Ground Truth : torch.Size([5, 120, 160])
</code></pre>
<p>I am using below code snippet to find the loss using CrossEntropy</p>
<pre><code class="lang-auto">loss_seg = criterion_seg(x, label)
#x = output from argmax
#label = ground truth
</code></pre>
<p>While doing so I am getting below error</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-40-e1fd37d7abc6&gt; in &lt;module&gt;
      8 
      9     print("epoch",epoch)
---&gt; 10     train_loss, model = train_model(model, "seg", seg_train_loader, criterion_blob, criterion_seg, optimizer, avDev)
     11     loss_details[epoch] = train_loss
     12     print("train loss",train_loss)

&lt;ipython-input-29-f9e95c6cbb49&gt; in train_model(model, mode, train_loader, criterion_blob, criterion_seg, optimizer, device)
     36 #             print(label.size())
     37             optimizer.zero_grad()
---&gt; 38             loss_seg = criterion_seg(torch.tensor(x, dtype=torch.long, device=device), label)
     39 
     40 

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--&gt; 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    914     def forward(self, input, target):
    915         return F.cross_entropy(input, target, weight=self.weight,
--&gt; 916                                ignore_index=self.ignore_index, reduction=self.reduction)
    917 
    918 

~/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2007     if size_average is not None or reduce is not None:
   2008         reduction = _Reduction.legacy_get_string(size_average, reduce)
-&gt; 2009     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2010 
   2011 

~/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in log_softmax(input, dim, _stacklevel, dtype)
   1315         dim = _get_softmax_dim('log_softmax', input.dim(), _stacklevel)
   1316     if dtype is None:
-&gt; 1317         ret = input.log_softmax(dim)
   1318     else:
   1319         ret = input.log_softmax(dim, dtype=dtype)

RuntimeError: "host_softmax" not implemented for 'Long'
</code></pre>
<p>Kindly help me to get rid of this error.<br>
TIA</p>

KFrank: <p>Hi Abishek!</p>
<aside class="quote no-group" data-username="abhi8569" data-post="1" data-topic="74951" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhi8569/48/21951_2.png" class="avatar"> abhi8569:</div>
<blockquote>
<p>I have 3 different class to segment whihc is denoted by [0,1,2] in the ground truth image. These are the output from different steps:</p>
<pre><code class="lang-auto">Model Output : torch.Size([5, 3, 120, 160]) #batch,channel,height,width
Argmax Output : torch.Size([5, 120, 160]) #check maximum along the channel axis
Ground Truth : torch.Size([5, 120, 160])
</code></pre>
<p>I am using below code snippet to find the loss using CrossEntropy</p>
<pre><code class="lang-auto">loss_seg = criterion_seg(x, label)
#x = output from argmax
#label = ground truth
</code></pre>
</blockquote>
</aside>
<p>I assume that <code>criterion_seg</code> is something you wrote yourself<br>
that basically forwards its arguments to <code>F.cross_entropy()</code>.</p>
<p>You most likely should pass <code>Model Output</code> in place of <code>x</code> as<br>
the first argument of <code>criterion_seg</code>, and don’t use <code>argmax()</code><br>
at all.</p>
<aside class="quote no-group">
<blockquote>
<p>While doing so I am getting below error</p>
<pre><code class="lang-auto">RuntimeError: "host_softmax" not implemented for 'Long'
</code></pre>
</blockquote>
</aside>
<p>This is (most likely) telling you that your are passing the <code>Long</code><br>
result of <code>argmax()</code> to <code>F.cross_entropy()</code> which is expecting<br>
<code>Float</code> as its “predictions” <code>input</code>.  (<code>cross_entropy()</code>'s<br>
<code>target</code> – your <code>label</code> – should, however, be a <code>LongTensor</code><br>
containing integer class labels ranging over <code>[0, 1, 2]</code>).</p>
<p>Without seeing your code for <code>criterion_seg()</code> I don’t really<br>
know, but I would have expected you to get a dimensions<br>
mismatch error before the type error.</p>
<p>Based on what I think you are doing, <code>x</code> should be your model<br>
output, a <code>FloatTensor</code> of shape <code>[5, 3, 120, 160]</code>, where<br>
<code>nBatch = 5</code>, and <code>nClass = 3</code>, containing raw-score <em>logits</em><br>
that run from <code>-inf</code> to <code>inf</code>.  <code>label</code> should be your ground truth,<br>
a <code>LongTensor</code> of shape <code>[5, 120, 160]</code>, containing integer<br>
class labels in <code>[0, nClass - 1]</code>, inclusive.</p>
<p>The point is you <em>don’t</em> want to convert a set of <code>nClass</code> logits<br>
(or probabilities) to a single integer class label (by using<br>
<code>argmax()</code>) and then pass it to <code>cross_entropy()</code>, because<br>
that is not what <code>cross_entropy()</code> is expecting.</p>
<p>Good luck.</p>
<p>K. Frank</p>

abhi8569: <p>Hi Frank,</p>
<p>Thank you again for the solution. Just passing the output from the model into the Cross_Entropy() worked.</p>
<p>Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Ayush_Aditya: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/1/91a7d090804e36cf4ba9a07ce8cecf78ef88447d.png" alt="image" data-base62-sha1="kMwM1pRm0FGM0AyYTQ48QUc67Zb" width="663" height="406"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d76618904776d37eae53d465ba662ef714f3c35.png" alt="image" data-base62-sha1="4cDtyF5DrA9xdCH10C7TF6byd9P" width="659" height="403"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/8/a/8ad618eb8fe99167e884b39d9984b74bd1aaeb90.jpeg" alt="image" data-base62-sha1="jOcAJ3BdCuQ0MqMd2DhcJLA424g" width="659" height="413"><br>
facing the same issue and don’t know how debug</p>

ptrblck: <p>You are passing the arguments in the wrong order. Use <code>loss_function(outputs, labels)</code> and it should work.<br>
Also, you can post code snippets by wrapping them into three backticks ```, which makes debugging easier.</p>

Ayush_Aditya: <p>thanks and I didn’t understand what you meant by adding three backticks. Can you give me an example.</p>