CrossEntropyLoss for float type target

yuri: <p>It is obvious why CrossEntropyLoss() only accepts Long type targets. however, I ran it on Pycharm IDE with float type targets and it worked!! My target is a 2D heatmap with 0 background and a spot that has float values between 0 and 1. The same example raised an error on Colab.<br>
The question is whether Python explicitly assigns a Long type value as a class to every different value?</p>

anantguptadbl: <p><a class="mention" href="/u/yuri">@yuri</a> Please share your code snippet</p>

KFrank: <p>Hi Yuri!</p>
<aside class="quote no-group" data-username="yuri" data-post="1" data-topic="141676" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/71c47a/48.png" class="avatar"> yuri:</div>
<blockquote>
<p>It is obvious why CrossEntropyLoss() only accepts Long type targets.</p>
</blockquote>
</aside>
<p>As of pytorch version 1.10, <a href="https://pytorch.org/docs/1.10.0/generated/torch.nn.CrossEntropyLoss.html" rel="noopener nofollow ugc">CrossEntropyLoss</a> will accept either integer<br>
class labels (<code>torch.int64</code>) or per-class probabilities (<code>torch.float32</code><br>
or <code>torch.float64</code>) as its target.</p>
<aside class="quote no-group">
<blockquote>
<p>however, I ran it on Pycharm IDE with float type targets and it worked!!<br>
…<br>
The same example raised an error on Colab.</p>
</blockquote>
</aside>
<p>I assume that your pycharm platform was using pytorch 1.10 or later<br>
while colab was using a version prior to 1.10.  You could probe this by<br>
printing out <code>torch.__version__</code> on both platforms.</p>
<p>(Using probabilistic (so-called "soft’) labels with <code>CrossEntropyLoss</code> is a<br>
perfectly reasonable thing to do, and it’s nice that pytorch now supports<br>
this directly.)</p>
<p>Best.</p>
<p>K. Frank</p>

yuri: <aside class="quote no-group quote-modified" data-username="KFrank" data-post="3" data-topic="141676">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>torch.<strong>version</strong></p>
</blockquote>
</aside>
<p>Hello K. Frank,<br>
thank you for your reply,<br>
actually, I checked the PyTorch version on Colab and it is 1.10 <strong>BUT</strong> what I also noticed is that the CrossEntropyLoss method internally switches to the probabilistic mode if Input and Target have the same size. The question now how does this mathematically work? especially that Target contains float with arbitrary positive values not just between 0 and 1</p>

KFrank: <p>Hi Yuri!</p>
<aside class="quote no-group" data-username="yuri" data-post="4" data-topic="141676" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/71c47a/48.png" class="avatar"> yuri:</div>
<blockquote>
<p>The question now how does this mathematically work?</p>
</blockquote>
</aside>
<p>The formula for probabilistic mode is given the the <code>CrossEntropyLoss</code><br>
documentation.</p>
<aside class="quote no-group">
<blockquote>
<p>especially that Target contains float with arbitrary positive values not just between 0 and 1</p>
</blockquote>
</aside>
<p>As you have recognized, it is possible to pass in invalid values for<br>
<code>target</code> in the probabilistic case, namely values that do not represent<br>
a valid (discrete) probability distribution.  That is, individual values can<br>
lie outside of <code>[0.0, 1.0]</code>, or the values can sum to a value other than<br>
<code>1.0</code>.</p>
<p><code>CrossEntropyLoss</code> doesn’t validate this condition – it simply applies<br>
the formula given in the documentation and will not return a sensible<br>
result if you violate it.  (For example, if some of the <code>target</code> values are<br>
negative, you could get a negative loss that could cause your training<br>
to diverge.)  It’s up to you to pass in valid <code>target</code> values.</p>
<p>Best.</p>
<p>K. Frank</p>

yuri: <p>Hello K Frank,<br>
thank you for your reply.<br>
I don’t think that Target has to have values between 0.0 and 1.0 as “weights”, the scaling parameter that will be applied to each Class could easily pull the target values out of that range.<br>
Indeed negative values are problematic</p>