CrossEntropyLoss() function in PyTorch

Shar: <p>Hello,<br>
I tried to search for this question in the internet, but I didn’t find a strict answer. I’m confused.</p>
<p>How is the cross entropy loss is calculated using torch.nn.CrossEntropyLoss() ?</p>
<p>Is it the sum of log probabilities of the correct class? or is it the sum of log probabilities of the correct class + log of (1 -probabilities) of the wrong classes?</p>
<p>Because in the first case it will a negative log likelihood if I get it correctly.</p>
<p>Thanks</p>

mMagmer: <p><a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html</a></p>
<blockquote>
<p>Note that this case is equivalent to the combination of <a href="https://pytorch.org/docs/stable/generated/torch.nn.LogSoftmax.html#torch.nn.LogSoftmax" rel="noopener nofollow ugc"><code>LogSoftmax</code></a> and <a href="https://pytorch.org/docs/stable/generated/torch.nn.NLLLoss.html#torch.nn.NLLLoss" rel="noopener nofollow ugc"><code>NLLLoss</code></a>.</p>
</blockquote>

Shar: <p>Thanks for your reply!</p>
<p>I just wonder, what is the weight <code>w_y_n</code> that is multiplied with the log probability?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/f/3fbd2df9a8891d24728a31552f2f8be61ad79a48.png" data-download-href="https://discuss.pytorch.org/uploads/default/3fbd2df9a8891d24728a31552f2f8be61ad79a48" title="Screenshot 2021-12-09 at 5.39.13 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fbd2df9a8891d24728a31552f2f8be61ad79a48_2_690x257.png" alt="Screenshot 2021-12-09 at 5.39.13 PM" data-base62-sha1="95RoynVMkOk04VGeIe6dveKIwbu" width="690" height="257" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fbd2df9a8891d24728a31552f2f8be61ad79a48_2_690x257.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fbd2df9a8891d24728a31552f2f8be61ad79a48_2_1035x385.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fbd2df9a8891d24728a31552f2f8be61ad79a48_2_1380x514.png 2x" data-dominant-color="EFEFF0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2021-12-09 at 5.39.13 PM</span><span class="informations">1904×710 122 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Eta_C: <pre data-code-wrap="python"><code class="lang-python">torch.nn.CrossEntropyLoss(weight=None, size_average=None, ignore_index=- 100, reduce=None, reduction='mean', label_smoothing=0.0)
</code></pre>
<p>Note that <code>CrossEntropyLoss</code> has an option <code>weight=None</code> and</p>
<blockquote>
<ul>
<li><strong>weight</strong> (<a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor" rel="noopener nofollow ugc"><em>Tensor</em></a><em>,</em> <em>optional</em>) – a manual rescaling weight given to each class. If given, has to be a Tensor of size C</li>
</ul>
</blockquote>