Reduce the penalty for misclassifying few classes

Abhilash: <p>I’m a beginner in Computer Vision, Pytorch and working on the multi class classification problem ( MNIST), how do I adjust the penalty for misclassifying, for example, a 5 as a 6, because a human would do similar mistake as they look pretty close ( for a terrible image), or 1 as 7 in few images. The weight parameter in loss function is not really suitable for this, as it’s for imbalanced datasets and I’m not dealing with imbalanced datasets. Do I need to write few if conditions in the custom loss functions or is there a built in way to define less penalty for classifying it wrong between few classes.</p>

KFrank: <p>Hi Abhilash!</p>
<aside class="quote no-group" data-username="Abhilash" data-post="1" data-topic="121305" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhilash/48/38312_2.png" class="avatar"> Abhilash:</div>
<blockquote>
<p>how do I adjust the penalty for misclassifying, for example, a 5 as a 6, because a human would do similar mistake as they look pretty close ( for a terrible image), or 1 as 7 in few images.</p>
</blockquote>
</aside>
<p>The most straightforward approach would be to use cross entropy as<br>
your loss function, but to use probabilistic ("soft’) labels, rather than<br>
categorical (“hard”) labels.</p>
<p>Pytorch does not have a soft-label version of cross-entropy loss built<br>
in, but it is easy to implement one.  See this post:</p>
<aside class="quote quote-modified" data-post="2" data-topic="69501">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/soft-cross-entropy-loss-tf-has-it-does-pytorch-have-it/69501/2">Soft Cross Entropy Loss (TF has it does Pytorch have it)</a> 
  </div>
  <blockquote>
    Hello Raaj! 

I do not believe that pytorch has a “soft” cross-entropy function built in. 
But you can implement it using pytorch tensor operations, so you should 
get the full benefit of autograd and gpu acceleration. 
See this (pytorch version 0.3.0) script: 
import torch
torch.__version__

# define "soft" cross-entropy with pytorch tensor operations
def softXEnt (input, target):
    logprobs = torch.nn.functional.log_softmax (input, dim = 1)
    return  -(target * logprobs).sum() / input.shap…
  </blockquote>
</aside>

<p>This post explains how to use soft labels to reduce the penalty for certain<br>
misclassifications:</p>
<aside class="quote quote-modified" data-post="2" data-topic="114208">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/label-smoothing-for-only-a-subset-of-classes/114208/2">Label smoothing for only a subset of classes</a> 
  </div>
  <blockquote>
    Hi macazinc! 

Here’s what I think you’re asking: 
You have a multi-class (30 classes) classification problem.  You know 
that for most of your classes, your ground-truth target labels are 
correct, but your labels sometimes mix up two of your classes, say 4 
and 9,  Let’s say that a sample labelled 4 is actually a 9 25% of the 
time, and that a sample labelled 9 is actually a 4 10% of the time. 
You will (most likely) want to use cross-entropy loss, but pytorch only 
provides a version that tak…
  </blockquote>
</aside>

<p>Best.</p>
<p>K. Frank</p>