Train a custom classifier with limited number of classes

username: silvester
cooked: <p>I need to train a classifier on Cifar10, such that it could classify the images with the labels 0 and 1 and for the labels other than these two labels the classifier provide unbiased (neutral) results. I try to train it using the following code with BCE loss:</p>
<pre><code class="lang-auto">       input = model(x)
       not_ind_tgt = ~((target==0) | (target==1))
       tgt_hot = F.one_hot(target, 10).float()
       multi_label = 1./8. * torch.ones(1,10).float()
       multi_label[0, 0: 2] = 0.0
       tgt_hot[not_ind_tgt] = multi_label
       loss = nn.BCEWithLogitsLoss()(input, tgt_hot).mean()
</code></pre>
<p>However, the accuracy of the trained classifier on 0 and 1 classes is zero.</p>

username: chetan_patil
cooked: <ol>
<li>If you look at the docs, <code>nn.BCEWithLogitsLoss()</code> expects input and target of shape <code>(N,*)</code>.<br>
Essentially, this means that if you use input of shape <code>(N,10)</code>, then 10 individual BCE-Loss() will be performed, its sort of a multi-label BCE which is not your end goal.</li>
<li>You must rather use a <code>nn.CrossEntropyLoss()</code> because you have two labels which are explicitly coded as zeroth and first class.</li>
</ol>
<p>For example, let’s say <code>input</code> is of shape <code>(3,10)</code>, with 3 being the batch-size.<br>
Your <code>target</code> must be of shape <code>(3,1)</code> or just <code>(3, )</code> and each entry in this variable will be the class associated for that particular batch.<br>
In your case, the <code>target</code> must span between only 0 and 1, since you are focusing only on the zeroth and first class.</p>

username: silvester
cooked: <p>The data outside of these two labels is in my dataset and I want the classifier be neutral to those labels, i.e., gives zero probability for labels 0 and 1 for images which have different labels than 0 and 1. So, the targets can not be the span of 0 and 1.</p>

username: chetan_patil
cooked: <p>Okay, got it.!<br>
In that case, you could use something like <code>class_weights</code> in the  loss function.<br>
For more details, you can see the link <a href="https://pytorch.org/docs/stable/nn.html#crossentropyloss" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.html#crossentropyloss</a></p>

username: silvester
cooked: <p>I have 10 classes and each class has 1000 images in the dataset. It isn’t clear for me what are the weights in this case.<br>
Do you think this could gives good accuracy on 0 and 1 labels? I tried but I failed.</p>

username: chetan_patil
cooked: <p>You can find the class-weights using this link <a href="https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html" rel="nofollow noopener">https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html</a><br>
And just to make things clear about your problem of classifier providing neutral results.<br>
Consider a batchsize of 1.</p>
<ol>
<li>When target is the 0th class. Loss will propogate normally.</li>
<li>When target is the 1st class. Loss will propogate normally.</li>
<li>When target is between 2nd to 9th class. How should the loss propogate ?</li>
</ol>

username: silvester
cooked: <p>I applied the extreme weights [10, 10, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01,], however the accuracy of the classes 0 and 1 is still lower than the others.</p>