How to train with imbalance class for classification?

John1231983: <p>Hello all, I have read some related question about imbalance classification; however, I did not find the answer. My dataset is imbalance class that shows the distribution in below</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/a/abae99cd4e445efe0506272d63e636b56a32de80.png" data-download-href="https://discuss.pytorch.org/uploads/default/abae99cd4e445efe0506272d63e636b56a32de80" title="Screenshot%20from%202018-08-20%2010-25-03"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/a/abae99cd4e445efe0506272d63e636b56a32de80_2_690x395.png" alt="Screenshot%20from%202018-08-20%2010-25-03" data-base62-sha1="ouLJ85MWxZK8AMpQuSSzoz722go" width="690" height="395" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/a/abae99cd4e445efe0506272d63e636b56a32de80_2_690x395.png, https://discuss.pytorch.org/uploads/default/original/2X/a/abae99cd4e445efe0506272d63e636b56a32de80.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/a/abae99cd4e445efe0506272d63e636b56a32de80.png 2x" data-dominant-color="DDE9F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202018-08-20%2010-25-03</span><span class="informations">969×556 21.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I am using <code>WeightedRandomSampler</code> to handle the above problem. First, the <code>train_labels</code> is label of training set it likes <code>train_labels=[0, 2, 1, 2, 4, 2, 4, 3, 5...]</code></p>
<pre><code class="lang-auto">class_sample_counts=np.unique(train_labels, return_counts=True)[1]
weights = (1 / torch.Tensor(class_prob))
weighted_sampler = torch.utils.data.sampler.WeightedRandomSampler(weights, len(train_labels))
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, sampler=weighted_sampler)
</code></pre>
<p>The code works but the top-1 accuracy of training and validation goes 100% with Imagenet pretrain very fast, while the the testing accuracy very bad (30%). If I did not use <code>sampler</code> way, the accuracy of testing set is 70% and the accuracy of training and validation slowly grow-up. What is happen in my code? Thanks</p>

ptrblck: <p>I think your issue might be related to <a href="https://discuss.pytorch.org/t/some-problems-with-weightedrandomsampler/23242/2?u=ptrblck">this one</a>.<br>
Try to pass the weights for each sample to <code>WeightedRandomSampler</code> and see if it’s working.</p>

John1231983: <p>Thanks for your reply. I have looked at the code. Is <code>train_targets</code> same as <code>train_labels</code> in my question?</p>

ptrblck: <p>Yes, should be the same.</p>

John1231983: <p>I was successful to run it. However, it only support number of worker 1, while without sampling I can use number of worker up to 10. Do we have any way to speed up training? I feel without sampler, my code run 5 times faster</p>

sinAshish: <p>what should be set as <code>num_samples</code> in the WeightedRandomSampler.?<br>
Say that I have a 10 classes and I’ve initialized the weights as the inverse of the count of each class.<br>
I know that I can’t set it as batch_size.</p>

ptrblck: <p>The weights should be assigned to each sample, not only the inverse class count as described in the previous post.<br>
Usually the <code>num_samples</code> are then set to the length of the <code>Dataset</code>, i.e. the number of samples.</p>