Multiclass classification: Skewed data problem and labeling

banikr: <p>Hello Altruists,<br>
I am working on a multiclass classification with image data. The training set has 9015 images of 7 different classes.<br>
Target labeling looks like <code>0,1,0,0,0,0,0</code><br>
But the dataset is very much skewed to one class having 68% images and lowest amount is 1.1% belongs to another class. Please take a look at the figure below:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/7/d7f1eb1b088b3eef8df65e6d16abc55711374e49.png" alt="image" data-base62-sha1="uOkXLbukdamXnaTPs3xMcqhyvNL" width="164" height="219"></p>
<ul>
<li>So my first question is: how can I augment/stratify the dataset with all classes having nearly same percentage of samples?</li>
</ul>
<p>Moreover, for this multiclass classification how do I format the labels? For example, in MNIST the labels are 0 to 9 for different digit images.</p>
<ul>
<li>Shall I also format the class labels alike 1~7?</li>
</ul>
<p>Thanks in advance.</p>

tom: <blockquote>
<p>So my first question is: how can I augment/stratify the dataset with all classes having nearly same percentage of samples?</p>
</blockquote>
<p>The typical thing to do is keep a list-of-lists, say, <code>self.data</code> that contains for each class or stratum (first index) a list of examples. Then in your dataset’s <code>__getitem__(self, idx)</code> you use <code>cls_no = idx % len(self.data)</code> to get the class and then <code>x = self.data[cls_no][(idx // len(self.data)) % len(self.data[cls_no])]</code> to as the sample to return. You will then have to choose the epoch length appropriately, e.g. <code>len(self.data) * max([len(x) for x in self.data])</code> or so. If you have a length shorter than that you might want to shuffle the order of the examples in self.data[…] after each epoch.</p>
<blockquote>
<p>Shall I also format the class labels alike 1~7?</p>
</blockquote>
<p>No, start at 0.</p>
<p>Best regards</p>
<p>Thomas</p>

banikr: <p>Thanks.</p>
<p>What if I oversample all the minor classes to the largest class (<span class="hashtag">#6120</span>) in a list and then do the sorting/dataset from the list?<br>
This way there will be 7*6120 = 42840 image training.<br>
Does that anyhow compromise the training process?<br>
One thing I understand I need to input all the images from all classes in the network.</p>
<p>After creating the list of lists as you mentioned, the image name list has following class and samples:</p>
<p><code>0: 877, 1: 6120, 2: 470, 3: 307, 4: 962, 5: 91, 6: 118 </code></p>

tom: <aside class="quote no-group" data-username="banikr" data-post="3" data-topic="68437">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/banikr/48/54353_2.png" class="avatar"> banikr:</div>
<blockquote>
<p>Does that anyhow compromise the training process?</p>
</blockquote>
</aside>
<p>The training epoch will be ~5x longer, but that’s just loosing some time.<br>
If you want, it skews the meaning of the training loss (which you could collect per class and then apply a weighted mean if you want to compare to the validation loss). Other than that it seems to me that rebalancing the training data is a relatively straightforward path to dealing with imbalanced classification. For me, it seams to be more reliable to get good results than loss weighting.</p>
<p>Best regards</p>
<p>Thomas</p>

banikr: <p>According to this <a href="https://arxiv.org/abs/1710.05381" rel="nofollow noopener">class imbalance paper</a> the authors mention that oversampling the minority class is the best solution. So I was thinking about giving it a try.<br>
Since I have a balanced class now, do you think that training loss will be skewed, as the network will see the same minor class images more frequently than majority class ones?</p>