Increase number of images

aleemsidra: <p>I am trying to replicate a paper where they have physically increased the data set size as stated: <em>The strategy  is  data  augmentation,  we  augment  the  ischaemia  class  by  using  8  data augmentation techniques, i.e flip horizontal and vertical, rotate-180 degree, add Gaussian noise, crop, shear, scale and adjust contrast  by  gamma  2.0,  yielded  a  total  2,043  patches  after  augmentation</em>.</p>
<p>Currently, I am using the <code>transforms.Compose</code> which augments images on the fly without physically changing the number of images. How can I actually increase the number of images as mentioned in the paper?</p>

ptrblck: <p>Since each sample will be transformed on-the-fly, you could increase the number of epochs, which should have the same effect as directly creating the augmented images and using them in a single epoch.</p>

aleemsidra: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> I want to physically increase the number of images as stated in the paper because the data set is imabalnaced (class1: 2555, class 2: 227, class 3: 621, class 4: 2552 images). If I increase the epochs, transforms will be applied equally to all of them. And the classification will be based on the majority class.</p>

ptrblck: <p>You could use different approaches:</p>
<ul>
<li>you could oversample the minority classes using a <code>WeightedRandomSampler</code> and increase the number of epochs</li>
<li>alternatively, you could duplicate indices of the minority classes using a <code>Subset</code> with the desired sample indices</li>
<li>or you could directly create copies of the minority class samples (this would use more memory, since you are copying real data)</li>
</ul>

aleemsidra: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, I am trying to use WeightedRandomSampler for handling imbalance in the dataset. However, the intuition behind it is not clear to me. My target labels are in form of one-hot encoded vectors as below.</p>
<pre><code class="lang-auto">train_labels.head(5)
</code></pre>
<div class="md-table">
<table>
<thead>
<tr>
<th></th>
<th>none</th>
<th>infection</th>
<th>ischaemia</th>
<th>both</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>1</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>1</td>
<td>1</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>2</td>
<td>0</td>
<td>1</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>3</td>
<td>0</td>
<td>1</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>4</td>
<td>0</td>
<td>1</td>
<td>0</td>
<td>0</td>
</tr>
</tbody>
</table>
</div><p>Below are the steps, I used to calculate for the weighted random sampler. Please correct me if I am wrong with the interpretation of any steps.</p>
<ol>
<li>Count the number of samples per class in the dataset</li>
</ol>
<pre><code class="lang-auto">class_sample_count = np.array(train_labels.value_counts()) 
class_sample_count
array([2555, 2552,  621,  227])
</code></pre>
<ol start="2">
<li>Calculate the weight associated with each class</li>
</ol>
<pre><code class="lang-auto">weight = 1. / class_sample_count 
weight
array([0.00039139, 0.00039185, 0.00161031, 0.00440529])
</code></pre>
<ol start="3">
<li>Calculate the weight for each of the samples in the dataset.</li>
</ol>
<pre><code class="lang-auto">samples_weight = np.array(weight[train_labels])
print(samples_weight[1], samples_weight[2] )
[0.00039185 0.00039139 0.00039139 0.00039139] #label 0 in actual data 
[0.00039139 0.00039185 0.00039139 0.00039139] #label 1 in actual data
</code></pre>
<p>The dimension of <code>samples_weight</code> comes to be [5955, 4]. 5955 are the total no. of images in the original set, and 4 corresponds to the total number of classes.<br>
Now how this mapping has been done? Since class <code>weight</code> for class 0 is 0.00039139 (obtained in step 2). How were the rest of the three entries picked up for class 0?</p>
<ol start="4">
<li>Convert the np.array to tensor</li>
</ol>
<pre><code class="lang-auto">samples_weight = torch.from_numpy(samples_weight)
samples_weight
tensor([[0.0004, 0.0004, 0.0004, 0.0004],
        [0.0004, 0.0004, 0.0004, 0.0004],
        [0.0004, 0.0004, 0.0004, 0.0004],
        ...,
        [0.0004, 0.0004, 0.0004, 0.0004],
        [0.0004, 0.0004, 0.0004, 0.0004],
        [0.0004, 0.0004, 0.0004, 0.0004]], dtype=torch.float64)
</code></pre>
<p>After conversion to tensor, all the samples appear to have the same value in all four enteries? Then how does <code>Weighted Random Sampling</code> is oversampling the minority class?</p>
<p>I will be grateful for any leads. Thank you.</p>

ptrblck: <p>One-hot encoded targets are not expected and you should transform them to class indices via <code>target = torch.argmax(target, 1)</code>. An example of using <code>WeightedRandomSampler</code> can be found <a href="https://discuss.pytorch.org/t/how-to-handle-imbalanced-classes/11264/2">here</a>.</p>

aleemsidra: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> thank you for your help and patience with me. I have followed your link.  I converted the one-hot encoded to class indices.</p>
<pre><code class="lang-auto">labels = np.argmax(train_labels.loc[:, 'none':'both'].values, axis=1)
labels = torch.from_numpy(labels)
labels
tensor([0, 0, 1,  ..., 1, 0, 0])
</code></pre>
<pre><code class="lang-auto">a = torch.unique(samples_weight)
a
tensor([0.0004, 0.0004, 0.0016, 0.0044], dtype=torch.float64)
</code></pre>
<p>Now the tensor has a unique weight for each of the target labels. Now for the intuition: the sampler picks the labels with higher weight more frequently in a batch than the samples with lower weight? So, overall in a batch, the sampling is balanced irrespective of the class distribution in the dataset?</p>
<p>I computed the<code> weighted random sampler</code> from the original dataset. After that, I split the dataset into the train and validation set. For both sets, the same sampler should be used. right? As it gives is weight is computed from overall distribution in the original set.</p>

ptrblck: <aside class="quote no-group" data-username="aleemsidra" data-post="7" data-topic="132075">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aleemsidra/48/38723_2.png" class="avatar"> aleemsidra:</div>
<blockquote>
<p>Now the tensor has a unique weight for each of the target labels.</p>
</blockquote>
</aside>
<p>Don’t forget to create the weight for each sample, not only the classes, as shown in the example.</p>
<aside class="quote no-group" data-username="aleemsidra" data-post="7" data-topic="132075">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aleemsidra/48/38723_2.png" class="avatar"> aleemsidra:</div>
<blockquote>
<p>Now for the intuition: the sampler picks the labels with higher weight more frequently in a batch than the samples with lower weight? So, overall in a batch, the sampling is balanced irrespective of the class distribution in the dataset?</p>
</blockquote>
</aside>
<p>Yes, the higher the weight the more likely this sample will be drawn. Also yes, you can balance the class distribution in a batch, but note that it’s still a random process so you shouldn’t expect to see a perfectly balanced batch in each iteration.</p>
<aside class="quote no-group" data-username="aleemsidra" data-post="7" data-topic="132075">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aleemsidra/48/38723_2.png" class="avatar"> aleemsidra:</div>
<blockquote>
<p>I computed the<code> weighted random sampler</code> from the original dataset. After that, I split the dataset into the train and validation set. For both sets, the same sampler should be used. right? As it gives is weight is computed from overall distribution in the original set.</p>
</blockquote>
</aside>
<p>It depends on your use case and I don’t know if you want to balance the validation dataset. Usually it’s used as a proxy for the final test set (i.e. unknown, “real world” samples). If your final application would also use an imbalanced class distribution, you would have to decide which metric you care about.<br>
My thinking would be to treat the validation dataset similar to the test one, so that you can indeed use the validation metric as a proxy for the test one, but let’s wait what others are thinking.</p>