New batch in each epoch

muhammedcanpirincci: <p>Hello. I am trying to use augmented and not augmented dataset in each epoch(for example: augmented in one epoch not augmented in different epoch) but i couldn’t figure out how to do it. My approach was loading DataLoader in each epoch again and again but I think it’s wrong. Because when i print indexes in <strong>getitem</strong> in my Dataset, there are a lot of duplicate indexes.<br>
Here is my code for training: <a href="https://www.codepile.net/pile/7l32AJzN" class="inline-onebox" rel="noopener nofollow ugc">CodePile | Easily Share Piles of Code</a><br>
Here is my code for the dataset: <a href="https://www.codepile.net/pile/Dgjm9V0e" class="inline-onebox" rel="noopener nofollow ugc">CodePile | Easily Share Piles of Code</a><br>
How can i achieve what i want?<br>
Thanks in advance.</p>

mMagmer: <p>hi, you’re not clear about what you want. at least i don’t get it.<br>
also re-creating dataloader can not cause duplicate index.</p>
<pre><code class="lang-auto">import torch
class Dataset1(torch.utils.data.Dataset):


    def __init__(self,lenght):
        self.idx = torch.arange(lenght)**2

  

    def __len__(self):

        return len(self.idx)

    def __getitem__(self, i):
        return i , self.idx[i]
    
    
train_sampler = torch.utils.data.SubsetRandomSampler(torch.arange(100))
Training_Data = Dataset1(100)

train_loader = torch.utils.data.DataLoader(Training_Data, batch_size=4, sampler=train_sampler,
                                           num_workers=2, pin_memory=False)


idx = []
for x,y in train_loader:
    print(x,y)
    idx.append(x)
all_index , _= torch.sort(torch.cat(idx))
print(all_index)
</code></pre>
<blockquote>
<p>output :<br>
tensor([91, 55, 82, 31]) tensor([8281, 3025, 6724,  961])<br>
tensor([71, 23, 29, 25]) tensor([5041,  529,  841,  625])<br>
tensor([70, 10, 26, 93]) tensor([4900,  100,  676, 8649])<br>
tensor([75, 96, 36, 85]) tensor([5625, 9216, 1296, 7225])<br>
tensor([48, 86, 38, 22]) tensor([2304, 7396, 1444,  484])<br>
tensor([44, 46, 98,  3]) tensor([1936, 2116, 9604,    9])<br>
tensor([12, 33, 27, 99]) tensor([ 144, 1089,  729, 9801])<br>
tensor([43, 65, 16,  2]) tensor([1849, 4225,  256,    4])<br>
tensor([63, 97, 51, 47]) tensor([3969, 9409, 2601, 2209])<br>
tensor([69, 45, 76, 56]) tensor([4761, 2025, 5776, 3136])<br>
tensor([32, 11, 68, 64]) tensor([1024,  121, 4624, 4096])<br>
tensor([52, 39, 18,  5]) tensor([2704, 1521,  324,   25])<br>
tensor([ 0, 79, 92, 35]) tensor([   0, 6241, 8464, 1225])<br>
tensor([77, 54, 21, 60]) tensor([5929, 2916,  441, 3600])<br>
tensor([81,  1, 57, 58]) tensor([6561,    1, 3249, 3364])<br>
tensor([ 7, 13, 84, 94]) tensor([  49,  169, 7056, 8836])<br>
tensor([67, 14, 80, 89]) tensor([4489,  196, 6400, 7921])<br>
tensor([83, 20, 53, 37]) tensor([6889,  400, 2809, 1369])<br>
tensor([62, 66, 15, 78]) tensor([3844, 4356,  225, 6084])<br>
tensor([59, 90, 17, 42]) tensor([3481, 8100,  289, 1764])<br>
tensor([72, 41, 95, 50]) tensor([5184, 1681, 9025, 2500])<br>
tensor([73, 28, 74, 49]) tensor([5329,  784, 5476, 2401])<br>
tensor([19, 34, 61, 30]) tensor([ 361, 1156, 3721,  900])<br>
tensor([ 8, 87,  6,  9]) tensor([  64, 7569,   36,   81])<br>
tensor([40, 88, 24,  4]) tensor([1600, 7744,  576,   16])<br>
tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,<br>
18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,<br>
36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,<br>
54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,<br>
72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89,<br>
90, 91, 92, 93, 94, 95, 96, 97, 98, 99])</p>
</blockquote>

muhammedcanpirincci: <p>Thank you so much for your answer. I just wanna understand how sampling works. In one iteration i want my program to select batch that has augmented images and in another iteration i want my program to select batch that doesnt have augmented image.</p>

muhammedcanpirincci: <p>Also i researched and i couldnt understand anything about sampling. I mean what does random sampler is doing?? What is the purpose of that? If you answer this, it would be amazing too</p>

mMagmer: <p>in your code , it’s for creating val and train set .<br>
this is one way of spliting your dataset to train and val set.<br>
if you set sampler to <code>None</code>, dataloader choose samples from all semple in your dataset.</p>
<aside class="quote no-group" data-username="muhammedcanpirincci" data-post="3" data-topic="142302">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/muhammedcanpirincci/48/46002_2.png" class="avatar"> muhammedcanpirincci:</div>
<blockquote>
<p>i want my program to select batch that doesnt have augmented image.</p>
</blockquote>
</aside>
<p>transform doesn’t  work this way by default . in most case dataset getitem applis transform to  single sample.<br>
if you want , you can return both transfomed and orginal image in getitem and use one that you want.</p>
<pre><code class="lang-auto">def __getitem__(self, i):
        img , t = ...#load image and label
        return img ,self.transform(img),t

</code></pre>

muhammedcanpirincci: <p>Thank you so much. So sampler is getting data that i want with random indexes or with sequential order.</p>

mMagmer: <p><code>torch.utils.data.SubsetRandomSampler</code> by default is random with equal probability for each sample and without replacing.</p>

muhammedcanpirincci: <p>What do you mean by “random with equal probability for each sample”?<br>
I thought if we have dataset like this:x,y,x,x,y,y,x,x,y,x,x,y,x,x,x,x,y,y,y,x,x,y,y,y,x,y<br>
We are sampling it randomly for each batch and in for loop each batch will look like this : x,x,x,y (if batch size is 4) (indexes are randomly chosen)</p>

mMagmer: <p>yes .<br>
but you can use<br>
<code>torch.utils.data.WeightedRandomSampler</code><br>
<a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.WeightedRandomSampler" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/data.html#torch.utils.data.WeightedRandomSampler</a><br>
to set different weight for each sample.</p>

muhammedcanpirincci: <p>man. You helped this guy a lot. Thank you so much.</p>