Creating Custom Dataset from inbuilt pytorch datasets along with data transformations

username: Ajinkya_Ambatwar
cooked: <p>Hi I am quite new to pytorch. I was trying to implement transfer learning with CIFAR10 and resnet18 model built in. For that what I am intending to do is first download original dataset and apply some transformations onto it and the take 500 samples from each class among the 10 classes and create a new dataset with 5000 total training samples(instead of 50,000 samples in original CIFAR10).<br>
In that case, I am just extracting and arranging the indices of the samples of the set as per their class and then randomly picking 500 of them for each class. But with that method I am not able to achieve any performance from the network even if I am doing data augmentations on the original set.<br>
SO  is there a way to truly get the 500/per class data(using some custom dataloader) and then apply data augmentation on that to achieve the performance?<br>
Thanks in advance!!</p>

username: Oli
cooked: <p>Hey, I wrote this dataset for you that gets a subset of CIFAR10 <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:">. Just set the n_images_per_class to 500 and it should be ready to use!</p>
<pre><code class="lang-auto">from torchvision import datasets
from collections import defaultdict, deque
import itertools


class Cifar5000(datasets.CIFAR10):
    def __init__(self, path, transforms, train=True):
        super().__init__(path, train, download=True)
        self.transforms = transforms
        self.n_images_per_class = 5
        self.n_classes = 10
        self.new2old_indices = self.create_idx_mapping()

    def create_idx_mapping(self):
        label2idx = defaultdict(lambda: deque(maxlen=self.n_images_per_class))
        for original_idx in range(super().__len__()):
            _, label = super().__getitem__(original_idx)
            label2idx[label].append(original_idx)

        old_idxs = set(itertools.chain(*label2idx.values()))
        new2old_indices = {}
        for new_idx, old_idx in enumerate(old_idxs):
            new2old_indices[new_idx] = old_idx

        return new2old_indices

    def __len__(self):
        return len(self.new2old_indices)

    def __getitem__(self, index):
        index = self.new2old_indices[index]
        im, label = super().__getitem__(index)
        return self.transforms(im), label
</code></pre>

username: Ajinkya_Ambatwar
cooked: <p>Thanks <a class="mention" href="/u/oli">@Oli</a> will definitely check on this one!!<img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=9" title=":grinning:" class="emoji" alt=":grinning:"></p>

username: Ajinkya_Ambatwar
cooked: <p><a class="mention" href="/u/oli">@Oli</a> With your given method I created the dataset. But when I checked</p>
<pre><code class="lang-python">len(trainset.data) 
</code></pre>
<p>it’s still 50,000. On the other hand when I checked the trainloader details it showed number of data points to be 5,000? Can you please explain what’s happening here?<br>
Thanks!!<img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

username: Oli
cooked: <p>What is trainset? If you do len(Cifar5000_dataset) it should give 5000. That’s why we implemented the<br>
method</p>
<pre><code class="lang-auto">def __len__(self):
        return len(self.new2old_indices)
</code></pre>

username: Ajinkya_Ambatwar
cooked: <p>So after creating the class I created the object called trainset as</p>
<pre><code class="lang-auto">trainset = Cifar500(path = "./data",transforms = transform)
</code></pre>
<p>Then I am checking the number of samples as</p>
<pre><code class="lang-auto">len(trainset.data)
</code></pre>
<p>which returns 50,000(not 5,000).<br>
I created the trainloader with the trainset as</p>
<pre><code class="lang-auto">trainloader = torch.utils.data.DataLoader(trainset, batch_size =32 ,shuffle=True, num_workers=2)
</code></pre>
<p>And</p>
<pre><code class="lang-auto">trainloader.dataset
</code></pre>
<p>prints</p>
<pre><code class="lang-bash">Dataset Cifar500
    Number of datapoints: 5000
    Root location: ./data
    Split: Train
    Compose(
    ToTensor()
    Normalize(mean=(0.4914, 0.4822, 0.4465), std=(0.2023, 0.1994, 0.201))
)
</code></pre>
<p>(5000 datapoints)<br>
I am not able to understand which actually is being fed to the network 5000 datapoints or 50,000 samples.</p>

username: Oli
cooked: <aside class="quote no-group" data-username="Ajinkya_Ambatwar" data-post="6" data-topic="58270">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ajinkya_ambatwar/48/11828_2.png" class="avatar"> Ajinkya_Ambatwar:</div>
<blockquote>
<p>len(trainset.data)</p>
</blockquote>
</aside>
<p>If you change this to len(trainset) it should work <img src="https://discuss.pytorch.org/images/emoji/apple/speedboat.png?v=12" title=":speedboat:" class="emoji" alt=":speedboat:" loading="lazy" width="20" height="20"><br>
That’s the one which is being used</p>

username: Ajinkya_Ambatwar
cooked: <p><a class="mention" href="/u/oli">@Oli</a>  I checked. Yes it is 5,000. Thanks for your precious help:pray:<img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"><br>
By the way, with this implementation does the trainloader picks random 500 samples during each epoch or it is sampled only once at start?</p>

username: Oli
cooked: <p><a class="mention" href="/u/ajinkya_ambatwar">@Ajinkya_Ambatwar</a> You’re welcome <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>It samples it only once at start, but it can be different each time you run your python file. If you want it to be the same every time you run your python script, you can use this seed function at the very beginning of your program</p>
<pre><code class="lang-auto">def seed_program(seed=0):
  ''' Seed for reproducability '''
  random.seed(seed)
  np.random.seed(seed)
  torch.manual_seed(seed)
  torch.cuda.manual_seed_all(seed)
  # torch.backends.cudnn.deterministic = True # You can add this
</code></pre>

username: Ajinkya_Ambatwar
cooked: <p><a class="mention" href="/u/oli">@Oli</a> Yes I am aware of that. Thanks for all the clarification!!<br>
Also I had one another unrelated doubt. How can we add dropout to inbuilt pytorch models like say resnet18?I know that if we are custom building the network from scratch then we can add dropout layers using nn.dropout. But with built in models how can we do that?</p>

username: Oli
cooked: <p>I don’t know a way other than building a custom network. But you don’t have to build it from scratch, the resnet18 code is on github so you can copy that and change the source code <img src="https://discuss.pytorch.org/images/emoji/apple/sailboat.png?v=9" title=":sailboat:" class="emoji" alt=":sailboat:"></p>

username: Ajinkya_Ambatwar
cooked: <p>Oh thanks…That should help!!<img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=9" title=":grinning:" class="emoji" alt=":grinning:"></p>