Dataloader on two datasets

Deeply: <p>We are writing some code to read two different datasets <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html#data-loading-and-processing-tutorial" rel="nofollow noopener">based on the tutorial</a>, and thus, we will have:</p>
<pre><code class="lang-auto">train_set1, test_set1,  
train_set2, test_set2
</code></pre>
<p>We want to investigate each one separately, and both of them in a third experiment.<br>
What is the best way to do this?</p>

rasbt: <p>The best way to deal with that would be to create two <code>Dataset</code> classes if the datasets are differently structured, I’d say, and re-use a single Dataset class if the datasets are similarly structured (e.g., in the typical train &amp; test case).</p>
<p>Say I have downloaded the CelebA dataset. I would first make a text file with the file paths of the training samples and labels and a text file with the test samples and labels:</p>
<p>a) ‘celeba_gender_attr_train.txt’<br>
b) ‘celeba_gender_attr_test.txt’</p>
<p>A file would look like this:</p>
<pre><code class="lang-auto">	                ClassLabel
000001.jpg	0
000002.jpg	0
000003.jpg	1
...
</code></pre>
<p>Then I would create a Dataset class where the “info” text file is a instantiation argument, e.g.:</p>
<pre><code class="lang-python">class CelebaDataset(Dataset):
    """Custom Dataset for loading CelebA face images"""

    def __init__(self, txt_path, img_dir, transform=None):
    
        df = pd.read_csv(txt_path, sep=" ", index_col=0)
        self.img_dir = img_dir
        self.txt_path = txt_path
        self.img_names = df.index.values
        self.y = df['ClassLabel'].values
        self.transform = transform

    def __getitem__(self, index):
        img = Image.open(os.path.join(self.img_dir,
                                      self.img_names[index]))
        
        if self.transform is not None:
            img = self.transform(img)
        
        label = self.y[index]
        return img, label

    def __len__(self):
        return self.y.shape[0]
</code></pre>
<p>Then maybe adding a custom transform:</p>
<pre><code class="lang-python">custom_transform = transforms.Compose([transforms.Grayscale(),                                       
                                       transforms.ToTensor()])
</code></pre>
<p>And finally I would create 2 dataset loaders from the Dataset class:</p>
<pre><code class="lang-python">train_dataset = CelebaDataset(txt_path='celeba_gender_attr_train.txt',
                              img_dir='img_align_celeba/',
                              transform=custom_transform)

train_loader = DataLoader(dataset=test_dataset,
                          batch_size=128,
                          shuffle=True,
                          num_workers=4)
</code></pre>
<p>and</p>
<pre><code class="lang-python">test_dataset = CelebaDataset(txt_path='celeba_gender_attr_test.txt',
                              img_dir='img_align_celeba/',
                              transform=custom_transform)

test_loader = DataLoader(dataset=test_dataset,
                          batch_size=128,
                          shuffle=True,
                          num_workers=4)
</code></pre>
<p>Then during training, you could do sth like</p>
<pre><code class="lang-auto">for epoch in range(num_epochs):
    for batch_idx, features in enumerate(train_loader):
        # train model on the training dataset
for batch_idx, features in enumerate(test_loader):
       # evaluate model on test dataset
</code></pre>
<p>THe test/train split is just an example. You could do the same thing for multiple training datasets and so forth</p>
<p>E.g.,</p>
<pre><code class="lang-auto">for epoch in range(num_epochs):
    for batch_idx, features in enumerate(train_loader_1):
        # train model on the training dataset #1
    for batch_idx, features in enumerate(train_loader_2):
        # train model on the training dataset #2
for batch_idx, features in enumerate(test_loader):
       # evaluate model on test dataset
</code></pre>

Deeply: <p>Thanks for the nice and detailed explanation. I prefer to do two  <code>Dataset classes</code> , and I do like the answer (and the idea) you are suggesting. Hence, the testing should be something like:</p>
<pre><code class="lang-auto">for batch_idx, features in enumerate(test_loader_1):
       # evaluate model on test dataset #1
       # test_loss += criterion(...)
for batch_idx, features in enumerate(test_loader_2):
       # evaluate model on test dataset #2
       # test_loss += criterion(...)
</code></pre>

Naman-ntc: <p>Hey you could do this :<br>
After making two dataset classes, make a third class called fusion class that contains the two dataset classes.<br>
The __ get __ function calls randomly or alternatively get function of the two constituiting data classes.</p>
<p><a href="https://github.com/xingyizhou/pytorch-pose-hg-3d/blob/master/src/datasets/fusion.py" rel="nofollow noopener">See this link </a> for example of how you can do that. You can also create balance between the datasets images to load in that class.</p>
<p>This is probably just bit neater to do! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Deeply: <aside class="quote no-group" data-username="Naman-ntc" data-post="4" data-topic="18504">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/naman-ntc/48/32044_2.png" class="avatar"> Naman-ntc:</div>
<blockquote>
<p>s probably just bit nea</p>
</blockquote>
</aside>
<p>Very nice.<br>
I opted to do it this way, and it worked like a clock. In fact, one can fuse more than two datasets this way.</p>

nabsabs: <p>Hi <a class="mention" href="/u/naman-ntc">@Naman-ntc</a>! I would like to see your github code link but the link is dead. Can you post the code here or update the link?</p>

achaiah: <p>FYI, just to update this topic in case someone else is looking for a good answer, there is now a <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.ConcatDataset" rel="nofollow noopener">ConcatDataset</a> in Pytorch that does pretty much what the author was looking for.</p>

nabsabs: <p>hi <a class="mention" href="/u/achaiah">@achaiah</a>.  ConcatDataset will work for the OP but what if we have 2 datasets that do not have the same number of samples? I’ve tried augmenting my smaller sampled dataset and using ConcatDataset to combine the two into one dataloader but I’m not convinced it’s doing what its supposed to.   Do you have a workaround to create two dataloaders and use them for training?</p>

Deeply: <p><code>ConcatDataset</code> works well when you have 2 datasets with different number of samples.</p>

nabsabs: <p>but the unbalanced nature of the dataset created by <code>ConcatDataset</code> will not lead to balanced training, right? I know this is a dataset issue but I want to be able to oversample the underrepresented dataset and   <strong>then</strong> use <code>ConcatDataset</code> or find a better way to use two datasets.</p>

Deeply: <p>You’d better concatenate them as they are; then do sampling according to the lower size, or randomly sample, or whatever; which is way better than augmenting probably.<br>
The alternative is to use a <a href="https://pytorch.org/docs/stable/_modules/torch/nn/modules/loss.html" rel="nofollow noopener">weighted loss</a>.</p>
<p><em>If provided, the optional argument <code>weight</code> should be a 1D Tensor assigning weight to each of the classes. This is particularly useful when you have an unbalanced training set.</em></p>

nabsabs: <p>thanks <a class="mention" href="/u/deeply">@Deeply</a>! Oversampling for me is done through augmentation using the <code>transforms</code> class. Can you elaborate what you mean on “randomly sample”? Do you know of a code example of how to randomly sample the lower size dataset?</p>

Deeply: <p>Have a look at <code>torch.utils.data.</code>RandomSampler` ( <em>data_source</em> ,  <em>replacement=False</em> ,  <em>num_samples=None</em> )<br>
in <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.RandomSampler" rel="nofollow noopener">RandomSampler</a></p>

achaiah: <p><a class="mention" href="/u/nabsabs">@nabsabs</a> I’ve seen this linked somewhere on the forums before and tbh I don’t know of the quality of the code but you could try this <a href="https://github.com/ufoym/imbalanced-dataset-sampler" rel="nofollow noopener">imbalanced-dataset-sampler</a></p>

Naman-ntc: <p>Hi, Check the updated link <a href="https://github.com/xingyizhou/pytorch-pose-hg-3d/blob/master/src/lib/datasets/fusion_3d.py" rel="nofollow noopener">here</a>.<br>
Although I don’t know much about <code>ConcatDataset</code>, feel free to weigh in your options.</p>

lurenyi233: <p>Hi dear all, it seems I can’t use ConcatDataset in my problem as “train_set1” and “train_set2” have different sizes.<br>
Do you have any idea to deal with this problem?</p>

Deeply: <p>I’ve used ConcatDataset  on different dataset sizes. For example, if you’d like to concatenate training and validation datasets you can do this:</p>
<p><code>train_set = torch.utils.data.ConcatDataset( [train_set, validate_set])</code></p>
<p>You’ll need to make sure that both datasets return the same items, else, you’ll have a problem.</p>

lurenyi233: <p>Hi, Deeply, Thank you for your reply!<br>
The problem I met is that,</p>
<p>If I use DataLoader later like this:</p>
<pre><code class="lang-auto">train_set = torch.utils.data.ConcatDataset( [train_set1, train_set2])
train_loader = torch.utils.data.DataLoader(train_set, batch_size=128, shuffle=True)

for i, (features, labels) in enumerate(train_loader):
    train_features =features
    train_labels = labels
</code></pre>
<p>there will be an error:<br>
<code>" RuntimeError: stack expects each tensor to be equal size".</code><br>
As in traditional DataLoader, all the samples in the train_set need to have the same size.</p>
<p>If I modify my collate_fn function according to this post: <a href="https://discuss.pytorch.org/t/how-to-create-a-dataloader-with-variable-size-input/8278/3">How to create a dataloader with variable-size input</a>. The batch of training sample data is a large List, while The input of the CNN model can only be Tensor.<br>
So I meet another error:<br>
<code>"TypeError: conv1d(): argument 'input' (position 1) must be Tensor, not list"</code></p>
<p>Do you know how to solve this problem?</p>