Define a dataset with transforms, then splitting for validation

Wesley_Neill: <p>I’m currently loading up some data in the following way. <code>MNIST</code> is a custom dataset that looks pretty much identical to the one in the official tutorial, so nothing special there. <code>to_dtype</code> is a custom transform that does exactly what you would expect, and is also formatted after the official tutorial.</p>
<pre><code class="lang-auto">transform = transforms.Compose([transforms.ToPILImage(),
                                transforms.RandomRotation(10, fill=(0,)),
                                transforms.RandomHorizontalFlip(),
                                transforms.RandomPerspective(),
                                transforms.RandomAffine(10),
                                transforms.ToTensor(),
                                to_dtype(),
                                transforms.Normalize((0.5,), (0.5,))])

trainset = MNIST('data/train.csv', transform=transform)

N = len(trainset)
split = (N - int(np.floor(N*.2)), int(np.floor(N*.2)))
trainset, validset = torch.utils.data.random_split(trainset, split)

trainload = DataLoader(trainset, batch_size=32, shuffle=True, num_workers=4)
validload = DataLoader(validset, batch_size=32, shuffle=True, num_workers=4)
</code></pre>
<p>apparently <code>.random_split()</code> does not return two datasets of the same type as was passed. I therefore cannot access the <code>transform</code> attribute to turn it off in the validation set.</p>
<p>well, I can with</p>
<pre><code class="lang-auto">validset.dataset.transform=None
</code></pre>
<p>but that turns off transforms for the training set too.</p>
<p>Should I be worried about this? Is having augmented validation data bad?</p>

ptrblck: <p>Instead of using <code>random_split</code>, you could create two datasets, one training dataset with the random transformations, and another validation set with its corresponding transformations.<br>
Once you have created both datasets, you could randomly split the data indices e.g. using <code>sklearn.model_selection.train_test_split</code>. These indices can then be passed to <code>torch.utils.data.Subset</code> together with their datasets in order to create the final training and validation dataset.</p>

Wesley_Neill: <p>Okay, that sounds like a good solution! And since you are suggesting it, I assume the answer to my question is that augmenting the validation set is a bad idea?</p>
<p>Actually, <code>train_test_split</code> works really well with how I defined my dataset class! I can just do this, and it ends up solving my augmentation problem, and I think is a little cleaner:</p>
<pre><code class="lang-auto">train_df = pd.read_csv('data/train.csv')
X, y = train_df.iloc[:, 1:], train_df.iloc[:, 0]
x_train, x_valid, y_train, y_valid = model_selection.train_test_split(X, y)

trainset = MNIST(pd.concat([x_train, y_train], axis=1), transform=transform)
validset = MNIST(pd.concat([x_valid, y_valid], axis=1), transform=transforms.ToTensor())

trainload = DataLoader(trainset, batch_size=32, shuffle=True, num_workers=4)
validload = DataLoader(validset, batch_size=32, shuffle=True, num_workers=4)
</code></pre>

ptrblck: <aside class="quote no-group" data-username="Wesley_Neill" data-post="3" data-topic="87218">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wesley_neill/48/25709_2.png" class="avatar"> Wesley_Neill:</div>
<blockquote>
<p>I assume the answer to my question is that augmenting the validation set is a bad idea?</p>
</blockquote>
</aside>
<p>I would say it’s not the usual approach and don’t see a lot of reasons to augment the validation set (but there might be valid use cases I’m not aware of).</p>