Individual transforms on subsets

Yuandi_Wu: <p>Hello all,</p>
<p>New to PyTorch and deep learning. I am running into an issue regarding applying transforms to my training and test subsets. I have a combined dataset, in which I used the scikit learn train test split to separate into my training and test sets.</p>
<p>combined_dataset = datasets.ImageFolder(“DiBAS-Images/train”, transform=None)</p>
<pre><code class="lang-auto">def train_val_split(dataset, val_split=0.25):
    train_idx, val_idx = train_test_split(list(range(len(dataset))), test_size=val_split)
    dataset_train = torch.utils.data.Subset(dataset, train_idx)
    dataset_val = torch.utils.data.Subset(dataset, val_idx)
    return dataset_train, dataset_val

dataset_train, dataset_val = train_val_split(combined_dataset)
</code></pre>
<p>I wanted to apply separate transforms to my train and test sets. For my training set:</p>
<pre><code class="lang-auto">transform = transforms.Compose([transforms.Resize((224,224)),
                                  transforms.RandomHorizontalFlip(),
                                  transforms.RandomRotation(10),
                                  transforms.RandomAffine(0, shear=10, scale=(0.8, 1.2)),
                                  transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2),
                                  transforms.ToTensor(),  
                                  transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))
                                  ])

</code></pre>
<p>and for my test set:</p>
<pre><code class="lang-auto">transform = transforms.Compose([transforms.Resize((224,224)),
                                transforms.ToTensor(),  #Mean      # Standard Deviation
                                transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))
                                ]) 
</code></pre>
<p>I am following the method outlined here:</p><aside class="quote quote-modified" data-post="5" data-topic="85416">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-apply-another-transform-to-an-existing-dataset/85416/5">How to apply another transform to an existing Dataset?</a> 
  </div>
  <blockquote>
    Transformations work with Subsets as seen here: 
# not recommended approach of setting the transform attribute late
dataset = datasets.MNIST(root='/home/pbialecki/python/data', train=True, transform=None, download=False)

print(type(dataset[0][0]))
# &gt; &lt;class 'PIL.Image.Image'&gt;

dataset = torch.utils.data.Subset(dataset, indices=torch.arange(10))
print(type(dataset[0][0]))
# &gt; &lt;class 'PIL.Image.Image'&gt;

dataset.dataset.transform = transforms.Compose([
                transforms.RandomResizedCrop…
  </blockquote>
</aside>

<pre><code class="lang-auto">dataset_train.dataset.transform = transforms.Compose([transforms.Resize((224,224)),
                                  transforms.RandomHorizontalFlip(),
                                  transforms.RandomRotation(10),
                                  transforms.RandomAffine(0, shear=10, scale=(0.8, 1.2)),
                                  transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2),
                                  transforms.ToTensor(),  
                                  transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))
                                  ])
</code></pre>
<p>However, I am not sure if this applies the transform to both subsets, as the dataset_train. dataset method accesses the combined dataset. Am I correct in this assumption, and if so, how would one apply two separate transforms to two separate subsets from the same combined dataset?</p>

Online_Profile: <p>I am stuck at the same problem. However there seems to be some solutions in - <a href="https://discuss.pytorch.org/t/torch-utils-data-dataset-random-split/32209/3">here</a></p>