Transform and Image Data Augmentation

wenyugu: <p>The documentation for torchvision.transform seems to be not clear enough. There are several questions I have.</p>
<ol>
<li>Does Compose apply each transform to every image sequentially. If order matters, what if I want to don’t want to apply transform in a composite way? (i.e. if I want to apply either flipping and then normalization or cropping followed by normalization for every image?)</li>
<li>How do I know the number of extra images gained due to augmentation? Does number of transform in Compose determine the amount of augmentation since, in my case, I want heavy augmentation as I only have 10 samples per class.</li>
</ol>

ptrblck: <ol>
<li>Yes, the order of transformations will stay the same, if you don’t use <a href="https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.RandomOrder">transforms.RandomOrder</a> or manipulate the list in another way. You could use <a href="https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.RandomApply">transforms.RandomApply</a> or <code>RandomChoice</code> it that fits your use case. Otherwise just add a condition to switch between both approaches (e.g. in your <code>Dataset</code>'s <code>__getitem__</code>).</li>
<li>Each image will be transformed randomly on-the-fly so no images will be generated and the length of your <code>Dataset</code> stays the same.</li>
</ol>

wenyugu: <p>For 2, what if I want more augmented images ?</p>

ptrblck: <p>Each iteration will transform your images, so you might just either use more epochs or artificially increase the length of your datset by setting the desired lengths in its <code>__len__</code> method and manipulate the <code>index</code> in its <code>__getitem__</code> with e.g. a modulo operation.<br>
However, both approaches would yield the same result, so I would just increase the number of epochs.</p>

Zakia: <p>Dear concern, I would like to increase the number of data by torchvision.transform.compose but instead of increasing the number of epochs I would like to go for the second option proposed by <a class="mention" href="/u/ptrblck">@ptrblck</a> in the latest reply.</p>
<p>I have changed the length of my data in <code>__len__</code>  by multiplying by 5 (now I have 5 times larger number of training data) but I do not see how to change the <code>index</code>  in its  <code>__getitem__</code>.</p>
<p>I am not sure to modify this part of the index. An example of this modification would be highly appreciated.</p>
<pre><code class="lang-auto">class PatientDataset(Dataset):
    def __init__(self, file_names, to_augment=False, transform=None, mode='train', problem_type=None):
        self.file_names = file_names
        self.to_augment = to_augment
        self.transform = transform
        self.mode = mode
        self.problem_type = problem_type

    def __len__(self):
        return 5*len(self.file_names)

    def __getitem__(self, idx):
        img_file_name = self.file_names[idx]
        image = load_image(img_file_name)
        mask = load_mask(img_file_name, self.problem_type)

        data = {"image": image, "mask": mask}
        
        augmented = self.transform(**data)
        image, mask = augmented["image"], augmented["mask"]

        if self.mode == 'train':
            if self.problem_type == 'binary':
                return img_to_tensor(image), torch.from_numpy(np.expand_dims(mask, 0)).float()
            else:
                return img_to_tensor(image), torch.from_numpy(mask).long()
        else:
            return img_to_tensor(image), str(img_file_name)
</code></pre>

ptrblck: <p>The mentioned <code>modulo</code> should work:</p>
<pre><code class="lang-python">class PatientDataset(Dataset):
    def __init__(self):
        self.data = torch.arange(10).float().view(10, 1)

    def __len__(self):
        return 5*len(self.data)

    def __getitem__(self, idx):
        idx = idx % len(self.data)
        x = self.data[idx]
        return x

dataset = PatientDataset()
loader = DataLoader(dataset, batch_size=2)

for x in loader:
    print(x)
</code></pre>

Zakia: <p>It worked. Thank you very much.</p>

Sameer_Verma: <p>From what I understand, a new transformed image will be generated in every epoch. Is it possible to use the same transformed images generated in the first epoch i.e. apply pytorch transformations just once?</p>

ptrblck: <p>You could iterate the <code>Dataset</code> containing the random transformations once and store all augmented tensors. Once this is done you could then create a <code>TensorDataset</code> which only returns the already transformed tensors.<br>
Let me know, if that would work for you.</p>

mikey_t: <p>I think this suggestion by <a class="mention" href="/u/ptrblck">@ptrblck</a> would work for me. I’m relatively new to Pytorch and not exactly sure what it would look like to iterate the dataset and store the augmented tensors. <a class="mention" href="/u/sameer_verma">@Sameer_Verma</a> would you be willing to post the code you used to do this?</p>