Making additional data with Data Augmentation using pytorch

username: amy2
cooked: <p>I am trying to understand how the data augmentation works in pytorch, so I started with the exemple in the official documentation the faces exemple from my understanding the augmentation in pytorch does not increase the number of samples (does not crete additional ones) but at every epoch it makes random alterations to the existing ones. know if I want to use data augmentation to make additional data. what modifications should I do this is the exemple of the documentation</p>
<blockquote>
<p>import os<br>
import torch<br>
import pandas as pd<br>
from skimage import io, transform<br>
import numpy as np<br>
import matplotlib.pyplot as plt<br>
from torch.utils.data import Dataset, DataLoader<br>
from torchvision import transforms, utils</p>
</blockquote>
<pre><code># Ignore warnings
import warnings
warnings.filterwarnings("ignore")

plt.ion()   # interactive mode


 class FaceLandmarksDataset(Dataset):
 """Face Landmarks dataset."""

  def __init__(self, csv_file, root_dir, transform=None):
"""
   Arguments:
    csv_file (string): Path to the csv file with annotations.
    root_dir (string): Directory with all the images.
    transform (callable, optional): Optional transform to be applied
        on a sample.
     """
  self.landmarks_frame = pd.read_csv(csv_file)
  self.root_dir = root_dir
  self.transform = transform

 def __len__(self):
   return len(self.landmarks_frame)

def __getitem__(self, idx):
  if torch.is_tensor(idx):
    idx = idx.tolist()

img_name = os.path.join(self.root_dir,
                        self.landmarks_frame.iloc[idx, 0])
image = io.imread(img_name)
landmarks = self.landmarks_frame.iloc[idx, 1:]
landmarks = np.array([landmarks], dtype=float).reshape(-1, 2)
sample = {'image': image, 'landmarks': landmarks}

if self.transform:
    sample = self.transform(sample)

return sample
# class Rescale (object):
     #....
#class RandomCrop (object):
   #  ....
#class Totensor (object):
    # ....

transformed_dataset = FaceLandmarksDataset(csv_file='data/faces/face_landmarks.csv',
                                   root_dir='data/faces/',
                                   transform=transforms.Compose([
                                       Rescale(256),
                                       RandomCrop(224),
                                       ToTensor()
                                   ]))

for i, sample in enumerate(transformed_dataset):
  print(i, sample['image'].size(), sample['landmarks'].size())

  if i == 3:
break

 dataloader = DataLoader(transformed_dataset, batch_size=4,
                 shuffle=True, num_workers=
</code></pre>

username: ptrblck
cooked: <p>You can save the augmented data in each iteration. However, why would you want to do it?</p>

username: amy2
cooked: <p>I actually want to applied on the dataset I am working with .But first I wanted to understand how it works on a simple exemple.<br>
the thing is that I have a small dataset and after the training I got stuck with 0.9 training accuracy but 0.6 validation accuracy .<br>
I saw someone posting about it saying that data augmentation can be done by adding additional samples not only making alterations on existing ones<br>
I wated to try to add aditionnal data perhaps the results gets better<br>
But didn’t know how<br>
I saw something about adding another loop in the get_item function is that correct <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
like this discussion I want to have for every image multiple augmented images and pass them to the network</p><aside class="quote quote-modified" data-post="1" data-topic="139720">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/flock1/48/12220_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/understand-data-augmentation-in-pytorch/139720">Understand data augmentation in PyTorch</a> <a class="badge-category__wrapper " href="/c/data/37"><span data-category-id="37" data-drop-close="true" class="badge-category " title="Topics related to DataLoader, Dataset, torch.utils.data, pytorch/data, and TorchArrow."><span class="badge-category__name">data</span></span></a>
  </div>
  <blockquote>
    Hi, 
I was wondering if I could get a better understanding of data Augmentation in PyTorch. From what I know, data augmentation is used to increase the number of data points when we are running low on them. So we use transforms to transform our data points into different types. I am suing data transformation like this: 
transform_img = transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        # transforms.Nor…
  </blockquote>
</aside>


username: ptrblck
cooked: <p>It makes sense to add data augmentation, but I would not try to save the augmented samples.<br>
The standard workflow would be to apply the random transformations inside the <code>Dataset.__getitem__</code> method on each loaded (original) sample. This would create a new, augmented sample in each batch and you could still keep the original dataset small.</p>

username: amy2
cooked: <p>yes that’s what I want to apply the random transformations inside the <strong>getitem</strong> function is that any documentation or exemple that I can understand from how it should be done <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: ptrblck
cooked: <p>You could check <a href="https://pytorch.org/tutorials/beginner/basics/data_tutorial.html#creating-a-custom-dataset-for-your-files">this tutorial</a>, which shows how <code>transforms</code> are passed to the <code>Dataset</code> and used in the <code>__getitem__</code>. This tutorial also shows how a custom <code>Dataset</code> is created. In case you are using any built-in <code>Dataset</code> it should most likely provide a <code>transform</code> argument in its <code>__init__</code>.</p>

username: amy2
cooked: <p>thank you very much for your help!</p>

username: amy2
cooked: <p>just one other question in the getitem function it calls transforms and after that you define the transformations exemple and call them in the dataloader<br>
is there a possibility to call the transformation exemple functions in the getitem directly ?<br>
<a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: ptrblck
cooked: <p>I’m not sure how to understand the question, so let me try to break down the logic.<br>
The <code>__getitem__</code> method first loads the <code>img_path</code> for the current <code>index</code> and loads the image via <code>read_image</code>. The corresponding <code>label</code> is loaded afterwards.<br>
Next we check if <code>self.transform</code> is defined (default is <code>None</code>) and if so, we apply it on the <code>image</code>. Now <code>image</code> is transformed using the <code>self.transform</code> transformations, which are defined as <code>ToTensor()</code> in this example, but can contain a other (random) transformations, too.<br>
The original <code>image</code> is now gone since the augmented tensor replaced <code>image</code>.<br>
The same applies to the <code>label</code> using <code>self.target_transform</code>.<br>
Finally, the <code>image</code> and <code>label</code> tensor are returned.</p>
<p>I’m thus currently unsure what</p>
<aside class="quote no-group" data-username="amy2" data-post="8" data-topic="190560">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/amy2/48/62675_2.png" class="avatar"> amy2:</div>
<blockquote>
<p>is there a possibility to call the transformation exemple functions in the getitem directly ?</p>
</blockquote>
</aside>
<p>means.</p>

username: J_Johnson
cooked: <p>Sounds like your model is suffering from overfitting. Image augmentation is one approach to address it. Also, experimenting with a higher <code>p</code> in your dropout layers is another technique that can address overfitting and can be used in parallel to augmentation.</p>
<p>Saving augmented images for reuse would have the effect of increasing overfitting. Instead, you might try experimenting with increasing the number and level of transforms.</p>
<p>Just keep in mind that if your model is overfitting at 90% train and 60% val, the actual model performance might cap out in the 70-80% range. One naive approach for data scientists starting out is thinking that the 90% value is realistic and that they only need to bring up the validation to that level. But the reality is that 90% is likely unrealistic and an average of the train and val accuracy is a more realistic expectation for where your model performance caps out at, given that dataset.</p>

username: amy2
cooked: <p>I saw someone suggesred creating other loops for the training and validation  inside the getitem method and apply the transformation function</p>

username: Lucas_Eduardo
cooked: <p>Augmentation is basically a processing that is done with one dataset item right after it’s fetched.</p>
<p>The detail is that is must be done with both the image and the ground truth.</p>
<p>How to do it? Use a ready made library for that. For compute vision I recommend using albumentations.</p>

username: JimS
cooked: <p>When doing augmentations it is important you consider what augmentations make sense. Not all do.</p>
<p>I can’t tell for sure from your code but it looks like perhaps you are processing faces? That will limit the usable augmentations you can do. Mirroring a face top-to-bottom might not make sense but mirroring it left to right might. Offsetting horizontally and vertically might make sense. Small rotations might make sense but large ones not.</p>
<p>In my work, the images have no real  “up” or “down” or “left” and “right” so we do rotations, left-right and top-bottom inversions, and random horizontal and vertical offsets. We create the augmentations and save them before training. We embed the augmentations in the image’s name so we know how the image was created and and augment image can always be tracked back to the original one. This allows us to do checks such as if all the augmented images classify in the same groups as the parent image.</p>
<p>I also require that all augmentations of an image must end up in the same set: Train, Test, or Validate.</p>

username: amy2
cooked: <p>yes I understand that but my problem is not with the augmentation<br>
it’s with the size of the dataset it’s very small and I want to add addional sample to augment the size of the dataset not only making alterations to the existing ones<br>
But th implementation in pytorch doesn’t do that so I was wondering about the modification that I should make to this script to make the aditional samples <a class="mention" href="/u/jims">@JimS</a><br>
in another post someone suggested to use multiple training loop inside the --init–function and in the training concatenate them and used them for the training</p>