Customizing dataloader, data transform is not applied

username: wonchulSon
cooked: <pre><code class="lang-auto"># Customizing my dataloader
class MyDataset():
    def __init__(self, cropped_1x32_dataset, targets):
        for i in range(32):
            self.__setattr__('data_{}'.format(i), cropped_1x32_dataset[i])
        self.targets = targets
        
    def __getitem__(self, index):
        for i in range(32):
            globals()['data_{}'.format(i)] = self.__getattribute__('data_{}'.format(i))[index]
        y = self.targets[index]
        return [globals()['data_{}'.format(i)] for i in range(32)], y
    
    def __len__(self):
        return len(self.data_0)

# train
my_train_dataset = MyDataset(train_cropped_1x32_dataset, train_dataset.targets)
my_train_loader = torch.utils.data.DataLoader(dataset = my_train_dataset,
                                              batch_size = batch_size,
                                              shuffle = True,
                                              num_workers=4)
</code></pre>
<pre><code class="lang-auto"># main
def train(epoch):
    model.train()
    train_loss = 0
    total = 0
    correct = 0
    
    for batch_idx, (cropped_1x32_dataset, target) in enumerate(my_train_loader):
        for i in range(32):   
            cropped_1x32_dataset[i] = cropped_1x32_dataset[i].to(device)
        target = target.to(device)

        optimizer.zero_grad()
        output = model(cropped_1x32_dataset)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()
</code></pre>
<p>In <code>main</code>, I checked the data.<br>
But it doesn’t applied any transforms both <code>RandomHorizontalFlip</code> and <code>Normalize</code>.<br>
How can I fix this?</p>
<p>Thank you.</p>

username: JuanFMontesinos
cooked: <p>Hi,<br>
If you create a custom dataloader you have to apply those transformations.</p>
<pre><code class="lang-auto"># Customizing my dataloader
class MyDataset():
    def __init__(self, cropped_1x32_dataset, targets):
        for i in range(32):
            self.__setattr__('data_{}'.format(i), cropped_1x32_dataset[i])
        self.targets = targets
        
    def __getitem__(self, index):
        for i in range(32):
            globals()['data_{}'.format(i)] = self.__getattribute__('data_{}'.format(i))[index]
        y = self.targets[index]
        return [globals()['data_{}'.format(i)] for i in range(32)], y
    
    def __len__(self):
        return len(self.data_0)
</code></pre>
<p>Look at your code, you are calling transformations nowhere.</p>

username: wonchulSon
cooked: <p><a class="mention" href="/u/juanfmontesinos">@JuanFMontesinos</a><br>
Should I call transformations in <code>__init__</code>?<br>
When I load data, I called transformations like under.<br>
Let me know where should I edit the code <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"><br>
Thank you.</p>
<pre><code class="lang-auto"># load train data
transform_train = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize(mean=(0.4914, 0.4824, 0.4467),
                             std=(0.2471, 0.2436, 0.2616))
])

train_dataset = datasets.CIFAR10(root='../../data/cifar10/',
                                 train=True,
                                 transform=transform_train,
                                 download=True)
</code></pre>

username: JuanFMontesinos
cooked: <p>If I properly understood, you replaced CIFAR10 dataset by your own by loading them from CIFAR10 dataset right?</p>
<p>The point is that you are getting the data somehow from the dataset directory rather than iterating the dataset. My guess is that CIFAR10 call function is never run, consequently, transforms are never applied.<br>
If you want to precompute transforms I would recommend to iterate over dataset instead of getting data like this <code>train_dataset.data[i]</code><br>
If you want to apply transforms on-the-fly you should apply them here</p>
<pre><code class="lang-auto">    def __getitem__(self, index):
        for i in range(32):
            globals()['data_{}'.format(i)] = self.__getattribute__('data_{}'.format(i))[index]
        y = self.targets[index] 
        return [globals()['data_{}'.format(i)] for i in range(32)], y
</code></pre>

username: wonchulSon
cooked: <p><a class="mention" href="/u/juanfmontesinos">@JuanFMontesinos</a></p>
<p>Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>