Large performance difference when using torchvision EMNIST dataset vs custom dataset

Danibahnani: <p>Hi! I am training a classification model on the EMNIST dataset. I noticed big differences in the convergence of the loss when using the EMNIST dataset from torchvision (<code>torchvision.datasets.EMNIST</code>), instead of loading from CSV. I tested using the same parameters for the model and with the same random seed.</p>
<p>I constructed a custom dataset class to fetch data from csv files:</p>
<pre><code class="lang-auto">def load_csv_data(csv_path:str):
 #function to load data from csv, used to create img_data &amp; label_data
    data_df = pd.read_csv(csv_path)
    labels = (data_df.iloc[:,0]).values #first column, labels are uppercase &amp; lowecase, e.g "a" or "A" = 1, shift range from 1-26 to 0-25
    raw_pixels = data_df.iloc[:,1:].values #pixel values , transpose to get correct orientation

    imgs = raw_pixels.reshape(-1,28,28) #emnist images are 28*28 pixels - 3d ndarray
    return imgs, labels

class EMNISTDataset(Dataset):
    def __init__(self, img_data:ndarray, label_data:ndarray, transform, target_transform, subset_indices:slice,):
        super().__init__()
        self.img_subset = img_data[subset_indices]
        self.label_subset = label_data[subset_indices]

        if self.img_subset.shape[0] != self.label_subset.shape[0]:
              raise RuntimeError(f"Img subset len does not match label subset len: \n Img subset len: {self.img_subset.shape[0]} \n Label subset len:{self.label_subset.shape[0]}")
        self.transform = transform
        self.target_transform = target_transform
  
     def __len__(self):
        data_len = self.label_subset.shape[0]
        return data_len
     
     def __getitem__(self, idx):
        X = self.img_subset[idx,:,:]
        y = int(self.label_subset[idx])
        if self.transform:
            X = self.transform(X) #applies torchvision transform
        if self.target_transform:
              y = self.target_transform(y) #applies torchvision transform
          return X, y
</code></pre>
<p>I am using these transforms:</p>
<pre><code class="lang-auto">img_transform = v2.Compose([
    v2.Resize(28),
    v2.Compose([v2.ToImage(), v2.ToDtype(torch.float32, scale=True)]), # to tensor
    v2.RandomHorizontalFlip(p=1),#100% probability
    v2.RandomRotation(degrees=(90,90)), #flip 90 degrees
    ])
target_transform = lambda y:y-1
</code></pre>
<p>This is the Crossentropyloss from multiple runs, the worse ones (around 3.2) are from my custom dataset, while the better ones are from the torchvision dataset:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/7/076f34633715992d85d1daa29148b90fc201a7f6.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/076f34633715992d85d1daa29148b90fc201a7f6" title="Crossentropyloss"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/7/076f34633715992d85d1daa29148b90fc201a7f6_2_690x362.jpeg" alt="Crossentropyloss" data-base62-sha1="13LB1nyq7FuS3go8LHb2zsprhEa" width="690" height="362" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/7/076f34633715992d85d1daa29148b90fc201a7f6_2_690x362.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/0/7/076f34633715992d85d1daa29148b90fc201a7f6_2_1035x543.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/7/076f34633715992d85d1daa29148b90fc201a7f6_2_1380x724.jpeg 2x" data-dominant-color="FAF7F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Crossentropyloss</span><span class="informations">1920×1009 70.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Compare a batch created by the <code>torchvision</code> dataset with one created by your custom dataset w.r.t. its stats (e.g. <code>mean</code>, <code>min</code>, <code>max</code>, <code>std</code>) to see if the processing differs.</p>

Danibahnani: <p>I compared the batches and found that there were big differences, e.g. max: 0.33 vs 3e-18. I fixed it by converting the image arrays to PIL images.</p>