Why is the accuracy difference so much when I use the image data set and pytorch's own data set directly?

ma3252788: <p>For example, for the cifar10 data set, directly using the data set that comes with pytorch, the accuracy rate can reach 96% under the same network structure, but after I converted cifar10 into a picture, I tested it and the accuracy rate was only 92%. why?</p>
<p>This is the previous code：</p>
<pre><code class="lang-auto">train_dataset = dset.CIFAR10(args.data_path, train=True, transform=train_transform, download=True)
test_dataset = dset.CIFAR10(args.data_path, train=False, transform=test_transform, download=True)
</code></pre>
<p>This is the modified code：</p>
<pre><code class="lang-auto">train_dataset = datasets.ImageFolder(root='/home/ubuntu/bigdisk/DataSets/cifar10/static/orig/train/',
                                         transform=train_transform
                                         )
    test_dataset = datasets.ImageFolder(root='/home/ubuntu/bigdisk/DataSets/cifar10/static/orig/test/',
                                        transform=test_transform
                                        )
</code></pre>
<pre><code class="lang-auto">train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True,
                                               num_workers=args.prefetch, pin_memory=True)
    test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=args.test_bs, shuffle=False,
                                              num_workers=args.prefetch, pin_memory=True)
</code></pre>

ptrblck: <p>Is this behavior reproducible, i.e. are you seeing a better performance using the <code>torchvision</code> dataset if you rerun the code multiple times with a different seed?</p>
<p>If so, are you using the “exact” same dataset (same number of images are might some be missing.corrupt), the same transformation, and the same model?</p>

ma3252788: <p>Thank you for your reply! As shown in the code above, I just modified the path to load the data set from<code> ~/DATASETS/cifar.python</code> to <code>root='/home/ubuntu/bigdisk/DataSets/cifar10/static/orig/train/</code>, Others such as train_transform have not changed. Only the path has changed. Using cifar.python, it can reach more than 96% each time, while using the img path converted by cifar.python, the accuracy rate is only 92% each time.<br>
this is the img : <div class="lightbox-wrapper"><a class="lightbox" href="https://ypyssl.machunjie.com/machunjie/20200812143211.png" title="" rel="nofollow noopener"><img src="https://ypyssl.machunjie.com/machunjie/20200812143211.png" alt="" width="690" height="153"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use xlink:href="#far-image"></use></svg><span class="filename"></span><span class="informations">972×216</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use xlink:href="#discourse-expand"></use></svg></div></a></div><br>
train for 50000, test for 10000</p>

ptrblck: <p>Just to understand the use case correctly:<br>
you are not using the <code>torchvision.dataset.CIFAR10</code> data, but downloaded the <code>cifar.python</code> dataset from another source.<br>
After changing the path, the accuracy of your model is constantly lower using exactly the same setup and different seeds?</p>

ma3252788: <p>Yes, the accuracy will be reduced, even if I use exactly the same parameters. In addition, I exported these imgs from cifar.python. Normally, the results should be the same, right?</p>

ma3252788: <p>In addition, I would like to add that my data set is <code>torchvision.datasets.CIFAR10(args.data_path, train=True, transform=train_transform, download=True)</code>, cifar.python is only automatically downloaded. So, this should be the official data set in torchivision.</p>

ptrblck: <p>The <code>torchvision.dataset.CIFAR10</code> downloads binary files, while it seems your dataset has folders which contain images. Did you create these folders manually from the <code>torchvision</code> data?<br>
Anyway, this wouldn’t explain the issue you are seeing.<br>
How many times did you rerun the training and could you post a code snippet to reproduce this issue?</p>