MNIST Dataloader Problem

akgulozlem: <p>Hi there <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>For some time, there have been issues about downloading MNIST dataset with torchvision. I have found a solution to this from<br>
<a href="https://stackoverflow.com/questions/66646604/http-error-503-service-unavailable-whan-trying-to-download-mnist-data" rel="noopener nofollow ugc">https://stackoverflow.com/questions/66646604/http-error-503-service-unavailable-whan-trying-to-download-mnist-data</a>.</p>
<p>After running this code and doing some adjustments in the folders of Google Colab, I tried  to load data to a dataloader by giving its path and parameters as explained in<br>
<em><a href="https://www.kaggle.com/beastlyprime/pytorch-beginner-cnn-for-mnist" class="inline-onebox" rel="noopener nofollow ugc">PyTorch beginner: CNN for MNIST | Kaggle</a></em></p>
<p>The code I am working on is below:</p>
<p>def get_train_loader(dataset):<br>
“”"<br>
Get train dataloader of source domain or target domain<br>
:return: dataloader<br>
“”"<br>
if dataset == ‘MNIST’:<br>
transform = transforms.Compose([<br>
transforms.Lambda(lambda x: x.repeat(3,1,1)),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean= dataset_mean, std= dataset_std)<br>
])<br>
data = datasets.MNIST(root= mnist_path + ‘/processed’, train= True, transform= transform, download=False ) <em><span class="hashtag">#This</span> line gives the problem</em><br>
dataloader = DataLoader(dataset= data, batch_size= batch_size, shuffle= True, drop_last= True)</p>
<p><strong>mnist_path</strong> is a string variable that shows the root of the MNIST file<br>
However when I run this, I get a runtime error:<br>
<em>RuntimeError: Dataset not found. You can use download=True to download it.</em></p>
<p>But this is not helping due to the issue I explained in the beginning. I am working on Google Colab, and I am working with MNIST and MNIST-M.  I have uploaded MNIST-M to <strong>Google Colab</strong> and unzipped it. MNIST is also there, it has two subfolders <strong>processed</strong> and <strong>raw</strong>. <strong>Processed</strong> folder also contains <strong>training.pt</strong> and <strong>test.pt</strong>. Can you help me solve this problem? Thank you so much :))</p>

ptrblck: <p>The folder structure sounds correct so are you sure the path is right?<br>
Also, what kind of issue were you seeing when downloading the MNIST dataset?<br>
In case you were hitting a server error, could you try to update <code>torchvision</code> to the nightly and rerun it, as another download mirror was added?</p>

Roj: <p>Unsure if you’ve solved this or not, but I ran into similar file i/o issues in Colab. The cleanest way I found for loading MNIST is to instead simply load it via keras, and then package it into a <code>DataLoader</code>. See the example <a href="https://discuss.pytorch.org/t/working-code-breaks-when-run-on-google-colab/120108/3">here</a>. (Caveat: Colab vomits when running the model, and as of this writing I still don’t know why).</p>

Novin_Nouri: <p>you can download dataset images from this <a href="https://www.kaggle.com/datasets/novinnouri/mnist-dataset/data" rel="noopener nofollow ugc">link</a></p>