Repository: pytorch/ignite

Issue:
Patch MNIST dataset downloading to fix CI
Currently, there is an issue with downloading MNIST dataset using torchvision. Let's introduce the following patch to our CI to fix it:
- https://github.com/pytorch/vision/issues/3500#issuecomment-790491487

Where to put that:
- create new step with the patch before [here](https://github.com/pytorch/ignite/blob/700f0e1325efc5dc0dce88d26284e51bc2a7c87c/.github/workflows/unit-tests.yml#L106)
- add patch [here](https://github.com/pytorch/ignite/blob/700f0e1325efc5dc0dce88d26284e51bc2a7c87c/.circleci/config.yml#L147).


Discussion:
@vfdev-5 Oh, I saw this issue today on Stack Overflow)

`wget` is all you need)

We can just patch download step in `MNIST`
Here is a python patch:

```python
import os
import subprocess as sp
from torchvision.datasets.mnist import MNIST, read_image_file, read_label_file
from torchvision.datasets.utils import extract_archive


def patched_download(self):
    """wget patched download method.
    """
    if self._check_exists():
        return

    os.makedirs(self.raw_folder, exist_ok=True)
    os.makedirs(self.processed_folder, exist_ok=True)

    # download files
    for url, md5 in self.resources:
        filename = url.rpartition('/')[2]
        download_root = os.path.expanduser(self.raw_folder)
        extract_root = None
        remove_finished = False

        if extract_root is None:
            extract_root = download_root
        if not filename:
            filename = os.path.basename(url)
        
        # Use wget to download archives
        sp.run(["wget", url, "-P", download_root])

        archive = os.path.join(download_root, filename)
        print("Extracting {} to {}".format(archive, extract_root))
        extract_archive(archive, extract_root, remove_finished)

    # process and save as torch files
    print('Processing...')

    training_set = (
        read_image_file(os.path.join(self.raw_folder, 'train-images-idx3-ubyte')),
        read_label_file(os.path.join(self.raw_folder, 'train-labels-idx1-ubyte'))
    )
    test_set = (
        read_image_file(os.path.join(self.raw_folder, 't10k-images-idx3-ubyte')),
        read_label_file(os.path.join(self.raw_folder, 't10k-labels-idx1-ubyte'))
    )
    with open(os.path.join(self.processed_folder, self.training_file), 'wb') as f:
        torch.save(training_set, f)
    with open(os.path.join(self.processed_folder, self.test_file), 'wb') as f:
        torch.save(test_set, f)

    print('Done!')


MNIST.download = patched_download
```
Looks awesome ! Could you please send as patch for us ? :)
Sure!)
I think MNIST is still not working now:
- https://github.com/pytorch/ignite/pull/1762/checks?check_run_id=2087318076#step:13:1

Reopen to fix it
Seems like that https://github.com/pytorch/vision/issues/3549
Now we have `503 Service Unavailable`
Maybe, it worth to host it somewhere in a sure place for our CI...
@vfdev-5 Yeah, Like pjreddie did:
https://pjreddie.com/projects/mnist-in-csv/

But he converted it to csv
We can host in the github releases which allow up to 2GB
Or maybe like that : https://github.com/facebookresearch/fairscale/blob/0491715f8be1f40923d920f1f15861d245ed3cb4/benchmarks/datasets/mnist.py#L15-L31 
I have an idea, what if we do the same as blefaudeux did, but instead of just static hosting, we transform our mnist repo into the action called `download-mnist` since `unit-tests` generate a lot of requests, is it okay for GitHub? What do you think?

Here is a dummy action that stores text file, and copies it during running in a workflow:

https://github.com/trsvchn/storage-action

And here is the usage:

https://github.com/trsvchn/gha-sandbox/runs/2088425278?check_suite_focus=true#step:6:4

As you can see we downloaded file using github machinery
I like this idea !
Let's host it on pytorch-ignite org.
Great!

Resolving patch:
diff --git a/examples/mnist/mnist_patch.py b/examples/mnist/mnist_patch.py
deleted file mode 100644
--- a/examples/mnist/mnist_patch.py
+++ /dev/null
@@ -1,69 +0,0 @@
-"""Patch to fix MNIST download issue as described here:
-- https://github.com/pytorch/ignite/issues/1737
-- https://github.com/pytorch/vision/issues/3500
-"""
-
-import os
-import subprocess as sp
-
-import torch
-from torchvision.datasets.mnist import MNIST, read_image_file, read_label_file
-from torchvision.datasets.utils import extract_archive
-
-
-def patched_download(self):
-    """wget patched download method.
-    """
-    if self._check_exists():
-        return
-
-    os.makedirs(self.raw_folder, exist_ok=True)
-    os.makedirs(self.processed_folder, exist_ok=True)
-
-    # download files
-    for url, md5 in self.resources:
-        filename = url.rpartition("/")[2]
-        download_root = os.path.expanduser(self.raw_folder)
-        extract_root = None
-        remove_finished = False
-
-        if extract_root is None:
-            extract_root = download_root
-        if not filename:
-            filename = os.path.basename(url)
-
-        # Use wget to download archives
-        sp.run(["wget", url, "-P", download_root])
-
-        archive = os.path.join(download_root, filename)
-        print("Extracting {} to {}".format(archive, extract_root))
-        extract_archive(archive, extract_root, remove_finished)
-
-    # process and save as torch files
-    print("Processing...")
-
-    training_set = (
-        read_image_file(os.path.join(self.raw_folder, "train-images-idx3-ubyte")),
-        read_label_file(os.path.join(self.raw_folder, "train-labels-idx1-ubyte")),
-    )
-    test_set = (
-        read_image_file(os.path.join(self.raw_folder, "t10k-images-idx3-ubyte")),
-        read_label_file(os.path.join(self.raw_folder, "t10k-labels-idx1-ubyte")),
-    )
-    with open(os.path.join(self.processed_folder, self.training_file), "wb") as f:
-        torch.save(training_set, f)
-    with open(os.path.join(self.processed_folder, self.test_file), "wb") as f:
-        torch.save(test_set, f)
-
-    print("Done!")
-
-
-def main():
-    # Patch download method
-    MNIST.download = patched_download
-    # Download MNIST
-    MNIST(".", download=True)
-
-
-if __name__ == "__main__":
-    main()


Test patch: