Repository: pytorch/ignite

Issue:
Datasets are downloading only on node 0, rank 0 in Ignite examples.
## 🐛 Bug description
Datasets are downloading only on node 0, rank 0 in Ignite examples. `idist.get_rank()` should be `idist.get_local_rank()`

https://github.com/pytorch/ignite/blob/13a42b90589112e67bcc6e1780e7c826a5a34199/examples/contrib/cifar10/main.py#L204-L214

---

https://github.com/pytorch/ignite/blob/13a42b90589112e67bcc6e1780e7c826a5a34199/examples/contrib/cifar10_qat/main.py#L189-L199

---

https://github.com/pytorch/ignite/blob/13a42b90589112e67bcc6e1780e7c826a5a34199/examples/contrib/transformers/main.py#L207-L219

cc @vfdev-5 for confirmation


Discussion:
@ydcjeff If you are not going to work on this. I can prepare a PR. the approach looks straightforward to me
> @ydcjeff If you are not going to work on this. I can prepare a PR. the approach looks straightforward to me

@Moh-Yakoub Sure! you can go ahead

Resolving patch:
diff --git a/examples/contrib/cifar10/main.py b/examples/contrib/cifar10/main.py
--- a/examples/contrib/cifar10/main.py
+++ b/examples/contrib/cifar10/main.py
@@ -203,14 +203,15 @@ def run(
 
 def get_dataflow(config):
     # - Get train/test datasets
-    if idist.get_rank() > 0:
-        # Ensure that only rank 0 download the dataset
+    if idist.get_local_rank() > 0:
+        # Ensure that only local rank 0 download the dataset
+        # Thus each node will download a copy of the dataset
         idist.barrier()
 
     train_dataset, test_dataset = utils.get_train_test_datasets(config["data_path"])
 
-    if idist.get_rank() == 0:
-        # Ensure that only rank 0 download the dataset
+    if idist.get_local_rank() == 0:
+        # Ensure that only local rank 0 download the dataset
         idist.barrier()
 
     # Setup data loader also adapted to distributed config: nccl, gloo, xla-tpu
diff --git a/examples/contrib/cifar10_qat/main.py b/examples/contrib/cifar10_qat/main.py
--- a/examples/contrib/cifar10_qat/main.py
+++ b/examples/contrib/cifar10_qat/main.py
@@ -188,14 +188,15 @@ def run(
 
 def get_dataflow(config):
     # - Get train/test datasets
-    if idist.get_rank() > 0:
-        # Ensure that only rank 0 download the dataset
+    if idist.get_local_rank() > 0:
+        # Ensure that only local rank 0 download the dataset
+        # Thus each node will download a copy of the dataset
         idist.barrier()
 
     train_dataset, test_dataset = utils.get_train_test_datasets(config["data_path"])
 
-    if idist.get_rank() == 0:
-        # Ensure that only rank 0 download the dataset
+    if idist.get_local_rank() == 0:
+        # Ensure that only local rank 0 download the dataset
         idist.barrier()
 
     # Setup data loader also adapted to distributed config: nccl, gloo, xla-tpu
diff --git a/examples/contrib/transformers/main.py b/examples/contrib/transformers/main.py
--- a/examples/contrib/transformers/main.py
+++ b/examples/contrib/transformers/main.py
@@ -206,16 +206,17 @@ def run(
 
 def get_dataflow(config):
     # - Get train/test datasets
-    if idist.get_rank() > 0:
-        # Ensure that only rank 0 download the dataset
+    if idist.get_local_rank() > 0:
+        # Ensure that only local rank 0 download the dataset
+        # Thus each node will download a copy of the dataset
         idist.barrier()
 
     train_dataset, test_dataset = utils.get_dataset(
         config["data_dir"], config["model"], config["tokenizer_dir"], config["max_length"]
     )
 
-    if idist.get_rank() == 0:
-        # Ensure that only rank 0 download the dataset
+    if idist.get_local_rank() == 0:
+        # Ensure that only local rank 0 download the dataset
         idist.barrier()
 
     # Setup data loader also adapted to distributed config: nccl, gloo, xla-tpu


Test patch: