Repository: pytorch/examples

Issue:
minGPT-ddp: AttributeError: 'Trainer' object has no attribute 'dataloader'
When executing examples/distributed/minGPT-ddp/mingpt/main.py

This error is raised when trying to train minGPT.

Python version: main branch

## Possible Solution
113     def _run_epoch(self, epoch: int, dataloader: DataLoader, train: bool = True):
114         #self.dataloader.sampler.set_epoch(epoch)
115         dataloader.sampler.set_epoch(epoch)


## Steps to Reproduce
Just run main.py

## Failure Logs [if any]

Traceback (most recent call last):
  File "/mnt/tier1/project/lxp/fmansouri/pytorch/examples/distributed/minGPT-ddp/mingpt/main.py", line 41, in <module>
    main()
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra
    _run_app(
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/_internal/utils.py", line 452, in _run_app
    run_and_report(
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/_internal/utils.py", line 216, in run_and_report
    raise ex
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/_internal/utils.py", line 213, in run_and_report
    return func()
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/_internal/utils.py", line 453, in <lambda>
    lambda: hydra.run(
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/home/users/fmansouri/.local/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/mnt/tier1/project/lxp/fmansouri/pytorch/examples/distributed/minGPT-ddp/mingpt/main.py", line 35, in main
    trainer.train()
  File "/mnt/tier1/project/lxp/fmansouri/pytorch/examples/distributed/minGPT-ddp/mingpt/trainer.py", line 144, in train
    self._run_epoch(epoch, self.train_loader, train=True)
  File "/mnt/tier1/project/lxp/fmansouri/pytorch/examples/distributed/minGPT-ddp/mingpt/trainer.py", line 114, in _run_epoch
    self.dataloader.sampler.set_epoch(epoch)
AttributeError: 'Trainer' object has no attribute 'dataloader'. Did you mean: 'test_loader'?





Discussion:

Resolving patch:
diff --git a/distributed/minGPT-ddp/mingpt/trainer.py b/distributed/minGPT-ddp/mingpt/trainer.py
--- a/distributed/minGPT-ddp/mingpt/trainer.py
+++ b/distributed/minGPT-ddp/mingpt/trainer.py
@@ -111,7 +111,7 @@ def _run_batch(self, source, targets, train: bool = True) -> float:
         return loss.item()
 
     def _run_epoch(self, epoch: int, dataloader: DataLoader, train: bool = True):
-        self.dataloader.sampler.set_epoch(epoch)
+        dataloader.sampler.set_epoch(epoch)
         for iter, (source, targets) in enumerate(dataloader):
             step_type = "Train" if train else "Eval"
             source = source.to(self.local_rank)


Test patch: