`init_process_group` not called when training on multiple-GPUs
Hi,
I’m trying to train a model on 2 GPUs. I do this by specifying Trainer(..., gpus=2). ddp_spawn should automatically be selected for the method, but I instead get the following message + error:
UserWarning: You requested multiple GPUs but did not specify a backend, e.g. `Trainer(
accelerator="dp"|"ddp"|"ddp2")`. Setting `accelerator="ddp_spawn"` for you.
  'You requested multiple GPUs but did not specify a backend, e.g.'
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
Traceback (most recent call last):
 File "train.py", line 186, in <module>
    main(sys.argv[1:])
  File "train.py", line 173, in main
    print(f"Logs for this experiment are being saved to {trainer.log_dir}")
  File ".../pypi__pytorch_lightning_python3_deps/pytorch_lightning/trainer/properties.py", line 137, in log_dir
    dirpath = self.accelerator.broadcast(dirpath)
  File ".../pypi__pytorch_lightning_python3_deps/pytorch_lightning/accelerators/accelerator.py", line 436, in broadcast
    return self.training_type_plugin.broadcast(obj, src)
  File ".../pypi__pytorch_lightning_python3_deps/pytorch_lightning/plugins/training_type/ddp_spawn.py", line 275, in broadcast
    return self.dist.broadcast(obj)
  File ".../pypi__pytorch_lightning_python3_deps/pytorch_lightning/distributed/dist.py", line 33, in broadcast
    broadcast_object_list(obj, 0, group=group or _group.WORLD)
  File ".../pypi__torch_python3_deps/torch/distributed/distributed_c10d.py", line 1700, in broadcast_object_list
    my_rank = get_rank()
  File ".../pypi__torch_python3_deps/torch/distributed/distributed_c10d.py", line 725, in get_rank
    default_pg = _get_default_group()
  File ".../pypi__torch_python3_deps/torch/distributed/distributed_c10d.py", line 358, in _get_default_group
    raise RuntimeError("Default process group has not been initialized, "
RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

I looked at the source code of ddp_spawn and it looks like it should print out a message when initializing ddp, but it didn’t.
Could I please have advice on how to correct this error.
Thank you!
The issue comes from the line
  File "train.py", line 173, in main
    print(f"Logs for this experiment are being saved to {trainer.log_dir}")
which tries to access trainer.log_dir outside of the trainer scope.
trainer.log_dir tries to broadcast the directory but fails as DDP hasn’t been initialized yet.
  File ".../pypi__pytorch_lightning_python3_deps/pytorch_lightning/trainer/properties.py", line 137, in log_dir
    dirpath = self.accelerator.broadcast(dirpath)
This is fixed in the 1.4 release as broadcast becomes a no-op in that case