ddp: trainer.test failure
Versions:
torch==1.5
pytorch-lightning==0.8.0rc1
cuda=10.1

Wit or without fp16, trainer.test(model) fails with
initializing ddp: LOCAL_RANK: 0/1 WORLD_SIZE:2
Traceback (most recent call last):
  File "finetune.py", line 791, in <module>
    main(args)
  File "finetune.py", line 738, in main
    trainer.test(model)
  File "/home/shleifer/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line
1096, in test
    self.fit(model)
  File "/home/shleifer/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line
876, in fit
    self.ddp_train(task, model)
  File "/home/shleifer/pytorch-lightning/pytorch_lightning/trainer/distrib_data_paral
lel.py", line 429, in ddp_train
    model.init_ddp_connection(self.proc_rank, self.world_size, self.is_slurm_managing
_tasks)
  File "/home/shleifer/pytorch-lightning/pytorch_lightning/core/lightning.py", line 9
60, in init_ddp_connection
    torch_distrib.init_process_group(torch_backend, rank=proc_rank, world_size=world_
size)
  File "/home/shleifer/.conda/envs/nb/lib/python3.7/site-packages/torch/distributed/$
istributed_c10d.py", line 364, in init_process_group
    raise RuntimeError("trying to initialize the default process group "
RuntimeError: trying to initialize the default process group twice!