Using the Trainer class more than once fails with "Address already in use" with the DDP backend
🐛 Bug
It is not possible to create and use the Trainer class more than once with the DDP backend since the program crashes the second time with RuntimeError: Address already in use.
To Reproduce
Here is a minimal code example which reproduces the issue:
import torch
from pytorch_lightning import Trainer
from pytorch_lightning.core.lightning import LightningModule
from torch.nn import functional as F
from torch.utils.data import DataLoader
from torch.utils.data import Dataset


class MyDataset(Dataset):
    def __getitem__(self, index: int):
        return torch.rand(10), torch.rand(10).type(torch.int64)

    def __len__(self) -> int:
        return 10


class LitModel(LightningModule):
    def __init__(self):
        super().__init__()
        self.l1 = torch.nn.Linear(10, 10)

    def forward(self, x):
        return torch.relu(self.l1(x.view(x.size(0), -1)))

    def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.mse_loss(y, y_hat)
        tensorboard_logs = {'train_loss': loss}
        return {'loss': loss, 'log': tensorboard_logs}

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=0.001)

    def train_dataloader(self):
        dataset = MyDataset()
        loader = DataLoader(dataset, batch_size=2, shuffle=True)
        return loader


def train():
    model = LitModel()

    trainer = Trainer(gpus=2, distributed_backend="ddp", max_epochs=2)
    trainer.fit(model)


train()
train()  # This second call fails
Expected behavior
It should be possible to use the Trainer object multiple times. In my case, it breaks my k-fold validation loop since I create a new Trainer for each fold.
Environment
* CUDA:
        - GPU:
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
        - available:         True
        - version:           10.2
* Packages:
        - numpy:             1.18.5
        - pyTorch_debug:     False
        - pyTorch_version:   1.5.0
        - pytorch-lightning: 0.8.4
        - tensorboard:       2.2.2
        - tqdm:              4.46.1
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                - ELF
        - processor:         x86_64
        - python:            3.8.3
        - version:           #109-Ubuntu SMP Fri Jun 19 11:33:10 UTC 2020