Repository: pytorch
Issue: When I ran the 'Get Started with Data Data Parallel' tutorial the code got 'RuntimeError '!There are no hints
Labels: triaged,module: data parallel

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->



## My code:
```
import torch
import torch.distributed as dist
import torch.nn as nn
import torch.optim as optim
import torch.multiprocessing as mp
from torch.nn.parallel import DistributedDataParallel as DDP
def setup(rank, world_size):
    init_method = "file:///D:/share"
    dist.init_process_group(
       "gloo",
       rank=rank,
       init_method=init_method,
       world_size=world_size)
    # initialize the process group
    dist.init_process_group("gloo", rank=rank, world_size=world_size)
def cleanup():
    dist.destroy_process_group()
class ToyModel(nn.Module):
    def __init__(self):
        super(ToyModel, self).__init__()
        self.net1 = nn.Linear(10, 10)
        self.relu = nn.ReLU()
        self.net2 = nn.Linear(10, 5)
    def forward(self, x):
        return self.net2(self.relu(self.net1(x)))
def demo_basic(rank, world_size):
    print(f"Running basic DDP example on rank {rank}.")
    setup(rank, world_size)
    # create model and move it to GPU with id rank
    model = ToyModel().to(rank)
    ddp_model = DDP(model, device_ids=[rank])
    loss_fn = nn.MSELoss()
    optimizer = optim.SGD(ddp_model.parameters(), lr=0.001)
    optimizer.zero_grad()
    outputs = ddp_model(torch.randn(20, 10))
    labels = torch.randn(20, 5).to(rank)
    loss_fn(outputs, labels).backward()
    optimizer.step()
    cleanup()
def run_demo(demo_fn, world_size):
    mp.spawn(demo_fn,
             args=(world_size,),
             nprocs=world_size,
             join=True)
if __name__ == "__main__":
    n_gpus = torch.cuda.device_count()
    run_demo(demo_basic, 1)
```



## The Error:

Running basic DDP example on rank 0.
Traceback (most recent call last):
  File "D:/Project/Transformer/reformer-pytorch/examples/enwik8_simple/torch_demo.py", line 71, in <module>
    run_demo(demo_basic, 1)
  File "D:/Project/Transformer/reformer-pytorch/examples/enwik8_simple/torch_demo.py", line 61, in run_demo
    mp.spawn(demo_fn,
  File "D:\Anaconda3\lib\site-packages\torch\multiprocessing\spawn.py", line 230, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "D:\Anaconda3\lib\site-packages\torch\multiprocessing\spawn.py", line 188, in start_processes
    while not context.join():
  File "D:\Anaconda3\lib\site-packages\torch\multiprocessing\spawn.py", line 150, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException: 

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "D:\Anaconda3\lib\site-packages\torch\multiprocessing\spawn.py", line 59, in _wrap
    fn(i, *args)
  File "D:\Project\Transformer\reformer-pytorch\examples\enwik8_simple\torch_demo.py", line 42, in demo_basic
    setup(rank, world_size)
  File "D:\Project\Transformer\reformer-pytorch\examples\enwik8_simple\torch_demo.py", line 15, in setup
    dist.init_process_group(
  File "D:\Anaconda3\lib\site-packages\torch\distributed\distributed_c10d.py", line 503, in init_process_group
    _update_default_pg(_new_process_group_helper(
  File "D:\Anaconda3\lib\site-packages\torch\distributed\distributed_c10d.py", line 588, in _new_process_group_helper
    pg = ProcessGroupGloo(
RuntimeError

## Environment

PyTorch version: 1.8.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.18.3

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce GTX 1050 Ti
Nvidia driver version: 461.72
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\cudnn64_7.dll
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] numpydoc==1.1.0
[pip3] torch==1.8.0
[pip3] torchaudio==0.8.0
[pip3] torchvision==0.9.0
[conda] blas                      1.0                         mkl    defaults
[conda] cudatoolkit               10.2.89              h74a9793_1    defaults
[conda] mkl                       2020.2                      256    defaults
[conda] mkl-service               2.3.0            py38h196d8e1_0    defaults
[conda] mkl_fft                   1.3.0            py38h46781fe_0    defaults
[conda] mkl_random                1.1.1            py38h47e9c7a_0    defaults
[conda] numpy                     1.19.2           py38hadc3359_0    defaults
[conda] numpy-base                1.19.2           py38ha3acd2a_0    defaults
[conda] numpydoc                  1.1.0              pyhd3eb1b0_1    defaults
[conda] pytorch                   1.8.0           py3.8_cuda10.2_cudnn7_0    pytorch
[conda] torchaudio                0.8.0                      py38    pytorch
[conda] torchvision               0.9.0                py38_cu102    pytorch