Error when using DistributedDataParallel on single-GPU machine

fan_percy: <p>Hi, I’m using a 4 GPUs machine with torch.distributed for training, and I want to do the inference with the trained model on another mahcine with only one GPU. But when I run the code like this:</p>
<blockquote>
<p>python -m torch.distributed.launch --nproc_per_node=1 visualizer_distributed.py.py</p>
</blockquote>
<p>I got an error</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "visualizer_distributed.py", line 21, in &lt;module&gt;
    model = torch.nn.parallel.DistributedDataParallel(model)
  File "H:\anaconda3\lib\site-packages\torch\nn\parallel\distributed.py", line 259, in __init__
    self.process_group = _get_default_group()
NameError: name '_get_default_group' is not defined
Traceback (most recent call last):
  File "H:\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "H:\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "H:\anaconda3\lib\site-packages\torch\distributed\launch.py", line 235, in &lt;module&gt;
    main()
  File "H:\anaconda3\lib\site-packages\torch\distributed\launch.py", line 231, in main
    cmd=process.args)
subprocess.CalledProcessError: Command '['H:\\anaconda3\\python.exe', '-u', 'visualizer_distributed.py', '--local_rank=0']' returned non-zero exit status 1.
</code></pre>
<p>Here is the snippet of code</p>
<pre><code class="lang-auto">model = ...
model = torch.nn.parallel.DistributedDataParallel(model)
model.load_state_dict(torch.load(model_params))
model.cuda()
</code></pre>
<p>When I set the  device_ids like</p>
<pre><code class="lang-auto">model = ...
model = torch.nn.parallel.DistributedDataParallel(model, device_ids=0)
torch.cuda.set_device(0)
model.load_state_dict(torch.load(model_params))
model.cuda()
</code></pre>
<p>I got:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "visualizer_distributed.py", line 21, in &lt;module&gt;
    model = torch.nn.parallel.DistributedDataParallel(model, device_ids=0)
  File "H:\anaconda3\lib\site-packages\torch\nn\parallel\distributed.py", line 259, in __init__
    self.process_group = _get_default_group()
NameError: name '_get_default_group' is not defined
Traceback (most recent call last):
  File "H:\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "H:\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "H:\anaconda3\lib\site-packages\torch\distributed\launch.py", line 235, in &lt;module&gt;
    main()
  File "H:\anaconda3\lib\site-packages\torch\distributed\launch.py", line 231, in main
    cmd=process.args)
subprocess.CalledProcessError: Command '['H:\\anaconda3\\python.exe', '-u', 'visualizer_distributed.py', '--local_rank=0']' returned non-zero exit status 1.
</code></pre>
<p>Does anyone know why the problem occurs and how to use DistributedDataParallel for inference on a single-GPU machine?</p>
<p>Thanks in advance!</p>

ptrblck: <p>Based on the error message it looks like you are using a Windows machine.<br>
I’m not familiar with Windows, but I thought it doesn’t support distributed applications.<br>
Were you also using Windows on the first machine?</p>

fan_percy: <p>Yes that’s right, I’m using Windows for inference and Ubuntu for training.</p>

ptrblck: <p>In that case I think you cannot use a distributed setup. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"><br>
However, since you have a single GPU on your Windows system, you won’t get any benefits anyway. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

fan_percy: <p>So what should I do if I want to use the distributed model for inference in the single-GPU windows?</p>
<p>I was using nn.DataParallel in these two machines, I must call nn.DataParallel(model) before loading the  model. I’m now just trying to do the same thing with DistributedDataParallel but got the problem. <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>

ptrblck: <p>I think the easiest way would be to store the <code>state_dict</code> without the <code>nn.DataParallel</code> <code>.module</code> attribute (I assume you are stuck there) as described <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-torch-nn-dataparallel-models">here</a>.</p>

fan_percy: <p>I see…</p>
<p>I will try to fix it, thank you for the help!</p>