Module 'torch.distributed' has no attribute 'is_initialized'

zimmer550: <p>I am running inference using mmdetection (<a href="https://github.com/open-mmlab/mmdetection" rel="nofollow noopener">https://github.com/open-mmlab/mmdetection</a>) and I get the above error for this piece of code;</p>
<pre><code class="lang-auto">    model = init_detector("faster_rcnn_r50_fpn_1x.py", "faster_rcnn_r50_fpn_1x_20181010-3d1b3351.pth", device='cuda:0')
    img = 'intersection_unlabeled/frame0.jpg'
    result = inference_detector(model, img)
    show_result_pyplot(img, result, model.CLASSES)
</code></pre>
<p>And the full error log is:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "C:/Users/sarim/PycharmProjects/thesis/pytorch_learning.py", line 355, in &lt;module&gt;
    test()
  File "C:/Users/sarim/PycharmProjects/thesis/pytorch_learning.py", line 338, in test
    model = init_detector("faster_rcnn_r50_fpn_1x.py", "faster_rcnn_r50_fpn_1x_20181010-3d1b3351.pth", device='cuda:0')
  File "C:\Users\sarim\PycharmProjects\thesis\mmdetection\mmdet\apis\inference.py", line 36, in init_detector
    checkpoint = load_checkpoint(model, checkpoint)
  File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\checkpoint.py", line 188, in load_checkpoint
    load_state_dict(model, state_dict, strict, logger)
  File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\checkpoint.py", line 96, in load_state_dict
    rank, _ = get_dist_info()
  File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\utils.py", line 21, in get_dist_info
    initialized = dist.is_initialized()
AttributeError: module 'torch.distributed' has no attribute 'is_initialized'
</code></pre>
<p>My pytorch version is 1.1.0</p>

ptrblck: <p>Could you check your PyTorch version again (<code>print(torch.__version__)</code>), as <a href="https://pytorch.org/docs/1.1.0/distributed.html#torch.distributed.is_initialized" rel="nofollow noopener"><code>torch.distributed.is_initialized()</code></a> is in Pytorch 1.1.0.</p>

zimmer550: <p>Of course, I did that before. Here is proof:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/6/6586617c104db6f9015ada023d6439b5c640c563.png" data-download-href="https://discuss.pytorch.org/uploads/default/6586617c104db6f9015ada023d6439b5c640c563" title="cmd"><img src="https://discuss.pytorch.org/uploads/default/original/2X/6/6586617c104db6f9015ada023d6439b5c640c563.png" alt="cmd" data-base62-sha1="eu88jhSEsT9ySQa3iiFcJdVckSL" width="690" height="404" data-dominant-color="121213"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">cmd</span><span class="informations">1103×646 21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Anyways, I got it to work on windows by commenting out the lines that use distributed training. I read here that pytorch doesn’t support distributed training on windows but it does so on Linux:</p><aside class="onebox githubissue" data-onebox-src="https://github.com/facebookresearch/maskrcnn-benchmark/issues/257">
  <header class="source">

      <a href="https://github.com/facebookresearch/maskrcnn-benchmark/issues/257" target="_blank" rel="noopener nofollow ugc">github.com/facebookresearch/maskrcnn-benchmark</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/facebookresearch/maskrcnn-benchmark/issues/257" target="_blank" rel="noopener nofollow ugc">AttributeError: module 'torch.distributed' has no attribute 'is_initialized'</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2018-12-10" data-time="03:36:20" data-timezone="UTC">03:36AM - 10 Dec 18 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2018-12-10" data-time="03:51:22" data-timezone="UTC">03:51AM - 10 Dec 18 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/shuxp" target="_blank" rel="noopener nofollow ugc">
          <img alt="shuxp" src="https://avatars.githubusercontent.com/u/7609868?v=4" class="onebox-avatar-inline" width="20" height="20">
          shuxp
        </a>
      </div>
    </div>

    <div class="labels">
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## ❓ Questions and Help
I installed according 'conda install pytorch-nightly-cp<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">u -c pytorch' (torch-nightly1.0.0.dev20181209), also I installed pytorch stable from official site following "conda install pytorch-cpu torchvision-cpu -c pytorch", but I still found this problem.


c:\users\xp\test\maskrcnn-benchmark\maskrcnn_benchmark\utils\comm.py in is_main_process()
     26 
     27 def is_main_process():
---&gt; 28     if not torch.distributed.is_initialized():
     29         return True
     30     return torch.distributed.get_rank() == 0
AttributeError: module 'torch.distributed' has no attribute 'is_initialized'


In my win10 system, my torch is 1.0.0, and 'dir(torch.distributed)' returns

`['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'deprecated', 'is_available', 'torch']`

In my ubuntu system, I installed a GPU version,  torch version is also 1.0.0, and 'dir(torch.distributed)' returns

`['AllgatherOptions', 'AllreduceOptions', 'Backend', 'BarrierOptions', 'BroadcastOptions', 'FileStore', 'GatherOptions', 'GroupMember', 'PrefixStore', 'ProcessGroup', 'ProcessGroupGloo', 'ProcessGroupNCCL', 'ReduceOp', 'ReduceOptions', 'ScatterOptions', 'Store', 'TCPStore', 'Work', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_backend', '_dist_broadcast_coalesced', '_dist_bucket_tensors', '_queue_reduction', '_sync_params', '_sync_reduction', 'all_gather', 'all_gather_multigpu', 'all_reduce', 'all_reduce_multigpu', 'barrier', 'broadcast', 'broadcast_multigpu', 'deprecated', 'destroy_process_group', 'dist_backend', 'distributed_c10d', 'gather', 'get_backend', 'get_rank', 'get_world_size', 'group', 'init_process_group', 'irecv', 'is_available', 'is_initialized', 'is_mpi_available', 'is_nccl_available', 'isend', 'new_group', 'recv', 'reduce', 'reduce_multigpu', 'reduce_op', 'register_rendezvous_handler', 'rendezvous', 'scatter', 'send', 'string_classes', 'timedelta', 'torch', 'warnings']
`

So why this happened, who can help? Many thanks.</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>And this makes sense because I encountered no such issue when running the same code on Google Colab</p>

ptrblck: <p>Thanks for checking and good to hear you’re figured it out. My second guess would also be that you’re using a platform which does not support distributed training.</p>

sytelus: <p>I wish <code>dist.is_initialized()</code> just returned always false instead of bombing out. This way the code is more cleaner between different platforms for non-distributed use. BTW, it seems same thing happens for methods like <code>is_gloo_available()</code> etc.</p>

mrshenli: <p>There is <code>torch.distributed.is_available()</code> API to check if <code>distributed</code> package is available. APIs from <code>distributed</code> package is only available when <code>is_available</code> returns true. Let me add that to our docs.</p>

mrshenli: <p><a href="https://github.com/pytorch/pytorch/pull/37021" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/37021</a> is landed. The API doc of <code>torch.distributed.is_available</code> is added to master.</p>