Can dist.TCPStore store NamedTuple?

JuyiLin: <p>I want to share some NamedTuple (such as mytuple) between different rank.  The following codes is possible?</p>
<pre><code class="lang-python">    tcpstore = dist.TCPStore(MASTER_ADDR,  MASTER_PORT, world_size,
                             MASTER_ADDR == LOCAL_ADDR)
    dist.init_process_group('nccl', store= tcpstore, rank=rank, world_size=world_size)
    if rank == 0:
        store.set("my1", mytuple)
    else:
        id = store.get("my1",mytuple)
</code></pre>
<p>I have read the<br>
<a href="https://discuss.pytorch.org/t/how-to-store-embeddings-from-different-ranks-in-distributeddataparallel-mode/94405/4" class="inline-onebox">How to store embeddings from different ranks in DistributedDataParallel mode? - #4 by mrshenli</a> . But I want to know if I have   8GPU, how could I init and pass 8 simplequeue?</p>

wanchaol: <p><a class="mention" href="/u/juyilin">@JuyiLin</a> could you share more about your motivation? dist.Store is only intended to be used by process group init, it’s not exposing to public arbitrary usage, it might work out of box for some cases, but it’s not guaranteed.</p>
<p>Specifically if you want to share tuple of tensors, you can dist.broadcast each tensor to each rank</p>

JuyiLin: <p>Thank you for your time! I have tried to use dist.scatter_object_list, but it failed. Could you have a look?</p><aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/88685">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/88685" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/88685" target="_blank" rel="noopener nofollow ugc">distributed.scatter_object_list shows RuntimeError: Tensors must be CUDA and dense</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2022-11-08" data-time="16:58:19" data-timezone="UTC">04:58PM - 08 Nov 22 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/LukeLIN-web" target="_blank" rel="noopener nofollow ugc">
          <img alt="LukeLIN-web" src="https://avatars.githubusercontent.com/u/60426396?v=4" class="onebox-avatar-inline" width="20" height="20">
          LukeLIN-web
        </a>
      </div>
    </div>

    <div class="labels">
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">### 📚 The doc issue

I tried  https://pytorch.org/docs/stable/distributed.html#t<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">orch.distributed.scatter_object_list codes 
```python
def run(rank, world_size, data,  dataset):
    os.environ['MASTER_ADDR'] = 'localhost'
    os.environ['MASTER_PORT'] = '12355'
    dist.init_process_group('nccl',  rank=rank, world_size=world_size)

    torch.cuda.set_device(rank)

    if rank == 0 :
        objects = ["foo", 12, {1: 2}] # any picklable object
    else:
        objects = [None, None, None]
    objects =objects
    outputlist = [None]
    dist.scatter_object_list(outputlist, objects, src=0)
    print(outputlist)


if __name__ == '__main__':
    dataset = Reddit('/data/Reddit')
    world_size = torch.cuda.device_count()
    data = dataset[0]

    print('Let\'s use', world_size, 'GPUs!')
    data_split = (data.train_mask, data.val_mask, data.test_mask)
    mp.spawn(
        run,
        args=(world_size, data, dataset),
        nprocs=world_size,
        join=True
    )
```

But it shows:
Traceback (most recent call last):
  File "smallcase_reddit_quiver_send.py", line 99, in &lt;module&gt;

  File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
    while not context.join():
  File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 150, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
    fn(i, *args)
  File "/root/share/gnnproject/quiver/smallcase_reddit_quiver_send.py", line 84, in run
    dist.scatter_object_list(outputlist, objects, src=0)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py", line 1919, in scatter_object_list
    broadcast(max_tensor_size, src=src, group=group)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py", line 1159, in broadcast
    work = default_pg.broadcast([tensor], opts)
RuntimeError: Tensors must be CUDA and dense

### Suggest a potential alternative/fix

_No response_</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>