Sending a bulk data by RPC

ph0123: <p>Hi all,</p>
<p>I am using RPC to send and receive data with multiple CPUs.<br>
Currently, I use “gloo” backend.<br>
My code is:</p>
<pre><code class="lang-auto">rpc.rpc_async(my_target, add_outlinks, args=(arr_send[i],source))
</code></pre>
<p>I have nearly 1 million objects.<br>
For each object, I send to (size-1) workers to run the function add_outlinks.</p>
<p>Totally, with 1 million objects, we have to send 1*(size-1) times.<br>
arr_send[i] is from 10 to 1 000 000 numbers.</p>
<p>I run the same function on UPC++, It takes around several minutes. However, with torch-RPC, I set timeout to 10000s ~ 166 minutes, and the program is topped by timeout.</p>
<p>Could you tell me the best way to send data with Pytorch?</p>
<p>Thanks,</p>

mrshenli: <p>Is it possible to consolidate multiple <code>arr_send[i]</code> into one RPC? Is there any reason that they need to be sent using dedicated RPCs?</p>
<p>Curious:</p>
<ol>
<li>After 10000s, how many objects are processed?</li>
<li>Is distributed autograd/optimizer also used in this case?</li>
<li>Are any of those CPUs locate on the same machine? (so that shm can be helpful)</li>
</ol>

ph0123: <p>I use as</p>
<pre><code class="lang-auto">def add_outlinks(arr, source):
    for dest in arr:
        if int(dest) in _local_dict:
            _local_dict[dest].in_links.append(int(source))
</code></pre>
<pre><code class="lang-auto">    rpc.init_rpc(my_name, rank=rank, world_size=size,rpc_backend_options=rpc.ProcessGroupRpcBackendOptions(num_send_recv_threads=16,rpc_timeout=datetime.timedelta(seconds=10000)))  # initial_rpc

    #CALL rpc TO OTHER RANKS
    if rank==0:
        print("add-link...")
    try:
        array_rpc = list(range(0, size))
        count=0
        for it in _local_dict:
            count = count+1
            arr_send = []
            for i in range(0, size):
                arr_send.append([])
            u = _local_dict[it]
            source = u.vertexId

            for i in u.links:
                arr_send[int(i) % size].append(int(i))
            for i in array_rpc:
                my_target = "worker" + str(i)
                if len(arr_send[i])&gt;0:
                    rpc.rpc_async(my_target, add_outlinks, args=(arr_send[i],source))
    except:
	   print("rank ",rank," run ",count,"/",len(_local_dict))
    rpc.api._wait_all_workers()
    print("shutdown.... rpc... ", rank)
    rpc.api._wait_all_workers()
    rpc.shutdown()

</code></pre>
<ul>
<li>arr_send[i] will send to rank i<br>
For elements in _local_dict, we can run parallel.</li>
</ul>
<p><strong>1.     After 10000s, how many objects are processed?</strong></p>
<p>–&gt; the outputs are as below. The worker0 is not see in the output. I try to print “count” value. But there is no output for “count” variable.</p>
<pre><code class="lang-auto">....
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:45462
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:44970
....

[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:19635
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:27553
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:44970
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:47501: Connection reset by peer
Traceback (most recent call last):
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
  File "pagerank.py", line 380, in init_process
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:44931: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 77, in wrapper
    return func(*args, **kwargs)
...
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:378] writev [2001:700:4a01:10::38]:22942: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 240, in shutdown
    _wait_all_workers()
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 77, in wrapper
    return func(*args, **kwargs)
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:5848: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:50095: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:29331
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:57022: Connection reset by peer
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 165, in _wait_all_workers
    args=(sequence_id, self_worker_name,),
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:15236
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:2720
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:23214
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:38547
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:50607
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 77, in wrapper
    return func(*args, **kwargs)
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
...
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:12173
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 554, in rpc_sync
    return fut.wait()
RuntimeError: Encountered exception in ProcessGroupAgent::enqueueSend: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:57022: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:3715
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:2693
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure

During handling of the above exception, another exception occurred:

[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:9877
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:3715
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
Traceback (most recent call last):
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:378] writev [2001:700:4a01:10::38]:22942: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:36780: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:47501: Connection reset by peer
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:44931: Connection reset by peer

[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:3213: Connection reset by peer
Traceback (most recent call last):
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:13723
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "pagerank.py", line 380, in init_process
    print("shutdown.... rpc... ", rank)
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:9140
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 77, in wrapper
    return func(*args, **kwargs)
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 240, in shutdown
    _wait_all_workers()
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 77, in wrapper
    return func(*args, **kwargs)
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 165, in _wait_all_workers
    args=(sequence_id, self_worker_name,),
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 77, in wrapper
    return func(*args, **kwargs)
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/site-packages/torch/distributed/rpc/api.py", line 554, in rpc_sync
    return fut.wait()

RuntimeError: Encountered exception in ProcessGroupAgent::enqueueSend: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:566] Read error [2001:700:4a01:10::38]:9889: Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
  File "/cluster/home/cnphuong/.conda/envs/Pytorch_ENV/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
KeyboardInterrupt
[E thread_pool.cpp:112] Exception in thread pool task: Application timeout caused pair closure
[E thread_pool.cpp:112] Exception in thread pool task: [/opt/conda/conda-bld/pytorch_1587428228634/work/third_party/gloo/gloo/transport/tcp/pair.cc:575] Connection closed by peer [2001:700:4a01:10::38]:34716



</code></pre>
<p>I changed the code to print count as</p>
<pre><code class="lang-auto">for it in _local_dict:
     if(rank==0:)
            count = count+1
           print(count)

</code></pre>
<p>–&gt; all elements in _local_dict is run. However, the program stopped by timeout.</p>
<p><strong>2.     Is distributed autograd/optimizer also used in this case?</strong></p>
<p>–&gt; Not yet. It is at <a href="https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html</a> ??</p>
<p><strong>3.     Are any of those CPUs locate on the same machine? (so that shm can be helpful)</strong><br>
–&gt;  I use 32 CPUs on this machine. (All of my CPU on the machine.)</p>
<p>Thanks</p>

ph0123: <p>I try with small data (~60 elements in _local_dict), and it worked.<br>
Perhaps, there are problems with a lot of rpc call?</p>
<p>For real data, size of _local_dict is nearly 190000 elements - 32 workers. With each element, I called 32 times for rpc(…).</p>
<p>For each worker, we have to call 190000<em>32 times for rpc(…). We have 32 workers. Hence, totally, there are 190000</em>32*32 times for calling RPC.</p>
<p>Is there any problem with that?</p>
<p>Thanks,</p>

ph0123: <p>Hi,<br>
I delete RPC and try to run with the local</p>
<pre><code class="lang-auto">_local_dict
</code></pre>
<p>each worker holds 1/32 of total objects. Then, I run the program. It takes nearly 5s.<br>
However, It can not finish with RPC (timeout=10000s).</p>
<p>Perhaps, the problem has belonged to the Queue size of RPC backend.</p>
<p>Please help!<br>
Thanks,</p>

ph0123: <p>Hi,<br>
I add future variable for the rpc.rpc_async.<br>
It worked. However, it is very slow.<br>
It took around 30 minutes to finish this procedure.</p>
<aside class="quote no-group quote-modified" data-username="ph0123" data-post="3" data-topic="84938">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ph0123/48/22888_2.png" class="avatar"> ph0123:</div>
<blockquote>
<pre><code class="lang-auto"> for it in _local_dict:
            count = count+1
            arr_send = []
            for i in range(0, size):
                arr_send.append([])
            u = _local_dict[it]
            source = u.vertexId

            for i in u.links:
                arr_send[int(i) % size].append(int(i))
            futs = [] #add here
            for i in array_rpc:
                my_target = "worker" + str(i)
                if len(arr_send[i])&gt;0:
                    futs.append(rpc.rpc_async(my_target, add_outlinks, args=(arr_send[i],source)))
            for fut in futs: #add here
                fut.wait() #add here
</code></pre>
</blockquote>
</aside>
<p>I try to put</p>
<pre><code class="lang-auto"> for fut in futs: #add here
                fut.wait() #add here
</code></pre>
<p>outside of the loop. The program is not finished and stopped by time out.</p>
<p>Could you explain why? and could you please suggest some ways to decrease the excuse time? I run the program with UPC++, it take some minutes to finish (~ 5-6 minutes)</p>
<p>Thanks</p>

mrshenli: <p>Sorry about the delay.</p>
<p><code>rpc.api._wait_all_workers()</code> is not supposed to be directly called by applications. I was only referencing it as one example to use rank 0 as a coordinator. The recommended way is doing sth like below:</p>
<pre data-code-wrap="python"><code class="lang-python">futs = []
for i in array_rpc:
    futs.append(rpc.rpc_async(my_target, add_outlinks, args=(arr_send[i],source)))

...

for fut in futs:
    fut.wait()
</code></pre>
<p>Allow me some time, still reading the contents above.</p>
<h2><a name="edits-1" class="anchor" href="#edits-1"></a>Edits</h2>
<p>I see you are already using future in the latest version.</p>
<blockquote>
<p>Could you explain why? and could you please suggest some ways to decrease the excuse time? I run the program with UPC++, it take some minutes to finish (~ 5-6 minutes)</p>
</blockquote>
<p>Given that the comp only takes 5s, it looks like pickle and comm takes the majority of the time? Curious, can you profile the time taken for one <code>rpc_sync</code>?</p>
<p>The communication pattern above looks like an allToAll. Is it guaranteed that the data sent from rank x to rank y will be of the same size. If so, you can use the <a href="https://pytorch.org/docs/master/distributed.html#torch.distributed.all_to_all" rel="noopener nofollow ugc"><code>allToAll API</code></a> (not in v1.5, only available on master now, will come to v1.6).</p>
<p>
Regarding ways to speed up, one easy attempt would be increasing the number of threads. The program currently sets <code>num_send_recv_threads=16</code>. You could try 64. This could help speeding up deserialization on the server side. But the serializer on the sender side still occurs inline, we probably should also offload that to thread-pool in future releases.
</p>
<p>I take that back. I think one major reason is that the pickle and execution both require Python GIL, and hence they won’t run in parallel on the server side. Can you try using <code>TorchScript</code> functions? You can find examples <a href="https://pytorch.org/docs/stable/rpc.html" rel="noopener nofollow ugc">here</a> by searching for <code>@torch.jit.script</code>. Even if we use <code>@torch.jit.script</code> functions, the pickle/unpickle would still require GIL as those are Python objects, but the execution can run in parallel.</p>

mrshenli: <aside class="quote no-group" data-username="ph0123" data-post="3" data-topic="84938">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ph0123/48/22888_2.png" class="avatar"> ph0123:</div>
<blockquote>
<p>the outputs are as below. The worker0 is not see in the output. I try to print “count” value. But there is no output for “count” variable.</p>
</blockquote>
</aside>
<p>It means the program didn’t git the <code>except</code> branch below:</p>
<pre data-code-wrap="python"><code class="lang-python">    except:
	   print("rank ",rank," run ",count,"/",len(_local_dict))
</code></pre>
<p>If there are exceptions occur when processing the RPC in the try block, it will be thrown locally when you call <code>fut.wait()</code> or <code>rref.to_here()</code>.</p>
<p>Given the above code, the timeout won’t occur in the try block, as <code>rpc_async</code> always returns immediately with a future object.</p>

mrshenli: <aside class="quote no-group" data-username="ph0123" data-post="3" data-topic="84938">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ph0123/48/22888_2.png" class="avatar"> ph0123:</div>
<blockquote>
<p>I use 32 CPUs on this machine. (All of my CPU on the machine.)</p>
</blockquote>
</aside>
<p>In this case, the new TensorPipe (coming in v1.6) RPC backend might be useful, as it provides a shm channel to send RPC data. Will keep you posted on that.</p>

ph0123: <p>Thank you for your responses.</p>
<p>I will wait the 1.6 version and try to work with rpc.ref.<br>
I will compare and share with you about the results.</p>
<p>Thanks,</p>

mrshenli: <p>Hey <a class="mention" href="/u/ph0123">@ph0123</a></p>
<p>Thanks. Regarding the <code>TorchScript</code> functions, double checked with the team, it’s only the serialization on the caller side is holding GIL as of today. The derserialization on the callee can run in parallel. Curious to see how much <code>TorchScript</code> functions can help.</p>

ph0123: <p>Hi,</p>
<p>I try torch.jit.script as</p>
<pre><code class="lang-auto">@torch.jit.script
def add_outlinks(arr, source):
    for dest in arr:
        if _local_dict.get(dest):
            _local_dict[dest].in_links.append(int(source))
</code></pre>
<p>On my program, I defined that _loca_dict is a global variable. Therefore, there is no error here. When I add “<span class="mention">@torch.jit.script</span>”. The terminal shown the error as</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/mnt/c/python_project/Pytorch/test.py", line 222, in &lt;module&gt;
    @torch.jit.script
  File "/home/cnphuong/.local/lib/python3.6/site-packages/torch/jit/__init__.py", line 1290, in script
    fn = torch._C._jit_script_compile(qualified_name, ast, _rcb, get_default_args(obj))
RuntimeError: 
attribute lookup is not defined on python value of type 'dict':
  File "/mnt/c/python_project/Pytorch/test.py", line 225
def add_outlinks(arr, source):
    for dest in arr:
        if _local_dict.get(dest):
           ~~~~~~~~~~~~~~~ &lt;--- HERE
            _local_dict[dest].in_links.append(int(source))
</code></pre>
<p>I have to install pytorch from source or not?<br>
My pytorch version is 1.5 and cpu only.</p>
<p>Thanks,</p>

mrshenli: <p>That probably means <code>TorchScript</code> does not support 
<code>dict.get</code>
 global variable yet.</p>
<p>cc JIT expert <a class="mention" href="/u/michael_suo">@Michael_Suo</a></p>
<p>Let me try this locally.</p>

mrshenli: <p>Just check with JIT team, <code>TorchScript</code> functions does not support global variables. One alternative is to do it through <code>RRef.local_value()</code>. Let me prepare an example for you.</p>

ph0123: <p>thank you so much!<br>
Best regards,</p>

mrshenli: <p>Hi <a class="mention" href="/u/ph0123">@ph0123</a></p>
<p>I tried this <a href="https://github.com/pytorch/pytorch/pull/39900" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/39900</a></p>
<p>But it actually exposes another gap in <code>TorchScript</code> function type annotation. JIT team is investigating.</p>

mrshenli: <p>Fixes are coming to master:</p>
<aside class="onebox githubpullrequest">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/pull/39932" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Pull Request">
    <svg width="60" height="60" class="github-icon" viewBox="0 0 12 16" aria-hidden="true"><path d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/pull/39932" target="_blank" rel="nofollow noopener">[rpc] use annotation_str for RRef type serialization</a>
    </h4>

    <div class="branches">
      <code>pytorch:gh/wanchaol/111/base</code> ← <code>pytorch:gh/wanchaol/111/head</code>
    </div>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-06-12" data-time="06:49:30" data-timezone="UTC">06:49AM - 12 Jun 20 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/wanchaol" target="_blank" rel="nofollow noopener">
          <img alt="wanchaol" src="https://avatars2.githubusercontent.com/u/9443650?v=4" class="onebox-avatar-inline" width="20" height="20">
          wanchaol
        </a>
      </div>

      <div class="lines" title="1 commits changed 2 files with 3 additions and 3 deletions">
        <a href="https://github.com/pytorch/pytorch/pull/39932/files" target="_blank" rel="nofollow noopener">
          <span class="added">+3</span>
          <span class="removed">-3</span>
        </a>
      </div>
    </div>

  </div>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
<aside class="onebox githubpullrequest">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/pull/39933" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Pull Request">
    <svg width="60" height="60" class="github-icon" viewBox="0 0 12 16" aria-hidden="true"><path d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/pull/39933" target="_blank" rel="nofollow noopener">[rpc] fix RRef alias annotation</a>
    </h4>

    <div class="branches">
      <code>pytorch:gh/wanchaol/112/base</code> ← <code>pytorch:gh/wanchaol/112/head</code>
    </div>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-06-12" data-time="06:49:39" data-timezone="UTC">06:49AM - 12 Jun 20 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/wanchaol" target="_blank" rel="nofollow noopener">
          <img alt="wanchaol" src="https://avatars2.githubusercontent.com/u/9443650?v=4" class="onebox-avatar-inline" width="20" height="20">
          wanchaol
        </a>
      </div>

      <div class="lines" title="1 commits changed 2 files with 24 additions and 3 deletions">
        <a href="https://github.com/pytorch/pytorch/pull/39933/files" target="_blank" rel="nofollow noopener">
          <span class="added">+24</span>
          <span class="removed">-3</span>
        </a>
      </div>
    </div>

  </div>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>