Convert torch tensors directly to cupy tensors?

username: tushar
cooked: <p>I know jumping through the conversion hoops with <code>cupy.array(torch_tensor.cpu().numpy())</code> is one option, but since the tensor is already in gpu memory, is there any equivalent to a .cupy() to directly get it into cupy?</p>
<p>Thanks</p>

username: smth
cooked: <p>yes, there’s an easy way. See this example:<br>
<aside class="onebox githubgist">
  <header class="source">
      <a href="https://gist.github.com/szagoruyko/dccce13465df1542621b728fcc15df53" target="_blank" rel="nofollow noopener">gist.github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://gist.github.com/szagoruyko/dccce13465df1542621b728fcc15df53" target="_blank" rel="nofollow noopener">https://gist.github.com/szagoruyko/dccce13465df1542621b728fcc15df53</a></h4>
<h5>cupy-pytorch-ptx.py</h5>
<pre><code class="Python">import torch
from cupy.cuda import function
from pynvrtc.compiler import Program
from collections import namedtuple

a = torch.randn(1,4,4).cuda()
b = torch.zeros(a.size()).cuda()

kernel = '''
extern "C"</code></pre>
This file has been truncated. <a href="https://gist.github.com/szagoruyko/dccce13465df1542621b728fcc15df53" target="_blank" rel="nofollow noopener">show original</a>

<p>
</p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

username: jekbradbury
cooked: <p>If you need to use cupy in order to run a kernel, like in szagoruyko’s gist, what Soumith posted is what you want. But that doesn’t create a full-fledged cupy ndarray object; to do that you’d need to replicate the functionality of <code>torch.tensor.numpy()</code>. In particular you need to account for the fact that numpy/cupy strides use bytes while torch strides use element counts; other than that you can use the same memory pointer etc.</p>

username: tushar
cooked: <p>Great! Thank you both.</p>
<blockquote>
<p>But that doesn’t create a full-fledged cupy ndarray object</p>
</blockquote>
<p>For now, I do just need to run a kernel, so the gist should work, but thanks for pointing this out.</p>

username: michaelklachko
cooked: <p>Can we now directly convert between cupy and pytorch tensors?</p>
<p>For example, this seem to work in Pytorch 1.4:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; import cupy
&gt;&gt;&gt;
&gt;&gt;&gt; t = torch.cuda.ByteTensor([2, 22, 222])
&gt;&gt;&gt; c = cupy.asarray(t)
&gt;&gt;&gt; c_bits = cupy.unpackbits(c)
&gt;&gt;&gt; t_bits = torch.as_tensor(c_bits, device="cuda")
&gt;&gt;&gt;
&gt;&gt;&gt; t_bits.view(-1, 8)
tensor([[0, 0, 0, 0, 0, 0, 1, 0],
        [0, 0, 0, 1, 0, 1, 1, 0],
        [1, 1, 0, 1, 1, 1, 1, 0]], device='cuda:0', dtype=torch.uint8)
</code></pre>
<p>Are there any issues with this code? Do the arrays remain on GPU during this conversion? Is tensor t being copied during conversion to cupy? I assume so, because I doubt Pytorch and CuPy are able to share the same cuda memory.</p>

username: senpai-a
cooked: <p>I think DLPack is the solution. According to <a href="https://docs-cupy.chainer.org/en/stable/reference/interoperability.html" rel="nofollow noopener">CuPy documentation</a>, you can do:</p>
<pre><code class="lang-auto">import cupy
import torch

from torch.utils.dlpack import to_dlpack
from torch.utils.dlpack import from_dlpack

# Create a PyTorch tensor.
tx1 = torch.randn(1, 2, 3, 4).cuda()

# Convert it into a DLPack tensor.
dx = to_dlpack(tx1)

# Convert it into a CuPy array.
cx = cupy.fromDlpack(dx)

# Convert it back to a PyTorch tensor.
tx2 = from_dlpack(cx.toDlpack())
</code></pre>

username: michaelklachko
cooked: <p>You don’t need dlpack. Direct conversion as in my example works fine.</p>

username: edavalosanaya
cooked: <p>I tried using <a class="mention" href="/u/michaelklachko">@michaelklachko</a> method and it works. However, the output of  <code>cupy.unpackbits</code> unpacks all the bits which might not be what you might want. <a class="mention" href="/u/senpai-a">@senpai-a</a> suggestion of using DLPack works well if you want to preserve the value with the right dtype.</p>

username: michaelklachko
cooked: <p>You misunderstood my example. You don’t need unpackbits or anything like that to pass data from CuPy to Pytorch. That was just something I wanted to do.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; t1 = torch.cuda.ByteTensor([2, 22, 222])
&gt;&gt;&gt; c1 = cupy.asarray(t1)
&gt;&gt;&gt; t2 = torch.as_tensor(c, device="cuda")
&gt;&gt;&gt; c2 = cupy.asarray(t2)

&gt;&gt;&gt; t1
tensor([  2,  22, 222], device='cuda:0', dtype=torch.uint8)
&gt;&gt;&gt; t2
tensor([  2,  22, 222], device='cuda:0', dtype=torch.uint8)
&gt;&gt;&gt; c1
array([  2,  22, 222], dtype=uint8)
&gt;&gt;&gt; c2
array([  2,  22, 222], dtype=uint8)
</code></pre>
<p>Note the dtype is preserved.</p>

username: edavalosanaya
cooked: <p>Sorry <a class="mention" href="/u/michaelklachko">@michaelklachko</a>, you are right. I misunderstood your approach. Thank you for the clarification.</p>