Repository: explosion/thinc

Issue:
Cupy TypeError when using asarray with torch tensors
Hi,

I'm chaining a custom feature extraction model with a PyTorchWrapper model. In the feature extraction part, I'm using asarray function on a torch tensor but as you can see below, cupy raises an error when MemoryPointer is called. Do you have any idea why that line raises an exception?

Thanks,

```
Traceback (most recent call last): 
...
  File "/storage/myuser/myproject/main.py", line 180, in forward
    features.append(model.ops.asarray2f(attrs, dtype="float32"))
  File "/storage/miniconda/envs/safirx-ml/lib/python3.9/site-packages/thinc/backends/ops.py", line 494, in asarray2f
    return cast(Floats2d, self.asarray(data, dtype=dtype))
  File "/storage/miniconda/envs/safirx-ml/lib/python3.9/site-packages/thinc/backends/cupy_ops.py", line 67, in asarray
    pointer = cupy.cuda.MemoryPointer(data.data_ptr())
  File "cupy/cuda/memory.pyx", line 343, in cupy.cuda.memory.MemoryPointer.__init__
TypeError: __init__() takes exactly 2 positional arguments (1 given)```


Discussion:
Hmm, this looks like a bug since from the cupy docs/code it looks like it should be `MemoryPointer(ptr, offset)`. I haven't tested it, but I suspect it should just be `cupy.cuda.MemoryPointer(data.data_ptr(), 0)`?
> Hmm, this looks like a bug since from the cupy docs/code it looks like it should be `MemoryPointer(ptr, offset)`. I haven't tested it, but I suspect it should just be `cupy.cuda.MemoryPointer(data.data_ptr(), 0)`?

I tried this but I got another TypeError, as the first argument of MemoryPointer is expected to be cupy.cuda.memory.BaseMemory but data.data_ptr() returns int.
Hmm, this is possibly all just relatively untested. We will look into it!

Resolving patch:
diff --git a/thinc/backends/cupy_ops.py b/thinc/backends/cupy_ops.py
--- a/thinc/backends/cupy_ops.py
+++ b/thinc/backends/cupy_ops.py
@@ -19,6 +19,8 @@
 from .numpy_ops import NumpyOps
 from . import _custom_kernels
 from ..types import DeviceTypes
+from ..util import torch2xp, tensorflow2xp, mxnet2xp
+from ..util import is_torch_array, is_tensorflow_array, is_mxnet_array
 
 
 @registry.ops("CupyOps")
@@ -73,17 +75,28 @@ def asarray(self, data, dtype=None):
         # This is sort of frustrating, but we can't easily otherwise pass
         # forward "unset".
         dtype = {"dtype": dtype} if dtype is not None else {}
+
+        # We'll try to perform a zero-copy conversion if possible.
+        array = None
+        cast_array = False
         if isinstance(data, cupy.ndarray):
-            return self.xp.asarray(data, **dtype)
-        elif hasattr(data, "data_ptr"):
-            # Handles PyTorch Tensors
-            pointer = cupy.cuda.MemoryPointer(data.data_ptr())
-            shape = data.stride()
-            array = self.xp.ndarray(shape, memptr=pointer, **dtype)
-            return array
+            array = self.xp.asarray(data, **dtype)
+        elif is_torch_array(data) and data.device.type == "cuda":
+            array = torch2xp(data)
+            cast_array = True
+        elif is_tensorflow_array(data) and "GPU:" in data.device:
+            array = tensorflow2xp(data)
+            cast_array = True
+        elif is_mxnet_array(data) and data.context.device_type != "cpu":
+            array = mxnet2xp(data)
+            cast_array = True
         else:
-            result = self.xp.array(data, **dtype)
-            return result
+            array = self.xp.array(data, **dtype)
+
+        if cast_array and dtype != {}:
+            array = array.astype(dtype["dtype"])
+
+        return array
 
     def maxout(self, X):
         return _custom_kernels.maxout(X)


Test patch:
diff --git a/thinc/tests/regression/test_issue564.py b/thinc/tests/regression/test_issue564.py
new file mode 100644
--- /dev/null
+++ b/thinc/tests/regression/test_issue564.py
@@ -0,0 +1,21 @@
+import pytest
+
+from thinc.api import CupyOps
+from thinc.util import has_torch, has_torch_gpu
+
+
+@pytest.mark.skipif(not has_torch, reason="needs PyTorch")
+@pytest.mark.skipif(not has_torch_gpu, reason="needs a GPU")
+def test_issue564():
+    import torch
+
+    if CupyOps.xp is not None:
+        ops = CupyOps()
+        t = torch.zeros((10, 2)).cuda()
+        a = ops.asarray(t)
+
+        assert a.shape == t.shape
+        ops.xp.testing.assert_allclose(
+            a,
+            ops.alloc2f(10, 2),
+        )