Repository: numba/numba

Issue:
np.transpose crashes when passed a tuple
Minimal reproducing code using `numba==0.43.1`:
```
from numba import jit
import numpy as np


def without_numba():
    t = (np.array([0, 1]),)
    print(np.transpose(t))

@jit(nopython=True)
def with_numba():
    t = (np.array([0, 1]),)
    print(np.transpose(t))


without_numba()
with_numba()
```

The error reported mentions:
```
Invalid use of Function(<function transpose at 0x08479ED0>) with argument(s) of type(s): (tuple(array(int32, 1d, C) x 1))
```

I tried to find a fix, e.g. attempted to add a call to `np.asarray` in `numpy_transpose` in `targets/arrayobj.py:1492`, but no luck. It actually revealed a second bug:

```
def without_numba():
    t = (np.array([0, 1]),)
    print(np.asarray(t))

@jit(nopython=True)
def with_numba():
    t = (np.array([0, 1]),)
    print(np.asarray(t))
```

I am a bit overwhelmed by Numba's code complexity, so I don't think I'll be able to find a fix on my own.


Discussion:
@Lucas-C thanks for using Numba and reporting this. I can reproduce this locally and it looks like a bug indeed. The Numpy implementation seems to be able to deal with tuples that contain arrays, whereas the Numba one doesn't.
@esc agree, the problem is fundamentally that `np.transpose` is wired through from `ndarray.transpose` in Numba, and there's a `tuple-of-array` type being passed which is not handled.
```python
from numba import jit
import numpy as np

@jit(nopython=True)
def with_numba():
    t = (np.array([0, 1]),) # <--- tuple(array,)
    print(np.transpose(t))  # <- transpose impl only takes arrays! There's no `tuple.transpose()`


with_numba.py_func()
with_numba()
```
adding a `np.asarray` does indeed expose a second issue in that it also doesn't handle `tuple-of-array`, e.g.:
```python
@jit(nopython=True)
def foo():
    return np.asarray((np.array([0, 1]),))

foo()
```
I think fixing the latter would permit fixing the former.

@Lucas-C so, I thought about this and spoke to my colleague, and I have come to the conclusion, that this will probably not be supported anytime soon. The main problem with this is, that in order to get a proper array from the initial tuple, we need to ensure that all the arrays contained within have the same length/size/shape. Unfortunately, this information isn't available at the typing stage, only information about the type and about the dimensionality is available then. We could do the conversion at runtime, but that will be quite hacky. Also, if you look at what NumPy does in case the arrays are not of the same length:

```
In [1]: import numpy as np

In [2]: a = (np.zeros(2), np.zeros(1))

In [3]: a
Out[3]: (array([0., 0.]), array([0.]))

In [4]: np.transpose(a)
Out[4]: array([array([0., 0.]), array([0.])], dtype=object)

In [5]: a = (np.zeros(2), np.zeros(2))

In [6]: np.transpose(a)
Out[6]:
array([[0., 0.],
       [0., 0.]])
```

As you can see, in the first case, NumPy will convert the tuple to an `object` array. I should mention that Numba has zero support for the `object` dtype.

What can be done, however, is improve the error message, and ensure that `np.transpose` doesn't accept tuples and issues an appropriate error message. The respective code is here:

https://github.com/numba/numba/blob/master/numba/targets/arrayobj.py#L1491

And there is some preliminary documentation here:

https://github.com/numba/numba/pull/3951

Lastly, to solve you original issue, I would suggest to make a 2-D array in the first place and then handle that. If that is not a possibility, maybe you could tell us a little more about your use-case and we will see if we can find a workaround for you?
Thank you very much for your work and this and detailed explanation !

My initial use case was to use [`np.argwhere`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.argwhere.html). Because it is not supported by numba currently and [under the hood](https://github.com/numpy/numpy/blob/v1.16.1/numpy/core/numeric.py#L842-L881) it calls `transpose(nonzero(a))`, I attempted to call those. But `np.nonzero` returns a tuple, hence this issue...
@Lucas-C can you wrap the output in a call to `np.asarray` from outside of JIT compiled code?
@Lucas-C I see where you are coming from, I just tried the following:

```
In [7]: @njit
   ...: def foo(a):
   ...:     return np.transpose(np.nonzero(a))
```

And that leads exactly to the issue you reported.
For reference the tuple is created here:

https://github.com/numba/numba/blob/master/numba/targets/arraymath.py#L2646
@Lucas-C you can use the following, potentially:

```
In [11]: @njit
    ...: def foo(a):
    ...:     return np.transpose(np.vstack(np.nonzero(a)))
```
Thank you @esc !
I missed the notification that you replied here, sorry.
Thanks for the suggestion. I won't be able to test it right now but it looks like a nice workaround.
As far as I'm concern, it fits my need
@Lucas-C no problem, glad I could help. @stuartarchibald shall we close this one?
@esc I think this needs resolving to close this ticket (either through fix or through new ticket to note the work needed):

> What can be done, however, is improve the error message, and ensure that np.transpose doesn't accept tuples and issues an appropriate error message

Resolving patch:
diff --git a/numba/targets/arrayobj.py b/numba/targets/arrayobj.py
--- a/numba/targets/arrayobj.py
+++ b/numba/targets/arrayobj.py
@@ -1491,6 +1491,8 @@ def array_transpose_vararg(context, builder, sig, args):
 
 @overload(np.transpose)
 def numpy_transpose(a, axes=None):
+    if isinstance(a, types.BaseTuple):
+        raise errors.UnsupportedError("np.transpose does not accept tuples")
 
     if axes is None:
         def np_transpose_impl(a, axes=None):


Test patch:
diff --git a/numba/tests/test_array_manipulation.py b/numba/tests/test_array_manipulation.py
--- a/numba/tests/test_array_manipulation.py
+++ b/numba/tests/test_array_manipulation.py
@@ -333,6 +333,10 @@ def check_err_invalid_args(pyfunc, arr, axes):
         check_err_axis_oob(arrs[3], (3, 1, 2, 5))
         check_err_axis_oob(arrs[3], (3, 1, 2, -5))
 
+        with self.assertRaises(TypingError) as e:
+            jit(nopython=True)(numpy_transpose_array)((np.array([0, 1]),))
+        self.assertIn("np.transpose does not accept tuples",
+                        str(e.exception))
 
     @tag('important')
     def test_expand_dims(self):