Repository: numpy/numpy

Issue:
numpy array printing regression for ndarray subclasses in NumPy 1.14
It looks like some of the array printing changes in NumPy 1.14 are causing issues in yt. Specifically, printing yt's object that represents a scalar with units is triggering a recursion error. Here is a simplified example that is sufficient to trigger the same error:

```python
import numpy as np


class MyArray(np.ndarray):
    def __new__(cls, inp):
        obj = np.asarray(inp).view(cls)
        return obj

    def __str__(self):
        return super(MyArray, self).__str__()

    def __getitem__(self, item):
        ret = super(MyArray, self).__getitem__(item)
        return MyArray(ret)


if __name__ == "__main__":
    a = MyArray(1)
    print(a)
```

This script triggers an infinite recursion error on NumPy 1.14.0 and prints `1` on NumPy 1.13.3.

Is there any chance that this can be fixed on NumPy's side for a 1.14.1 release? I'm honestly not sure how to fix this on the yt side without making major changes to our ndarray subclasses.

Ping @mhvk 


Discussion:
Also apologies for not testing NumPy's beta and bringing this up before you shipped 1.14.0 :(
Thanks for the minimal test case. I'll try to take a look at this, since the blame is likely on me and/or @ahaldane 
This only happens on 0d arrays - because `str(0d_array)` falls back on `str(0d_array[()])`.

The idea here it to make the string of a 0d array equivalent to the string of the scalar.

In your case, it's not clear how to get hold of the scalar any more to do that. Perhaps we simply need recursion detection that calls `self.view(ndarray)` and retries.
Yeah, this is sort of a quirk in our design, using an array subclass to represent a scalar. We probably shouldn't have done that but that ship sailed a long time ago. If there's a way to poison the recursion somehow that sounds like the best approach.
Here's what I think should happen:

* We should catch `RecursionError` in `ndarray.__str__`, and downcast to retry, with a warning suggesting that maybe `[()]` is not returning a scalar, and therefore `__str__` must be special-cased for 0d
* You should add a special case for 0d `__str__`
> You should add a special case for 0d `__str__`

See https://github.com/yt-project/yt/pull/1661

Thanks for the explanation.
> Also apologies for not testing NumPy's beta and bringing this up before you shipped 1.14.0 :(

It's expected :) I think of the *.0 releases as the true betas, it's why I move to them so quickly these days. There just aren't enough folks who test the the rc's and waiting around is just a waste of time.
Note that this is a problem even if in the above `__str__` is not overridden. Given that, if you subclass `ndarray` to add some new property to it, and you wish to support scalars, something like @ngoldbaum's implementation of recasting scalars as array scalars is the only possibility (we certainly use it in `Quantity` too), I think there will be a lot of other implementations that are going to have this problem, and it would seem wrong to even start giving warnings just for something as simple as doing `print(mysubclass)`. Instead, I would suggest we do one of:
1. Cast of array scalar to scalar via `[()]` only for `ndarray` itself, not for subclasses.
2. Explicitly check that after `[()]`, one now has an instance of a different class.

Option (2) is perhaps the safest, and also closest to the legacy behaviour.

p.s.  It is just luck that in `Quantity`, we just create a string from the ndarray view and then add the unit string afterwards...
> Explicitly check that after [()], one now has an instance of a different class.

This unfortunately is not safe, as we could be indexing an object array. We already have questionable legacy behavior in `ma.array` due to making this assumption, I'd prefer to not introduce it elsewhere.

Resolving patch:
diff --git a/numpy/core/arrayprint.py b/numpy/core/arrayprint.py
--- a/numpy/core/arrayprint.py
+++ b/numpy/core/arrayprint.py
@@ -435,14 +435,17 @@ def wrapper(self, *args, **kwargs):
 # gracefully handle recursive calls, when object arrays contain themselves
 @_recursive_guard()
 def _array2string(a, options, separator=' ', prefix=""):
-    # The formatter __init__s cannot deal with subclasses yet
-    data = asarray(a)
+    # The formatter __init__s in _get_format_function cannot deal with
+    # subclasses yet, and we also need to avoid recursion issues in
+    # _formatArray with subclasses which return 0d arrays in place of scalars
+    a = asarray(a)
 
     if a.size > options['threshold']:
         summary_insert = "..."
-        data = _leading_trailing(data, options['edgeitems'])
+        data = _leading_trailing(a, options['edgeitems'])
     else:
         summary_insert = ""
+        data = a
 
     # find the right formatting function for the array
     format_function = _get_format_function(data, **options)
@@ -468,7 +471,7 @@ def array2string(a, max_line_width=None, precision=None,
 
     Parameters
     ----------
-    a : ndarray
+    a : array_like
         Input array.
     max_line_width : int, optional
         The maximum number of columns the string should span. Newline
@@ -730,7 +733,7 @@ def recurser(index, hanging_indent, curr_width):
 
             if show_summary:
                 if legacy == '1.13':
-                    # trailing space, fixed number of newlines, and fixed separator
+                    # trailing space, fixed nbr of newlines, and fixed separator
                     s += hanging_indent + summary_insert + ", \n"
                 else:
                     s += hanging_indent + summary_insert + line_sep
@@ -1380,6 +1383,8 @@ def array_repr(arr, max_line_width=None, precision=None, suppress_small=None):
 
     return arr_str + spacer + dtype_str
 
+_guarded_str = _recursive_guard()(str)
+
 def array_str(a, max_line_width=None, precision=None, suppress_small=None):
     """
     Return a string representation of the data in an array.
@@ -1422,7 +1427,10 @@ def array_str(a, max_line_width=None, precision=None, suppress_small=None):
     # so floats are not truncated by `precision`, and strings are not wrapped
     # in quotes. So we return the str of the scalar value.
     if a.shape == ():
-        return str(a[()])
+        # obtain a scalar and call str on it, avoiding problems for subclasses
+        # for which indexing with () returns a 0d instead of a scalar by using
+        # ndarray's getindex. Also guard against recursive 0d object arrays.
+        return _guarded_str(np.ndarray.__getitem__(a, ()))
 
     return array2string(a, max_line_width, precision, suppress_small, ' ', "")
 


Test patch:
diff --git a/numpy/core/tests/test_arrayprint.py b/numpy/core/tests/test_arrayprint.py
--- a/numpy/core/tests/test_arrayprint.py
+++ b/numpy/core/tests/test_arrayprint.py
@@ -34,6 +34,55 @@ class sub(np.ndarray): pass
             "     [(1,), (1,)]], dtype=[('a', '<i4')])"
         )
 
+    def test_0d_object_subclass(self):
+        # make sure that subclasses which return 0ds instead
+        # of scalars don't cause infinite recursion in str
+        class sub(np.ndarray):
+            def __new__(cls, inp):
+                obj = np.asarray(inp).view(cls)
+                return obj
+
+            def __getitem__(self, ind):
+                ret = super(sub, self).__getitem__(ind)
+                return sub(ret)
+
+        x = sub(1)
+        assert_equal(repr(x), 'sub(1)')
+        assert_equal(str(x), '1')
+
+        x = sub([1, 1])
+        assert_equal(repr(x), 'sub([1, 1])')
+        assert_equal(str(x), '[1 1]')
+
+        # check it works properly with object arrays too
+        x = sub(None)
+        assert_equal(repr(x), 'sub(None, dtype=object)')
+        assert_equal(str(x), 'None')
+
+        # plus recursive object arrays (even depth > 1)
+        y = sub(None)
+        x[()] = y
+        y[()] = x
+        assert_equal(repr(x),
+            'sub(sub(sub(..., dtype=object), dtype=object), dtype=object)')
+        assert_equal(str(x), '...')
+
+        # nested 0d-subclass-object
+        x = sub(None)
+        x[()] = sub(None)
+        assert_equal(repr(x), 'sub(sub(None, dtype=object), dtype=object)')
+        assert_equal(str(x), 'None')
+
+        # test that object + subclass is OK:
+        x = sub([None, None])
+        assert_equal(repr(x), 'sub([None, None], dtype=object)')
+        assert_equal(str(x), '[None None]')
+
+        x = sub([None, sub([None, None])])
+        assert_equal(repr(x),
+            'sub([None, sub([None, None], dtype=object)], dtype=object)')
+        assert_equal(str(x), '[None sub([None, None], dtype=object)]')
+
     def test_self_containing(self):
         arr0d = np.array(None)
         arr0d[()] = arr0d