Repository: numpy/numpy

Issue:
repr change for arrays of size 1 even with np.set_printoptions(legacy=True) in numpy master
In scikit-learn we have seen failures in doctests for a few days on our build using numpy-dev. This is one of them:

```py
import numpy as np

try:
    np.set_printoptions(legacy=True)
except Exception:
    pass

print(repr(np.array([0.])))
print(repr(np.array([[0.]])))
print(repr(np.array([1., 2., 3.])))

```

With numpy master:
```
array([0.])
array([[0.]])
array([ 1.,  2.,  3.])
```

With numpy 1.13 (note the space before the 0):
```
array([ 0.])
array([[ 0.]])
array([ 1.,  2.,  3.])
```




Discussion:
It's a feature, not a bug ;) The default print formatting will change in 1.14. We are aware that this will break doctests. From the release notes
```
Changes to array printing, and the new "legacy" printing mode.
--------------------------------------------------------------
The new ``sign='-'`` option (see new features below) causes the ``repr`` of
float arrays to often omit a whitespace previously printed in the sign
position, and 0d arrays also now print with small whitespace and precision
differences (see changes below). These changes are likely to break downstream
user's doctests.

These new behaviors can be disabled to mostly reproduce numpy 1.13 behavior by
enabling the new "legacy" printing mode. This is enabled by calling
``np.set_printoptions(legacy=True)``, or using the new ``legacy`` argument
to ``np.array2string``.
```
Note that there are various ways to relax whitespace sensitivity of doctests which are probably worth looking into if you want to continue to rely on doctests. @ahaldane Further suggestions?
Note that there are more changes than you show, for instance
```
In [1]: print(repr(np.array([1., 2., 3.])))
array([1., 2., 3.])
```
Note missing spaces.
Oops, I see that you did use the legacy mode, so we can maybe fix that.
> Oops, I see that you did use the legacy mode, so we can maybe fix that.

That would be great!

For now our strategy is to use the legacy mode to have compatibility of our doctest with numpy 1.14dev and 1.13. When numpy 1.14 is released we will revisit our strategy. One possible solution is to update all of our doctests to work with 1.14 and just skip them for numpy <= 1.13.



~~I'm confused. So did numpy decide to definitely break, regardless of the sign/legacy setting?~~

(this was not the issue I meant to comment on, and this thread actually answered my question, sorry for the noise)
Something that was printed as 1.0 is now printed as 0.99999999(...?) is that also a feature?
What code exactly printed `0.99999999`? There is a new float printing algorithm in 1.14 which, barring a bug, should print just like a python3 float.
I'll take a look at the space in the legacy option, I'm pretty sure that is a fixable bug.
> What code exactly printed 0.99999999? There is a new float printing algorithm in 1.14 which, barring a bug, should print just like a python3 float.

There is a case for `np.set_printoptions(legacy=True)` to implement the pre-dragon4 printing. I am not sure it is worth the cost, I'll open a separate issue to discuss this. FWIW in scikit-learn we have managed to work around these issues by using doctest ellipsis.
Yes, we can discuss that. But if you see a number that is printed as `0.99999999` when it should be printed as `1.0` let us know. 

The goal of the new algorithm is to round "correctly" for all float sizes, so if there is an instance where it isn't rounding correctly, that is important.

Resolving patch:
diff --git a/numpy/core/arrayprint.py b/numpy/core/arrayprint.py
--- a/numpy/core/arrayprint.py
+++ b/numpy/core/arrayprint.py
@@ -39,7 +39,7 @@
 
 import numpy as np
 from . import numerictypes as _nt
-from .umath import absolute, not_equal, isnan, isinf
+from .umath import absolute, not_equal, isnan, isinf, isfinite
 from . import multiarray
 from .multiarray import (array, dragon4_positional, dragon4_scientific,
                          datetime_as_string, datetime_data, dtype, ndarray)
@@ -87,6 +87,10 @@ def _make_options_dict(precision=None, threshold=None, edgeitems=None,
     if sign not in [None, '-', '+', ' ']:
         raise ValueError("sign option must be one of ' ', '+', or '-'")
 
+    if legacy not in [None, False, '1.13']:
+        warnings.warn("legacy printing option can currently only be '1.13' or "
+                      "`False`", stacklevel=3)
+
     return options
 
 def set_printoptions(precision=None, threshold=None, edgeitems=None,
@@ -169,10 +173,14 @@ def set_printoptions(precision=None, threshold=None, edgeitems=None,
                     but if every element in the array can be uniquely
                     represented with an equal number of fewer digits, use that
                     many digits for all elements.
-    legacy : boolean, optional
-        If True, enables legacy printing mode, which approximates numpy 1.13
-        print output by including a space in the sign position of floats and
-        different behavior for 0d arrays.
+    legacy : string or `False`, optional
+        If set to the string `'1.13'` enables 1.13 legacy printing mode. This
+        approximates numpy 1.13 print output by including a space in the sign
+        position of floats and different behavior for 0d arrays. If set to
+        `False`, disables legacy mode. Unrecognized strings will be ignored
+        with a warning for forward compatibility.
+        
+        .. versionadded:: 1.14.0
 
     See Also
     --------
@@ -525,11 +533,14 @@ def array2string(a, max_line_width=None, precision=None,
                     but if every element in the array can be uniquely
                     represented with an equal number of fewer digits, use that
                     many digits for all elements.
-    legacy : boolean, optional
-        If True, enables legacy printing mode, which overrides the `sign`
-        option.  Legacy printing mode approximates numpy 1.13 print output,
-        which includes a space in the sign position of floats and different
-        behavior for 0d arrays.
+    legacy : string or `False`, optional
+        If set to the string `'1.13'` enables 1.13 legacy printing mode. This
+        approximates numpy 1.13 print output by including a space in the sign
+        position of floats and different behavior for 0d arrays. If set to
+        `False`, disables legacy mode. Unrecognized strings will be ignored
+        with a warning for forward compatibility.
+        
+        .. versionadded:: 1.14.0
 
     Returns
     -------
@@ -581,13 +592,13 @@ def array2string(a, max_line_width=None, precision=None,
     options = _format_options.copy()
     options.update(overrides)
 
-    if options['legacy']:
+    if options['legacy'] == '1.13':
         if a.shape == () and not a.dtype.names:
             return style(a.item())
     elif style is not np._NoValue:
         # Deprecation 11-9-2017  v1.14
         warnings.warn("'style' argument is deprecated and no longer functional"
-                      " except in 'legacy' mode",
+                      " except in 1.13 'legacy' mode",
                       DeprecationWarning, stacklevel=3)
 
     # treat as a null array if any of shape elements == 0
@@ -675,14 +686,14 @@ def _formatArray(a, format_function, rank, max_line_len,
 
 class FloatingFormat(object):
     """ Formatter for subtypes of np.floating """
-    def __init__(self, data, precision, floatmode, suppress_small, sign=False, **kwarg):
+    def __init__(self, data, precision, floatmode, suppress_small, sign=False,
+                 **kwarg):
         # for backcompatibility, accept bools
         if isinstance(sign, bool):
             sign = '+' if sign else '-'
 
-        self._legacy = False
-        if kwarg.get('legacy', False):
-            self._legacy = True
+        self._legacy = kwarg.get('legacy', False)
+        if self._legacy == '1.13':
             sign = '-' if data.shape == () else ' '
 
         self.floatmode = floatmode
@@ -693,6 +704,7 @@ def __init__(self, data, precision, floatmode, suppress_small, sign=False, **kwa
                 raise ValueError(
                     "precision must be >= 0 in {} mode".format(floatmode))
             self.precision = precision
+
         self.suppress_small = suppress_small
         self.sign = sign
         self.exp_format = False
@@ -701,40 +713,33 @@ def __init__(self, data, precision, floatmode, suppress_small, sign=False, **kwa
         self.fillFormat(data)
 
     def fillFormat(self, data):
-        with errstate(all='ignore'):
-            hasinf = isinf(data)
-            special = isnan(data) | hasinf
-            valid = not_equal(data, 0) & ~special
-            non_zero = data[valid]
-            abs_non_zero = absolute(non_zero)
-            if len(non_zero) == 0:
-                max_val = 0.
-                min_val = 0.
-                min_val_sgn = 0.
-            else:
-                max_val = np.max(abs_non_zero)
-                min_val = np.min(abs_non_zero)
-                min_val_sgn = np.min(non_zero)
-                if max_val >= 1.e8:
-                    self.exp_format = True
-                if not self.suppress_small and (min_val < 0.0001
-                                           or max_val/min_val > 1000.):
+        # only the finite values are used to compute the number of digits
+        finite_vals = data[isfinite(data)]
+
+        # choose exponential mode based on the non-zero finite values:
+        abs_non_zero = absolute(finite_vals[finite_vals != 0])
+        if len(abs_non_zero) != 0:
+            max_val = np.max(abs_non_zero)
+            min_val = np.min(abs_non_zero)
+            with errstate(over='ignore'):  # division can overflow
+                if max_val >= 1.e8 or (not self.suppress_small and
+                        (min_val < 0.0001 or max_val/min_val > 1000.)):
                     self.exp_format = True
 
-        if len(non_zero) == 0:
+        # do a first pass of printing all the numbers, to determine sizes
+        if len(finite_vals) == 0:
             self.pad_left = 0
             self.pad_right = 0
             self.trim = '.'
             self.exp_size = -1
             self.unique = True
         elif self.exp_format:
-            # first pass printing to determine sizes
             trim, unique = '.', True
-            if self.floatmode == 'fixed' or self._legacy:
+            if self.floatmode == 'fixed' or self._legacy == '1.13':
                 trim, unique = 'k', False
             strs = (dragon4_scientific(x, precision=self.precision,
                                unique=unique, trim=trim, sign=self.sign == '+')
-                    for x in non_zero)
+                    for x in finite_vals)
             frac_strs, _, exp_strs = zip(*(s.partition('e') for s in strs))
             int_part, frac_part = zip(*(s.split('.') for s in frac_strs))
             self.exp_size = max(len(s) for s in exp_strs) - 1
@@ -743,12 +748,14 @@ def fillFormat(self, data):
             self.precision = max(len(s) for s in frac_part)
 
             # for back-compatibility with np 1.13, use two spaces and full prec
-            if self._legacy:
-                self.pad_left = 2 + (not (all(non_zero > 0) and self.sign == ' '))
+            if self._legacy == '1.13':
+                # undo addition of sign pos below
+                will_add_sign = all(finite_vals > 0) and self.sign == ' '
+                self.pad_left = 3 - will_add_sign
             else:
-                # this should be only 1 or two. Can be calculated from sign.
+                # this should be only 1 or 2. Can be calculated from sign.
                 self.pad_left = max(len(s) for s in int_part)
-            # pad_right is not used to print, but needed for nan length calculation
+            # pad_right is only needed for nan length calculation
             self.pad_right = self.exp_size + 2 + self.precision
 
             self.unique = False
@@ -761,7 +768,7 @@ def fillFormat(self, data):
                                        fractional=True,
                                        unique=unique, trim=trim,
                                        sign=self.sign == '+')
-                    for x in non_zero)
+                    for x in finite_vals)
             int_part, frac_part = zip(*(s.split('.') for s in strs))
             self.pad_left = max(len(s) for s in int_part)
             self.pad_right = max(len(s) for s in frac_part)
@@ -776,11 +783,12 @@ def fillFormat(self, data):
                 self.trim = '.'
 
         # account for sign = ' ' by adding one to pad_left
-        if len(non_zero) > 0 and all(non_zero > 0) and self.sign == ' ':
+        if all(finite_vals >= 0) and self.sign == ' ':
             self.pad_left += 1
 
-        if any(special):
-            neginf = self.sign != '-' or any(data[hasinf] < 0)
+        # if there are non-finite values, may need to increase pad_left
+        if data.size != finite_vals.size:
+            neginf = self.sign != '-' or any(data[isinf(data)] < 0)
             nanlen = len(_format_options['nanstr'])
             inflen = len(_format_options['infstr']) + neginf
             offset = self.pad_right + 1  # +1 for decimal pt
@@ -833,7 +841,7 @@ def __init__(self, *args, **kwargs):
 def format_float_scientific(x, precision=None, unique=True, trim='k',
                             sign=False, pad_left=None, exp_digits=None):
     """
-    Format a floating-point scalar as a string in scientific notation.
+    Format a floating-point scalar as a decimal string in scientific notation.
 
     Provides control over rounding, trimming and padding. Uses and assumes
     IEEE unbiased rounding. Uses the "Dragon4" algorithm.
@@ -900,7 +908,7 @@ def format_float_positional(x, precision=None, unique=True,
                             fractional=True, trim='k', sign=False,
                             pad_left=None, pad_right=None):
     """
-    Format a floating-point scalar as a string in positional notation.
+    Format a floating-point scalar as a decimal string in positional notation.
 
     Provides control over rounding, trimming and padding. Uses and assumes
     IEEE unbiased rounding. Uses the "Dragon4" algorithm.
@@ -1182,7 +1190,8 @@ def array_repr(arr, max_line_width=None, precision=None, suppress_small=None):
     else:
         class_name = "array"
 
-    if _format_options['legacy'] and arr.shape == () and not arr.dtype.names:
+    if (_format_options['legacy'] == '1.13' and
+            arr.shape == () and not arr.dtype.names):
         lst = repr(arr.item())
     elif arr.size > 0 or arr.shape == (0,):
         lst = array2string(arr, max_line_width, precision, suppress_small,
@@ -1243,7 +1252,8 @@ def array_str(a, max_line_width=None, precision=None, suppress_small=None):
     '[0 1 2]'
 
     """
-    if _format_options['legacy'] and a.shape == () and not a.dtype.names:
+    if (_format_options['legacy'] == '1.13' and
+            a.shape == () and not a.dtype.names):
         return str(a.item())
 
     # the str of 0d arrays is a special case: It should appear like a scalar,


Test patch:
diff --git a/numpy/core/tests/test_arrayprint.py b/numpy/core/tests/test_arrayprint.py
--- a/numpy/core/tests/test_arrayprint.py
+++ b/numpy/core/tests/test_arrayprint.py
@@ -288,8 +288,7 @@ def test_0d_arrays(self):
         assert_warns(DeprecationWarning, np.array2string,
                                          np.array(1.), style=repr)
         # but not in legacy mode
-        np.set_printoptions(legacy=True)
-        np.array2string(np.array(1.), style=repr)
+        np.array2string(np.array(1.), style=repr, legacy='1.13')
 
     def test_float_spacing(self):
         x = np.array([1., 2., 3.])
@@ -334,6 +333,7 @@ def test_sign_spacing(self):
         assert_equal(repr(a), 'array([0., 1., 2., 3.])')
         assert_equal(repr(np.array(1.)), 'array(1.)')
         assert_equal(repr(b), 'array([1.234e+09])')
+        assert_equal(repr(np.array([0.])), 'array([0.])')
 
         np.set_printoptions(sign=' ')
         assert_equal(repr(a), 'array([ 0.,  1.,  2.,  3.])')
@@ -345,14 +345,20 @@ def test_sign_spacing(self):
         assert_equal(repr(np.array(1.)), 'array(+1.)')
         assert_equal(repr(b), 'array([+1.234e+09])')
 
-        np.set_printoptions(legacy=True)
+        np.set_printoptions(legacy='1.13')
         assert_equal(repr(a), 'array([ 0.,  1.,  2.,  3.])')
         assert_equal(repr(b),  'array([  1.23400000e+09])')
         assert_equal(repr(-b), 'array([ -1.23400000e+09])')
         assert_equal(repr(np.array(1.)), 'array(1.0)')
+        assert_equal(repr(np.array([0.])), 'array([ 0.])')
 
         assert_raises(TypeError, np.set_printoptions, wrongarg=True)
 
+    def test_float_overflow_nowarn(self):
+        # make sure internal computations in FloatingFormat don't
+        # warn about overflow
+        repr(np.array([1e4, 0.1], dtype='f2'))
+
     def test_sign_spacing_structured(self):
         a = np.ones(2, dtype='f,f')
         assert_equal(repr(a), "array([(1., 1.), (1., 1.)],\n"
@@ -420,6 +426,7 @@ def test_floatmode(self):
         assert_equal(repr(w[::5]),
             "array([1.0000e+00, 1.0000e+05, 1.0000e+10, 1.0000e+15, 1.0000e+20])")
         assert_equal(repr(wp), "array([1.2340e+001, 1.0000e+002, 1.0000e+123])")
+        assert_equal(repr(np.zeros(3)), "array([0.0000, 0.0000, 0.0000])")
         # for larger precision, representation error becomes more apparent:
         np.set_printoptions(floatmode='fixed', precision=8)
         assert_equal(repr(z),