Repository: numpy/numpy

Issue:
DOC: docstring for numpy.finfo.eps is incorrect
The [docstring](https://github.com/numpy/numpy/blob/97c35365beda55c6dead8c50df785eb857f843f0/numpy/core/getlimits.py#L33-L36) for `numpy.finfo` currently defines the `eps` attribute as:

> The smallest representable positive number such that 1.0 + eps != 1.0. [...]

That definition is not correct, at least in the common case of the IEEE 754 binary formats. For example, with the IEEE 754 binary64 format under the usual round-ties-to-even rounding mode, the stated definition gives a value of `2**-53 + 2**-105` (approx. `1.1102230246251568e-16`), which is a little over half of the correct value of `2**-52` (approx `2.220446049250313e-16`).

```
In [36]: eps = 2**-53 + 2**-105

In [37]: eps
Out[37]: 1.1102230246251568e-16

In [38]: 1.0 + eps != 1.0
Out[38]: True

In [39]: 1.0 + np.nextafter(eps, -np.inf) != 1.0
Out[39]: False

In [40]: np.finfo(float).eps
Out[40]: 2.2204460492503131e-16
```

Some possible rewordings:

> The smallest positive number such that 1.0 + eps is representable.

Or:

> The difference between 1.0 and the smallest representable float larger than 1.0.

Or it could be defined in terms of `nextafter` as:

> The value of `np.nextafter(1.0, np.inf) - 1.0`.

For comparison, section 5.2.4.2.2, paragraph 11 of the C99 standard defines the various `*_EPSILON` macros (`DBL_EPSILON`, `FLT_EPSILON`, ...) as:

> the difference between 1 and the least value greater than 1 that is representable in the given floating point type, b<sup>1− p</sup>

The docstring for `epsneg` is similarly incorrect.



Discussion:
Ah, looking at the [source](https://github.com/numpy/numpy/issues/6940), I see that `eps` is actually computed as the smallest _power of 2_ such that `1 + eps != eps`. That's not the same thing as the difference between `1.0` and the next representable float up from `1.0`, though the two things do happen to coincide for IEEE 754 and the round-ties-to-even rounding mode. So my suggested rewordings above are invalid. Perhaps something like:

> The smallest power of 2 such that 1.0 + eps is representable.

would be better? Technically, that `2` should be replaced with the floating-point radix, but that's probably unnecessarily pedantic. Does NumPy currently have any support for radix 10 or radix 16 floats?

[As an aside, I much prefer the C99-style definition of epsilon, since it involves only the format and doesn't depend on the semantics of floating-point addition. In particular, the C99 definition doesn't depend on the rounding mode currently in effect, while the `1 + eps != 1` definition does.]

A pointer to `np.spacing` and `np.nextafter` in the "See Also" section would also be potentially useful.

Here we are almost 4 years later and this hasn't been fixed.  I don't think that https://github.com/numpy/numpy/pull/14618 will be sufficient to close this issue.  

To clarify the bug report, the current definition of eps in the docstring is off by about a factor of 2 from the value of eps returned by np.finfo.  (The numbers I give below are for implementations with IEEE-754 64 bit floating point standard.)  I.e.,

np.finfo(1.0).eps = 2**-52 = 2.220446049250313e-16

However, the docstring defines eps as 

> "The smallest representable positive number such that 1.0 + eps != 1.0."

The actual value of eps that would satisfy the definition 1.0+eps_min != 1.0 is:

eps_min = 2**-53+2**-105 = 1.1102230246251568e-16

which is almost 1/2 of the present value of np.finfo(1.0).eps.  We probably don't want to change the value of np.finfo(1.0).eps, to keep backwards compatibility, but the docstring definition of eps needs to be corrected to match what is calculated.  The source code for np.finfo(1.0)eps uses numpy.MachAr, and looking at the documentation there, which allows other bases besides 2, I think the best definition of of eps is one of the early options that @mdickinson previously suggested.  I would also add an example value to the description:

> "The difference between 1.0 and the smallest representable float larger than 1.0.  (For 64 bit binary floats in the IEEE-754 standard, eps = 2**-52 ≅ 2.22e-16.)"

The docstring for epsneg is similarly incorrect.

I also just stumbled over this inaccuracy. This ought to be fixed.

For reference, Matlab [documents](https://de.mathworks.com/help/matlab/ref/eps.html) its `eps` (which has the same value) as: "the distance from 1.0 to the next larger double-precision number, that is, 2^-52."
The current definition is incorrect and confusing due to rounding issues. @gwhammett the proposed definition is both correct and more descriptive and encourage a PR to close this issue.

Resolving patch:
diff --git a/numpy/core/getlimits.py b/numpy/core/getlimits.py
--- a/numpy/core/getlimits.py
+++ b/numpy/core/getlimits.py
@@ -300,12 +300,13 @@ class finfo:
     bits : int
         The number of bits occupied by the type.
     eps : float
-        The smallest representable positive number such that
-        ``1.0 + eps != 1.0``.  Type of `eps` is an appropriate floating
-        point type.
-    epsneg : floating point number of the appropriate type
-        The smallest representable positive number such that
-        ``1.0 - epsneg != 1.0``.
+        The difference between 1.0 and the next smallest representable float
+        larger than 1.0. For example, for 64-bit binary floats in the IEEE-754
+        standard, ``eps = 2**-52``, approximately 2.22e-16.
+    epsneg : float
+        The difference between 1.0 and the next smallest representable float
+        less than 1.0. For example, for 64-bit binary floats in the IEEE-754
+        standard, ``epsneg = 2**-53``, approximately 1.11e-16.
     iexp : int
         The number of bits in the exponent portion of the floating point
         representation.
@@ -348,6 +349,8 @@ class finfo:
     --------
     MachAr : The implementation of the tests that produce this information.
     iinfo : The equivalent for integer data types.
+    spacing : The distance between a value and the nearest adjacent number
+    nextafter : The next floating point value after x1 towards x2
 
     Notes
     -----


Test patch: