Repository: pandas-dev/pandas

Issue:
Inconsistent behaviour in Timestamp.round
#### Code Sample

```python
>>> import pandas as pd
>>> a = pd.to_datetime('2000-01-01 0:0:0.124999360')
>>> b = pd.to_datetime('2018-01-01 0:0:0.124999360')
>>> a.round('us')
Timestamp('2000-01-01 00:00:00.124999')
>>> b.round('us')
Timestamp('2018-01-01 00:00:00.125000')
```
#### Problem description

According to the standard definition of [rounding to a multiple](https://en.wikipedia.org/wiki/Rounding#Rounding_to_a_specified_multiple) `b.round('us')`  in the example above gives the wrong answer.

I suspect that there is loss of precision due to a floating point conversion.

```
>>> round(a.value / 1e9, 6)
946684800.124999
>>> round(b.value / 1e9, 6)
1514764800.125
```

#### Expected Output

```
>>> b.round('us')
Timestamp('2018-01-01 00:00:00.124999')
```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.7.0.final.0
python-bits: 64
OS: Darwin
OS-release: 17.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: None
pip: 18.0
setuptools: 39.0.1
Cython: None
numpy: 1.15.1
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>



Discussion:
This is in https://github.com/pandas-dev/pandas/blob/3285bdc8e6e1b05d84b6cebea66fdd80fbb84400/pandas/_libs/tslibs/timestamps.pyx#L60 if you want to take a look.  There's a comment indicating that float precision has been a problem in the past.
By inspecting the code I see further problems...
E.g. <https://github.com/pandas-dev/pandas/blob/3285bdc8e6e1b05d84b6cebea66fdd80fbb84400/pandas/_libs/tslibs/timestamps.pyx#L83-L86> assumes that `unit` is a divisor of `1_000_000`.

Therefore
```python
>>> a.round('27ns')
Timestamp('2000-01-01 00:00:00.124999351')
```
is wrong. In fact I would expect `a.round('27ns').value  % 27 == 0` and *not* 
`a.round('27ns').value % 1_000_000 % 27 == 0` as per the current implementation.

Of course the semantics of  `a.round('27ns')` are questionable, but the current implementation appears at least arbitrary.

~I don't know if floating point arithmetic is used because it is considered *faster* than integer arithmetic, but it seems quite complex to retain full precision in all possible edge cases.~

I've just noticed that one costly bit of integer arithmetic is already present:
<https://github.com/pandas-dev/pandas/blob/3285bdc8e6e1b05d84b6cebea66fdd80fbb84400/pandas/_libs/tslibs/timestamps.pyx#L78>
In fact once you compute `values % unit` floor and ceil are trivial:
```python
floor = lambda values, unit: values - ( values % unit)
ceil = lambda values, unit: values + (-values % unit)
```
plus some code for handling under/overflow.

Rounding is a little bit more complicated, mainly because we first need to exactly define rounding semantics. If the maintainers confirm the present behaviour as a bug, (and not a feature), I can submit a PR.
same problem for `Timedelta`:
<https://github.com/pandas-dev/pandas/blob/3285bdc8e6e1b05d84b6cebea66fdd80fbb84400/pandas/_libs/tslibs/timedeltas.pyx#L1161-L1167>
but of course precision loss is very unlikely in real world applications.
This issue is exactly the same as #19206, only showing up on different edge case. Taking into account that the current code also patches #21262, I would suggest a entirely new implementation based on `int64` and to entirely drop floating point.

A similar rounding bug due to floating point arithmetic is numpy/numpy#11881
I have a proof of concept implementation in  miccoli/pandas@1adfd41 

Before issuing a PR I would like to receive some comments on following aspects:

- [ ] pure `int64` implementation (drops floating point and recourse to `np.round`, `np.floor`, `np.ceil`
- [ ] tie breaking rule for halfway numbers
   - I propose to break compatibility with the current floating point base rule, round to even, and opt in favour of a round towards -∞
- [ ] should also time deltas use the same `int64` implementation?
- [ ] general comments on the coding style.

Current code seems to me badly broken, but things concerning rounding are really complex, therefore a review of the proposed new semantics is necessary.
so does your soln break any tests?

seems a lot simpler though pls don’t change actual rounding semantics
follow numpy exactly

whether using numpy functions or not doesn’t matter

Yes, a few tests are broken, e.g.
<pre>
_____________ TestTimestampUnaryOps.test_round_frequencies[20130104 12:00:00-D-20130105] _____________

self = <pandas.tests.scalar.timestamp.test_unary_ops.TestTimestampUnaryOps object at 0x133f9fd30>
timestamp = '20130104 12:00:00', freq = 'D', expected = Timestamp('2013-01-05 00:00:00')

    @pytest.mark.parametrize('timestamp, freq, expected', [
        ('20130101 09:10:11', 'D', '20130101'),
        ('20130101 19:10:11', 'D', '20130102'),
        ('20130201 12:00:00', 'D', '20130202'),
        ('20130104 12:00:00', 'D', '20130105'),
        ('2000-01-05 05:09:15.13', 'D', '2000-01-05 00:00:00'),
        ('2000-01-05 05:09:15.13', 'H', '2000-01-05 05:00:00'),
        ('2000-01-05 05:09:15.13', 'S', '2000-01-05 05:09:15')
    ])
    def test_round_frequencies(self, timestamp, freq, expected):
        dt = Timestamp(timestamp)
        result = dt.round(freq)
        expected = Timestamp(expected)
>       assert result == expected
E       AssertionError: assert Timestamp('2013-01-04 00:00:00') == Timestamp('2013-01-05 00:00:00')

pandas/tests/scalar/timestamp/test_unary_ops.py:35: AssertionError
</pre>

The reason for dropping default `numpy` rounding mode (which follows IEEE754) is that IMHO "round to even" does not makes much sense in the `Timestamp` context.
- Current implementation:
```python
>>> pd.__version__
'0.23.4'
>>> pd.to_datetime('2018-01-01 12:00:00').round('1d')
Timestamp('2018-01-01 00:00:00')
>>> pd.to_datetime('2018-01-02 12:00:00').round('1d')
Timestamp('2018-01-03 00:00:00')
>>> pd.to_datetime('2018-01-03 12:00:00').round('1d')
Timestamp('2018-01-03 00:00:00')
>>> pd.to_datetime('2019-01-01 12:00:00').round('1d')
Timestamp('2019-01-02 00:00:00')
>>> pd.to_datetime('2019-01-02 12:00:00').round('1d')
Timestamp('2019-01-02 00:00:00')
>>> pd.to_datetime('2019-01-03 12:00:00').round('1d')
Timestamp('2019-01-04 00:00:00')
>>> d = pd.DatetimeIndex(start='2018-01-01 00:00:00', end='2018-12-31 23:59:59.999999999', freq='1h')
>>> (d.round('1d') - d).values.sum() / np.timedelta64(1, 'D')
-0.5
```
- Proposed implementation:
```python
>>> pd.__version__
'0.24.0.dev0+564.g1adfd4155'
>>> pd.to_datetime('2018-01-01 12:00:00').round('1d')
Timestamp('2018-01-01 00:00:00')
>>> pd.to_datetime('2018-01-02 12:00:00').round('1d')
Timestamp('2018-01-02 00:00:00')
>>> pd.to_datetime('2018-01-03 12:00:00').round('1d')
Timestamp('2018-01-03 00:00:00')
>>> pd.to_datetime('2019-01-01 12:00:00').round('1d')
Timestamp('2019-01-01 00:00:00')
>>> pd.to_datetime('2019-01-02 12:00:00').round('1d')
Timestamp('2019-01-02 00:00:00')
>>> pd.to_datetime('2019-01-03 12:00:00').round('1d')
Timestamp('2019-01-03 00:00:00')
>>> d = pd.DatetimeIndex(start='2018-01-01 00:00:00', end='2018-12-31 23:59:59.999999999', freq='1h')
>>> (d.round('1d') - d).values.sum() / np.timedelta64(1, 'D')
-182.5
```

Here you see that noon is rounded up or down to the midnight of the same or the following day depending on whether the day number (counted starting from the unix epoch '1970-01-01') is odd or even. On the contrary with my proposed implementation noon is always rounded down to midnight of the same day, which is more intuitive.

Of course a rigorous assessment of merits or demerits of both rounding modes requires an analysis of all the possible settings in which `Datetime.round` will be used. I would tend to favour an intuitive and predictable result over the reduction of the bias toward -∞. It is also debatable if in this context bias towards even is preferable with respect to bias toward -∞, but frankly I do not have an answer here, it is (for now) just a matter of taste.

I understand the need for not breaking compatibility with previous behaviour, so I will implement also the `RoundTo.NEAREST_HALF_EVEN` mode, to make a more informed decision possible.

Resolving patch:
diff --git a/pandas/core/indexes/datetimelike.py b/pandas/core/indexes/datetimelike.py
--- a/pandas/core/indexes/datetimelike.py
+++ b/pandas/core/indexes/datetimelike.py
@@ -11,7 +11,7 @@
 import numpy as np
 
 from pandas._libs import lib, iNaT, NaT
-from pandas._libs.tslibs.timestamps import round_ns
+from pandas._libs.tslibs.timestamps import round_nsint64, RoundTo
 
 from pandas.core.dtypes.common import (
     ensure_int64,
@@ -180,10 +180,10 @@ class TimelikeOps(object):
         """
     )
 
-    def _round(self, freq, rounder, ambiguous):
+    def _round(self, freq, mode, ambiguous):
         # round the local times
         values = _ensure_datetimelike_to_i8(self)
-        result = round_ns(values, rounder, freq)
+        result = round_nsint64(values, mode, freq)
         result = self._maybe_mask_results(result, fill_value=NaT)
 
         attribs = self._get_attributes_dict()
@@ -197,15 +197,15 @@ def _round(self, freq, rounder, ambiguous):
 
     @Appender((_round_doc + _round_example).format(op="round"))
     def round(self, freq, ambiguous='raise'):
-        return self._round(freq, np.round, ambiguous)
+        return self._round(freq, RoundTo.NEAREST_HALF_EVEN, ambiguous)
 
     @Appender((_round_doc + _floor_example).format(op="floor"))
     def floor(self, freq, ambiguous='raise'):
-        return self._round(freq, np.floor, ambiguous)
+        return self._round(freq, RoundTo.MINUS_INFTY, ambiguous)
 
     @Appender((_round_doc + _ceil_example).format(op="ceil"))
     def ceil(self, freq, ambiguous='raise'):
-        return self._round(freq, np.ceil, ambiguous)
+        return self._round(freq, RoundTo.PLUS_INFTY, ambiguous)
 
 
 class DatetimeIndexOpsMixin(DatetimeLikeArrayMixin):


Test patch:
diff --git a/pandas/tests/indexes/datetimes/test_scalar_compat.py b/pandas/tests/indexes/datetimes/test_scalar_compat.py
--- a/pandas/tests/indexes/datetimes/test_scalar_compat.py
+++ b/pandas/tests/indexes/datetimes/test_scalar_compat.py
@@ -11,6 +11,7 @@
 import pandas as pd
 
 from pandas import date_range, Timestamp, DatetimeIndex
+from pandas.tseries.frequencies import to_offset
 
 
 class TestDatetimeIndexOps(object):
@@ -124,7 +125,7 @@ def test_round(self, tz_naive_fixture):
         expected = DatetimeIndex(['2016-10-17 12:00:00.001501030'])
         tm.assert_index_equal(result, expected)
 
-        with tm.assert_produces_warning():
+        with tm.assert_produces_warning(False):
             ts = '2016-10-17 12:00:00.001501031'
             DatetimeIndex([ts]).round('1010ns')
 
@@ -169,6 +170,46 @@ def test_ceil_floor_edge(self, test_input, rounder, freq, expected):
         expected = DatetimeIndex(list(expected))
         assert expected.equals(result)
 
+    @pytest.mark.parametrize('start, index_freq, periods', [
+        ('2018-01-01', '12H', 25),
+        ('2018-01-01 0:0:0.124999', '1ns', 1000),
+    ])
+    @pytest.mark.parametrize('round_freq', [
+        '2ns', '3ns', '4ns', '5ns', '6ns', '7ns',
+        '250ns', '500ns', '750ns',
+        '1us', '19us', '250us', '500us', '750us',
+        '1s', '2s', '3s',
+        '12H', '1D',
+    ])
+    def test_round_int64(self, start, index_freq, periods, round_freq):
+        dt = DatetimeIndex(start=start, freq=index_freq, periods=periods)
+        unit = to_offset(round_freq).nanos
+
+        # test floor
+        result = dt.floor(round_freq)
+        diff = dt.asi8 - result.asi8
+        mod = result.asi8 % unit
+        assert (mod == 0).all(), "floor not a {} multiple".format(round_freq)
+        assert (0 <= diff).all() and (diff < unit).all(), "floor error"
+
+        # test ceil
+        result = dt.ceil(round_freq)
+        diff = result.asi8 - dt.asi8
+        mod = result.asi8 % unit
+        assert (mod == 0).all(), "ceil not a {} multiple".format(round_freq)
+        assert (0 <= diff).all() and (diff < unit).all(), "ceil error"
+
+        # test round
+        result = dt.round(round_freq)
+        diff = abs(result.asi8 - dt.asi8)
+        mod = result.asi8 % unit
+        assert (mod == 0).all(), "round not a {} multiple".format(round_freq)
+        assert (diff <= unit // 2).all(), "round error"
+        if unit % 2 == 0:
+            assert (
+                result.asi8[diff == unit // 2] % 2 == 0
+            ).all(), "round half to even error"
+
     # ----------------------------------------------------------------
     # DatetimeIndex.normalize
 
diff --git a/pandas/tests/scalar/timestamp/test_unary_ops.py b/pandas/tests/scalar/timestamp/test_unary_ops.py
--- a/pandas/tests/scalar/timestamp/test_unary_ops.py
+++ b/pandas/tests/scalar/timestamp/test_unary_ops.py
@@ -13,6 +13,7 @@
 from pandas._libs.tslibs import conversion
 from pandas._libs.tslibs.frequencies import INVALID_FREQ_ERR_MSG
 from pandas import Timestamp, NaT
+from pandas.tseries.frequencies import to_offset
 
 
 class TestTimestampUnaryOps(object):
@@ -70,7 +71,7 @@ def test_round_subsecond(self):
         assert result == expected
 
     def test_round_nonstandard_freq(self):
-        with tm.assert_produces_warning():
+        with tm.assert_produces_warning(False):
             Timestamp('2016-10-17 12:00:00.001501031').round('1010ns')
 
     def test_round_invalid_arg(self):
@@ -154,6 +155,46 @@ def test_round_dst_border(self, method):
         with pytest.raises(pytz.AmbiguousTimeError):
             getattr(ts, method)('H', ambiguous='raise')
 
+    @pytest.mark.parametrize('timestamp', [
+        '2018-01-01 0:0:0.124999360',
+        '2018-01-01 0:0:0.125000367',
+        '2018-01-01 0:0:0.125500',
+        '2018-01-01 0:0:0.126500',
+        '2018-01-01 12:00:00',
+        '2019-01-01 12:00:00',
+    ])
+    @pytest.mark.parametrize('freq', [
+        '2ns', '3ns', '4ns', '5ns', '6ns', '7ns',
+        '250ns', '500ns', '750ns',
+        '1us', '19us', '250us', '500us', '750us',
+        '1s', '2s', '3s',
+        '1D',
+    ])
+    def test_round_int64(self, timestamp, freq):
+        """check that all rounding modes are accurate to int64 precision
+           see GH#22591
+        """
+        dt = Timestamp(timestamp)
+        unit = to_offset(freq).nanos
+
+        # test floor
+        result = dt.floor(freq)
+        assert result.value % unit == 0, "floor not a {} multiple".format(freq)
+        assert 0 <= dt.value - result.value < unit, "floor error"
+
+        # test ceil
+        result = dt.ceil(freq)
+        assert result.value % unit == 0, "ceil not a {} multiple".format(freq)
+        assert 0 <= result.value - dt.value < unit, "ceil error"
+
+        # test round
+        result = dt.round(freq)
+        assert result.value % unit == 0, "round not a {} multiple".format(freq)
+        assert abs(result.value - dt.value) <= unit // 2, "round error"
+        if unit % 2 == 0 and abs(result.value - dt.value) == unit // 2:
+            # round half to even
+            assert result.value // unit % 2 == 0, "round half to even error"
+
     # --------------------------------------------------------------
     # Timestamp.replace