Repository: pandas-dev/pandas

Issue:
Rounding valid timestamps near daylight savings jumps should not throw NonExistentTimeError
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd

# valid time rounds incorrectly
pd.Timestamp('2018-03-11 01:59:00-0600', tz='America/Chicago').round(freq='5min')
# throws NonExistentTimeError: 2018-03-11 02:00:00
# should return pd.Timestamp('2018-03-11 03:00:00-0500', tz='America/Chicago')
```
#### Problem description

The timestamp is a valid time, so it should round to another valid timestamp. Specifically, it should round to the nearest 5min frequency valid time, which is the moment after the clock jumps.

#### Expected Output
```pd.Timestamp('2018-03-11 03:00:00-0500', tz='America/Chicago')```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-1070-aws
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: C.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.22.0
pytest: None
pip: 18.1
setuptools: 39.2.0
Cython: None
numpy: 1.14.0
scipy: 1.1.0
pyarrow: 0.9.0
xarray: None
IPython: 6.5.0
sphinx: None
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.6
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: 0.4.0
matplotlib: 2.1.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0.1
sqlalchemy: 1.2.10
pymysql: None
psycopg2: 2.7.5 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

</details>



Discussion:
This occurs after relocalization since rounding happens in local time. After https://github.com/pandas-dev/pandas/pull/22644 is in, we can introduce a `nonexistent` keyword to handle this behavior like we did `ambiguous` in https://github.com/pandas-dev/pandas/pull/22647
Thanks for the quick response! Agree that would solve this, but from a design perspective I don't think I understand why that would be the right way to handle it.

What's the usecase where you are trying to round a valid time and you would WANT to raise a NonExistentTimeError? Or even deeper, what would that mean? If you're working within valid times, rounding should always be an allowed operation (to another valid time), right?
Well operationally rounding `'2018-03-11 01:59:00-0600'` to the closest 5min would yield `'2018-03-11 02:00:00'` which does not exist hence `NonExistentTimeError` could be a valid output.

It is inconvenient that this is the _only_ output though, so giving user the ability to `'raise'`, or replace with `'NaT'` or `'shift'` to 3:00 (which what you want) would be a nice for the user to control instead of pandas dictating what the correct behavior should be. 

Resolving patch:
diff --git a/pandas/core/indexes/datetimelike.py b/pandas/core/indexes/datetimelike.py
--- a/pandas/core/indexes/datetimelike.py
+++ b/pandas/core/indexes/datetimelike.py
@@ -113,6 +113,17 @@ class TimelikeOps(object):
             Only relevant for DatetimeIndex
 
             .. versionadded:: 0.24.0
+        nonexistent : 'shift', 'NaT', default 'raise'
+            A nonexistent time does not exist in a particular timezone
+            where clocks moved forward due to DST.
+
+            - 'shift' will shift the nonexistent time forward to the closest
+              existing time
+            - 'NaT' will return NaT where there are nonexistent times
+            - 'raise' will raise an NonExistentTimeError if there are
+              nonexistent times
+
+            .. versionadded:: 0.24.0
 
         Returns
         -------
@@ -182,7 +193,7 @@ class TimelikeOps(object):
         """
     )
 
-    def _round(self, freq, mode, ambiguous):
+    def _round(self, freq, mode, ambiguous, nonexistent):
         # round the local times
         values = _ensure_datetimelike_to_i8(self)
         result = round_nsint64(values, mode, freq)
@@ -194,20 +205,22 @@ def _round(self, freq, mode, ambiguous):
         if 'tz' in attribs:
             attribs['tz'] = None
         return self._ensure_localized(
-            self._shallow_copy(result, **attribs), ambiguous
+            self._shallow_copy(result, **attribs), ambiguous, nonexistent
         )
 
     @Appender((_round_doc + _round_example).format(op="round"))
-    def round(self, freq, ambiguous='raise'):
-        return self._round(freq, RoundTo.NEAREST_HALF_EVEN, ambiguous)
+    def round(self, freq, ambiguous='raise', nonexistent='raise'):
+        return self._round(
+            freq, RoundTo.NEAREST_HALF_EVEN, ambiguous, nonexistent
+        )
 
     @Appender((_round_doc + _floor_example).format(op="floor"))
-    def floor(self, freq, ambiguous='raise'):
-        return self._round(freq, RoundTo.MINUS_INFTY, ambiguous)
+    def floor(self, freq, ambiguous='raise', nonexistent='raise'):
+        return self._round(freq, RoundTo.MINUS_INFTY, ambiguous, nonexistent)
 
     @Appender((_round_doc + _ceil_example).format(op="ceil"))
-    def ceil(self, freq, ambiguous='raise'):
-        return self._round(freq, RoundTo.PLUS_INFTY, ambiguous)
+    def ceil(self, freq, ambiguous='raise', nonexistent='raise'):
+        return self._round(freq, RoundTo.PLUS_INFTY, ambiguous, nonexistent)
 
 
 class DatetimeIndexOpsMixin(DatetimeLikeArrayMixin):
@@ -278,7 +291,8 @@ def _evaluate_compare(self, other, op):
         except TypeError:
             return result
 
-    def _ensure_localized(self, arg, ambiguous='raise', from_utc=False):
+    def _ensure_localized(self, arg, ambiguous='raise', nonexistent='raise',
+                          from_utc=False):
         """
         ensure that we are re-localized
 
@@ -289,6 +303,7 @@ def _ensure_localized(self, arg, ambiguous='raise', from_utc=False):
         ----------
         arg : DatetimeIndex / i8 ndarray
         ambiguous : str, bool, or bool-ndarray, default 'raise'
+        nonexistent : str, default 'raise'
         from_utc : bool, default False
             If True, localize the i8 ndarray to UTC first before converting to
             the appropriate tz. If False, localize directly to the tz.
@@ -305,7 +320,9 @@ def _ensure_localized(self, arg, ambiguous='raise', from_utc=False):
             if from_utc:
                 arg = arg.tz_localize('UTC').tz_convert(self.tz)
             else:
-                arg = arg.tz_localize(self.tz, ambiguous=ambiguous)
+                arg = arg.tz_localize(
+                    self.tz, ambiguous=ambiguous, nonexistent=nonexistent
+                )
         return arg
 
     def _box_values_as_index(self):


Test patch:
diff --git a/pandas/tests/scalar/timestamp/test_unary_ops.py b/pandas/tests/scalar/timestamp/test_unary_ops.py
--- a/pandas/tests/scalar/timestamp/test_unary_ops.py
+++ b/pandas/tests/scalar/timestamp/test_unary_ops.py
@@ -134,8 +134,8 @@ def test_floor(self):
         assert result == expected
 
     @pytest.mark.parametrize('method', ['ceil', 'round', 'floor'])
-    def test_round_dst_border(self, method):
-        # GH 18946 round near DST
+    def test_round_dst_border_ambiguous(self, method):
+        # GH 18946 round near "fall back" DST
         ts = Timestamp('2017-10-29 00:00:00', tz='UTC').tz_convert(
             'Europe/Madrid'
         )
@@ -155,6 +155,24 @@ def test_round_dst_border(self, method):
         with pytest.raises(pytz.AmbiguousTimeError):
             getattr(ts, method)('H', ambiguous='raise')
 
+    @pytest.mark.parametrize('method, ts_str, freq', [
+        ['ceil', '2018-03-11 01:59:00-0600', '5min'],
+        ['round', '2018-03-11 01:59:00-0600', '5min'],
+        ['floor', '2018-03-11 03:01:00-0500', '2H']])
+    def test_round_dst_border_nonexistent(self, method, ts_str, freq):
+        # GH 23324 round near "spring forward" DST
+        ts = Timestamp(ts_str, tz='America/Chicago')
+        result = getattr(ts, method)(freq, nonexistent='shift')
+        expected = Timestamp('2018-03-11 03:00:00', tz='America/Chicago')
+        assert result == expected
+
+        result = getattr(ts, method)(freq, nonexistent='NaT')
+        assert result is NaT
+
+        with pytest.raises(pytz.NonExistentTimeError,
+                           message='2018-03-11 02:00:00'):
+            getattr(ts, method)(freq, nonexistent='raise')
+
     @pytest.mark.parametrize('timestamp', [
         '2018-01-01 0:0:0.124999360',
         '2018-01-01 0:0:0.125000367',
diff --git a/pandas/tests/series/test_datetime_values.py b/pandas/tests/series/test_datetime_values.py
--- a/pandas/tests/series/test_datetime_values.py
+++ b/pandas/tests/series/test_datetime_values.py
@@ -253,7 +253,7 @@ def test_dt_round_tz(self):
 
     @pytest.mark.parametrize('method', ['ceil', 'round', 'floor'])
     def test_dt_round_tz_ambiguous(self, method):
-        # GH 18946 round near DST
+        # GH 18946 round near "fall back" DST
         df1 = pd.DataFrame([
             pd.to_datetime('2017-10-29 02:00:00+02:00', utc=True),
             pd.to_datetime('2017-10-29 02:00:00+01:00', utc=True),
@@ -282,6 +282,27 @@ def test_dt_round_tz_ambiguous(self, method):
         with pytest.raises(pytz.AmbiguousTimeError):
             getattr(df1.date.dt, method)('H', ambiguous='raise')
 
+    @pytest.mark.parametrize('method, ts_str, freq', [
+        ['ceil', '2018-03-11 01:59:00-0600', '5min'],
+        ['round', '2018-03-11 01:59:00-0600', '5min'],
+        ['floor', '2018-03-11 03:01:00-0500', '2H']])
+    def test_dt_round_tz_nonexistent(self, method, ts_str, freq):
+        # GH 23324 round near "spring forward" DST
+        s = Series([pd.Timestamp(ts_str, tz='America/Chicago')])
+        result = getattr(s.dt, method)(freq, nonexistent='shift')
+        expected = Series(
+            [pd.Timestamp('2018-03-11 03:00:00', tz='America/Chicago')]
+        )
+        tm.assert_series_equal(result, expected)
+
+        result = getattr(s.dt, method)(freq, nonexistent='NaT')
+        expected = Series([pd.NaT]).dt.tz_localize(result.dt.tz)
+        tm.assert_series_equal(result, expected)
+
+        with pytest.raises(pytz.NonExistentTimeError,
+                           message='2018-03-11 02:00:00'):
+            getattr(s.dt, method)(freq, nonexistent='raise')
+
     def test_dt_namespace_accessor_categorical(self):
         # GH 19468
         dti = DatetimeIndex(['20171111', '20181212']).repeat(2)