Repository: pandas-dev/pandas
Pull request: TST: add test for non UTC datetime split #14042
- [x] closes #14042
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Problem statement:
numpy.split on non-UTC, tz-aware data undergoes UTC roundtrip
#### Code Sample, a copy-pastable example if possible

```
import pandas as pd
import numpy as np
indices = pd.date_range('2016-01-01 00:00:00+0200', freq='S', periods=10)
np.split(indices, indices_or_sections=[])
Out:
[DatetimeIndex(['2015-12-31 20:00:00+02:00', '2015-12-31 20:00:01+02:00',
               '2015-12-31 20:00:02+02:00', '2015-12-31 20:00:03+02:00',
               '2015-12-31 20:00:04+02:00', '2015-12-31 20:00:05+02:00',
               '2015-12-31 20:00:06+02:00', '2015-12-31 20:00:07+02:00',
               '2015-12-31 20:00:08+02:00', '2015-12-31 20:00:09+02:00'],
              dtype='datetime64[ns, pytz.FixedOffset(120)]', freq='S')]
```
#### Expected Output

```
[DatetimeIndex(['2016-01-01 00:00:00+02:00', '2016-01-01 00:00:01+02:00',
               '2016-01-01 00:00:02+02:00', '2016-01-01 00:00:03+02:00',
               '2016-01-01 00:00:04+02:00', '2016-01-01 00:00:05+02:00',
               '2016-01-01 00:00:06+02:00', '2016-01-01 00:00:07+02:00',
               '2016-01-01 00:00:08+02:00', '2016-01-01 00:00:09+02:00'],
              dtype='datetime64[ns, pytz.FixedOffset(120)]', freq='S')]
```
#### output of `pd.show_versions()`

```
In [16]: pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Linux
OS-release: 4.2.0-38-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.2
setuptools: 25.2.0
Cython: 0.24.1
numpy: 1.11.1
scipy: 0.18.0
statsmodels: None
xarray: None
IPython: 1.2.1
sphinx: None
patsy: None
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: None
tables: 3.1.1
numexpr: 2.6.1
matplotlib: 1.3.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.999
httplib2: 0.8
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
boto: 2.42.0
pandas_datareader: None
```

Hints from issue discussion:
This is fixed in master, I dont know exactly which issue, @sinhrks fixed `__array__` though IIRC to propogate meta-data.

```
In [46]: indices = pd.date_range('2016-01-01 00:00:00+0200', freq='S', periods=10)
    ...: np.split(indices, indices_or_sections=[])
    ...: 
Out[46]: 
[DatetimeIndex(['2015-12-31 20:00:00+02:00', '2015-12-31 20:00:01+02:00', '2015-12-31 20:00:02+02:00', '2015-12-31 20:00:03+02:00', '2015-12-31 20:00:04+02:00', '2015-12-31 20:00:05+02:00',
                '2015-12-31 20:00:06+02:00', '2015-12-31 20:00:07+02:00', '2015-12-31 20:00:08+02:00', '2015-12-31 20:00:09+02:00'],
               dtype='datetime64[ns, pytz.FixedOffset(120)]', freq='S')]
```

If you'd like to do a PR with tests of this would be great.

@jreback the output you show is the incorrect output, so I don't think this is fixed. Seems there goes something wrong in `np.swapaxes(indices, 0, 0)`

oh, looks like it underwent a UTC round-trip then. I c. ok.

Ok, we can mark it, though @isacarnekvist we have no need for `np.split` as `.groupby` effectively does almost all partitioning (yes we have discussions of actually having a `.partition`, but that's another issue).

Looks fixed on master. Could use a test

```
In [10]: import pandas as pd
    ...: import numpy as np
    ...: indices = pd.date_range('2016-01-01 00:00:00+0200', freq='S', periods=10)
    ...: np.split(indices, indices_or_sections=[])
Out[10]:
[DatetimeIndex(['2016-01-01 00:00:00+02:00', '2016-01-01 00:00:01+02:00',
                '2016-01-01 00:00:02+02:00', '2016-01-01 00:00:03+02:00',
                '2016-01-01 00:00:04+02:00', '2016-01-01 00:00:05+02:00',
                '2016-01-01 00:00:06+02:00', '2016-01-01 00:00:07+02:00',
                '2016-01-01 00:00:08+02:00', '2016-01-01 00:00:09+02:00'],
               dtype='datetime64[ns, pytz.FixedOffset(120)]', freq='S')]

In [11]: pd.__version__
Out[11]: '1.0.0rc0+212.gca3bfcc54'
```

Commit message: TST: add test for non UTC datetime split

Patch to review:
diff --git a/pandas/tests/indexes/datetimes/test_datetime.py b/pandas/tests/indexes/datetimes/test_datetime.py
index 6217f225d496e..d61f6c5a8d19b 100644
--- a/pandas/tests/indexes/datetimes/test_datetime.py
+++ b/pandas/tests/indexes/datetimes/test_datetime.py
@@ -425,3 +425,22 @@ def test_index_map(self, name):
             ((2018,), range(1, 7)), names=[name, name]
         )
         tm.assert_index_equal(index, exp_index)
+
+    def test_split_non_utc(self):
+        indices = pd.date_range("2016-01-01 00:00:00+0200", freq="S", periods=10)
+        split_indices = np.split(indices, indices_or_sections=[])
+        valid_indices = DatetimeIndex(
+            [
+                "2016-01-01 00:00:00+02:00",
+                "2016-01-01 00:00:01+02:00",
+                "2016-01-01 00:00:02+02:00",
+                "2016-01-01 00:00:03+02:00",
+                "2016-01-01 00:00:04+02:00",
+                "2016-01-01 00:00:05+02:00",
+                "2016-01-01 00:00:06+02:00",
+                "2016-01-01 00:00:07+02:00",
+                "2016-01-01 00:00:08+02:00",
+                "2016-01-01 00:00:09+02:00",
+            ]
+        )
+        tm.assert_index_equal(split_indices[0], valid_indices)


Code review comments:
diff_hunk: @@ -425,3 +425,22 @@ def test_index_map(self, name):
             ((2018,), range(1, 7)), names=[name, name]
         )
         tm.assert_index_equal(index, exp_index)
+
+    def test_split_non_utc(self):
+        indices = pd.date_range("2016-01-01 00:00:00+0200", freq="S", periods=10)
+        split_indices = np.split(indices, indices_or_sections=[])
+        valid_indices = DatetimeIndex(
+            [
+                "2016-01-01 00:00:00+02:00",
+                "2016-01-01 00:00:01+02:00",
+                "2016-01-01 00:00:02+02:00",
+                "2016-01-01 00:00:03+02:00",
+                "2016-01-01 00:00:04+02:00",
+                "2016-01-01 00:00:05+02:00",
+                "2016-01-01 00:00:06+02:00",
+                "2016-01-01 00:00:07+02:00",
+                "2016-01-01 00:00:08+02:00",
+                "2016-01-01 00:00:09+02:00",
+            ]
+        )
path: pandas/tests/indexes/datetimes/test_datetime.py
text: @user1:
this is the same as `indices` but without a a freq, right?  if so, can you construct it as

```
expected = index.copy()
expected._set_freq(None)
```

@author:
Yes, thanks

diff_hunk: @@ -425,3 +425,22 @@ def test_index_map(self, name):
             ((2018,), range(1, 7)), names=[name, name]
         )
         tm.assert_index_equal(index, exp_index)
+
+    def test_split_non_utc(self):
+        indices = pd.date_range("2016-01-01 00:00:00+0200", freq="S", periods=10)
+        split_indices = np.split(indices, indices_or_sections=[])
path: pandas/tests/indexes/datetimes/test_datetime.py
text: @user1:
Could you name these variables `result` and `expected` so the comparison below can be

`tm.assert_index_equal(result, expected)`

@user2:
are there enough functions like np.split to merit a file like tests.indexes.datetimes.test_numpy_compat?

@author:
@user1 right, done.

@user2 np.split is the only function I know for which such an issue occurred. But this is my first PR for this project so maybe there are others that I haven't noticed.