Repository: pandas-dev/pandas
Pull request: BUG: Using DatetimeIndex.date with timezone returns incorrect date #2…

- [ ] closes #21230, closes https://github.com/pandas-dev/pandas/issues/21267
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Added 2 tests for DatetimeIndex.date in tests/indexes/datetimes/test_datetime.py. Please let me know if I should move them somewhere else.

Problem statement:
Datetime.index.date returns incorrect date post upgrade to version 0.23
#### Code Sample, a copy-pastable example if possible

```python
df1 = pd.DataFrame(data=[24, 25],
                   index=pd.DatetimeIndex(['2013-01-24 15:01:00+01:00',
                                           '2013-01-25 15:01:00+01:00'],
                                          dtype='datetime64[ns, CET]',
                                          name='Date', freq=None))
print(df1.index.date)
```
the above code prints:
[datetime.date(2013, 1, 23) datetime.date(2013, 1, 24)]

#### Problem description
Datetime.index.date does return incorrect dates. The behavious worked until version 0.22, and seems to be incorrect post upgrade to version 0.23

#### Expected Output
`[datetime.date(2013, 1, 24) datetime.date(2013, 1, 25)]`

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.5.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None
pandas: 0.23.0
pytest: 3.5.1
pip: 10.0.1
setuptools: 39.1.0
Cython: None
numpy: 1.12.1
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 6.4.0
sphinx: 1.7.4
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.2.2
openpyxl: None
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>

Hints from issue discussion:
Looks like a timezone issue since this works correctly without a timezone dtype. Investigation and PR's welcome!

```
In [7]: d = pd.DatetimeIndex(['2013-01-24 15:01:00+01:00','2013-01-25 15:01:00+01:00'])

In [8]: d.date
Out[8]:
array([datetime.date(2013, 1, 24), datetime.date(2013, 1, 25)],
      dtype=object)

In [9]: d.tz_localize('CET').date
Out[9]:
array([datetime.date(2013, 1, 23), datetime.date(2013, 1, 24)],
      dtype=object)
```
Hmm is this possibly on purpose?
```
import pandas as pd

df1 = pd.DataFrame(data=[24, 25],
                   index=pd.DatetimeIndex(['2013-01-24 15:01:00+01:00',
                                           '2013-01-25 15:01:00+01:00'],
                                          dtype='datetime64[ns, CET]',
                                          name='Date', freq=None))
print(type(df1.index))
```
Prints 

<class 'pandas.core.indexes.datetimes.DatetimeIndex'>

Looking at the index method of DatetimeIndex : 
https://github.com/pandas-dev/pandas/blob/master/pandas/core/indexes/datetimes.py#L2037
```
    @property
    def date(self):
        """
        Returns numpy array of python datetime.date objects (namely, the date
        part of Timestamps without timezone information).
        """
        return libts.ints_to_pydatetime(self.normalize().asi8, box="date")
```
So here the datetime.date obj is intentionally being returned without the timezone information.



I suppose this is due to https://github.com/pandas-dev/pandas/pull/18163, which improved the performance of the `.date` accessor, but probably forgot to deal with timezones.

Before it relied on the `date()` method of the individual timestamp objects, which is still correct: 

```
In [19]: df1.index[0]
Out[19]: Timestamp('2013-01-24 14:01:00+0100', tz='CET')

In [20]: df1.index[0].date()
Out[20]: datetime.date(2013, 1, 24)
```
cc @tmnhat2001 welcome to take a look if you would have time.

Commit message: BUG: Using DatetimeIndex.date with timezone returns incorrect date #21230

Patch to review:
diff --git a/doc/source/whatsnew/v0.23.1.txt b/doc/source/whatsnew/v0.23.1.txt
index f2bc81eea186b..b263cf6993b03 100644
--- a/doc/source/whatsnew/v0.23.1.txt
+++ b/doc/source/whatsnew/v0.23.1.txt
@@ -85,7 +85,7 @@ Indexing
 - Bug in :meth:`Series.reset_index` where appropriate error was not raised with an invalid level name (:issue:`20925`)
 - Bug in :func:`interval_range` when ``start``/``periods`` or ``end``/``periods`` are specified with float ``start`` or ``end`` (:issue:`21161`)
 - Bug in :meth:`MultiIndex.set_names` where error raised for a ``MultiIndex`` with ``nlevels == 1`` (:issue:`21149`)
--
+- Bug in :attr:`DatetimeIndex.date` where an incorrect date is returned when the input date has a timezone (:issue:`21230`)
 
 I/O
 ^^^
diff --git a/pandas/core/indexes/datetimes.py b/pandas/core/indexes/datetimes.py
index 83950f1d71633..1b18b264749e5 100644
--- a/pandas/core/indexes/datetimes.py
+++ b/pandas/core/indexes/datetimes.py
@@ -2040,7 +2040,7 @@ def date(self):
         Returns numpy array of python datetime.date objects (namely, the date
         part of Timestamps without timezone information).
         """
-        return libts.ints_to_pydatetime(self.normalize().asi8, box="date")
+        return libts.ints_to_pydatetime(self.asi8, box="date")
 
     def normalize(self):
         """
diff --git a/pandas/tests/indexes/datetimes/test_datetime.py b/pandas/tests/indexes/datetimes/test_datetime.py
index 1a5f12103595c..aa4c81b0648f6 100644
--- a/pandas/tests/indexes/datetimes/test_datetime.py
+++ b/pandas/tests/indexes/datetimes/test_datetime.py
@@ -27,6 +27,21 @@ def test_roundtrip_pickle_with_tz(self):
         unpickled = tm.round_trip_pickle(index)
         tm.assert_index_equal(index, unpickled)
 
+    def test_date_accessor_with_tz(self):
+        # GH 21230
+        from datetime import date
+        index = DatetimeIndex(['2013-01-24 15:01:00+01:00'],
+                              dtype='datetime64[ns, CET]', freq=None)
+
+        assert index.date == np.array(date(2013, 1, 24))
+
+    def test_date_accessor_without_tz(self):
+        # GH 21230
+        from datetime import date
+        index = DatetimeIndex(['2013-01-24 15:01:00+01:00'], freq=None)
+
+        assert index.date == np.array(date(2013, 1, 24))
+
     def test_reindex_preserves_tz_if_target_is_empty_list_or_array(self):
         # GH7774
         index = date_range('20130101', periods=3, tz='US/Eastern')


Code review comments:
diff_hunk: @@ -2040,7 +2040,7 @@ def date(self):
         Returns numpy array of python datetime.date objects (namely, the date
         part of Timestamps without timezone information).
         """
-        return libts.ints_to_pydatetime(self.normalize().asi8, box="date")
+        return libts.ints_to_pydatetime(self.asi8, box="date")
path: pandas/core/indexes/datetimes.py
text: @user1:
I _think_ this needs to be `self._local_timestamps()` instead of `self.asi8` since those are essentially epoch (UTC) timestamps and we want to return local values.

@author:
Thanks for the suggestion. That might be it.