Repository: pandas-dev/pandas

Issue:
BUG: .date and timezones
from [SO](http://stackoverflow.com/questions/34078316/tz-aware-datetime-series-producing-utc-based-date-output-in-pandas-series-app/34078346#34078346)

```
In [1]: s = Series(pd.to_datetime([1396670200, 1405733045, 1448248441],unit='s')).dt.tz_localize('UTC').dt.tz_convert('US/Eastern')

In [2]: s
Out[2]: 
0   2014-04-04 23:56:40-04:00
1   2014-07-18 21:24:05-04:00
2   2015-11-22 22:14:01-05:00
dtype: datetime64[ns, US/Eastern]
```

These are correct as they are in the local timezone, but we are not propogating the tz here and just returning a naive date (in local zone)

```
In [3]: s.dt.date
Out[3]: 
0    2014-04-04
1    2014-07-18
2    2015-11-22
dtype: object

In [4]: s.dt.date[0]
Out[4]: datetime.date(2014, 4, 4)
```

These are converted to UTC, then returning a naive date
These should be local & with a timezone attached

```
In [5]: s.apply(lambda x: x.date())
Out[5]: 
0    2014-04-05
1    2014-07-19
2    2015-11-23
dtype: object
```

further, test

`s.apply(lambda x: x)`



Discussion:
Python `datetime.date` is naive so I think that is the right thing.  However, clearly the second one needs to be localized.

This I believe is the same as #11800.  I have the tests ready to go once apply is changed as mentioned there.

@rockg did you have a PR for this?

I have not pushed it...will do so by tomorrow morning.

gr8 thanks 

I thought this would be fixed by #11564 but it is still around.  Was that your understanding?

```
>>> s.apply(lambda x: x.date())
Out[1]: 
0    2014-04-05
1    2014-07-19
2    2015-11-23
dtype: object
```

that was much more focused on timeseltas
this is a separate (smaller issue) I think

All right...I need some direction then.  [This](https://github.com/pydata/pandas/blob/master/pandas/core/series.py#L2240) is the problem spot.  Basically it gets the datetime64s out and converts those into Timestamps all of which are tz-unaware.  One solution is to remove the tz like `tz_localize(None)` and then get values from that (there doesn't seem to be tz_localize on a datetime64 series).  Other thoughts?

oh, that's non-conformant code, maybe something like:

```
        if needs_i8_conversion(values.dtype):
            f = lambda v: i8_boxer(v)
        mapped = lib.map_infer(values, f, convert=convert_dtype)
```

@rockg Did you have something? 
Otherwise will push this to the next release

I think this was actually fixed by some other pull requests (perhaps the referenced issues), but I cannot test for a couple days.  If not fixed, then you can push.

Hmm, not really clear what was actually the issue. This example from @jreback:

```
In [3]: s.dt.date
Out[3]: 
0    2014-04-04
1    2014-07-18
2    2015-11-22
dtype: object
```

Isn't this the correct and expected output? (master still gives the same)

The second issues was the `apply` version:

```
In [5]: s.apply(lambda x: x.date())
Out[5]: 
0    2014-04-05
1    2014-07-19
2    2015-11-23
dtype: object
```

where the UTC dates are given. This seems to be fixed in master:

```
In [15]: pd.__version__
Out[15]: '0.18.1+387.gae4ffac'

In [16]: s.apply(lambda x: x.date())
Out[16]: 
0    2014-04-04
1    2014-07-18
2    2015-11-22
dtype: object
```

Is s equal to [1] above and in US/Eastern?  If so, then master is now correct.  Before it was converting under the hood to UTC and then taking date of that which was wrong. 

Yes, I used the `s` from `[1]` at top of this issue. But note that the `Out[3]` of my previous post was just a copy of the the top post, so it was already like that when this issue was opened. So therefore I am somewhat confused (as this seems the correct output to me)

Yes, [3] is right.  There cannot be timezones attached to date objects.  For me the real issue was that `apply` was not returning the correct date in the local timezone (instead it was using the UTC date).

And that issue is now fixed so I think this can be closed.

I don't think this is fixed
the timestamps are converted to naive in
s.dt.date but should have the original timezone (as datetime.datetime)

I always understood `date` to return python date objects.  `normalize`returns midnight timestamps so I think they are meant to be fundamentally different.

that's right they should be datetime.datetime

WITH a tzinfo

they r missing the tzinfo

But I'm saying they should by python date objects so without a timezone.

Just like if you have a datetime.datetime and call `date` on it, you get the date object back.  There are other ways to get midnight such as `normalize`.  This is a change and I don't really see the purpose to it.

no that doesn't make sense - u have lost the tz for no reason (this is of course for a tz aware series - obviously a tz naive one this is already correct) 

try date on a tz-aware datetime.datetime 

```
>>> ts = pd.Timestamp("2016-1-1", tz="US/Eastern")
>>> ts
Timestamp('2016-01-01 00:00:00-0500', tz='US/Eastern')
>>> ts.date()
datetime.date(2016, 1, 1)
```

And raw datetime:

```
>>> import pytz
>>> from datetime import datetime
>>> d = datetime(2016, 1, 1)
>>> tz = pytz.timezone("US/Eastern")
>>> dtz = tz.localize(d)
>>> dtz
datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'US/Eastern' EST-1 day, 19:00:00 STD>)
>>> dtz.date()
datetime.date(2016, 1, 1)
>>> 
```

so the issue is that we _alway_ have `Timestamps` so we always have `tz`, and thus can preserve it. Ok I guess just need tests to validate that we are returning the _local_ tz date then.

@jreback So you propose that `s.dt.date` does not any longer return `datetime.date` objects, but rather a Timestamp (at midnight) with a tz? 
In any case it is not a bug currently, as for now, `date()` is supposed to give you a date (which by definition no longer has a timezone) and not a datetime/timestamp.

Of course such a change could be discussed, but I agree with @rockg that I also don't see the purpose of this change.

If we do not change it, this issue indeed only needs some tests to confirm the fixed apply behaviour to get closed.

no this is fine

though maybe should doc it somehow/where
naively I would have though we would preserve the tz meta data but apparently .date doesn't (which IMHO is odd)

I'll add tests/doc mention tonight and then we should be good.


Resolving patch:
diff --git a/pandas/tseries/index.py b/pandas/tseries/index.py
--- a/pandas/tseries/index.py
+++ b/pandas/tseries/index.py
@@ -1597,7 +1597,8 @@ def time(self):
     @property
     def date(self):
         """
-        Returns numpy array of datetime.date. The date part of the Timestamps.
+        Returns numpy array of python datetime.date objects (namely, the date
+        part of Timestamps without timezone information).
         """
         return self._maybe_mask_results(_algos.arrmap_object(
             self.asobject.values, lambda x: x.date()))


Test patch:
diff --git a/pandas/tests/series/test_datetime_values.py b/pandas/tests/series/test_datetime_values.py
--- a/pandas/tests/series/test_datetime_values.py
+++ b/pandas/tests/series/test_datetime_values.py
@@ -1,7 +1,7 @@
 # coding=utf-8
 # pylint: disable-msg=E1101,W0612
 
-from datetime import datetime
+from datetime import datetime, date
 
 import numpy as np
 import pandas as pd
@@ -410,3 +410,15 @@ def test_between(self):
         result = s[s.between(s[3], s[17], inclusive=False)]
         expected = s[5:16].dropna()
         assert_series_equal(result, expected)
+
+    def test_date_tz(self):
+        # GH11757
+        rng = pd.DatetimeIndex(['2014-04-04 23:56',
+                                '2014-07-18 21:24',
+                                '2015-11-22 22:14'], tz="US/Eastern")
+        s = Series(rng)
+        expected = Series([date(2014, 4, 4),
+                           date(2014, 7, 18),
+                           date(2015, 11, 22)])
+        assert_series_equal(s.dt.date, expected)
+        assert_series_equal(s.apply(lambda x: x.date()), expected)