Repository: pandas-dev/pandas

Issue:
Resample discards timezone information
It appears that resample is now dropping timezone information on the index. Is this expected?

```
a = pp.Series(index=pp.DatetimeIndex(['2013-01-01 06:00', '2013-01-01 07:00', '2013-01-02 06:00'],
                                     tz=pytz.timezone('America/Los_Angeles')),
              data=[1, 2, 3])
b = a.resample('D', how='max')
print repr(a.index.tz)
print repr(b.index.tz)
print a.index.tz == b.index.tz
```

produces the following in 0.16.1. Notably the timezones for the original series and the resampled series are the same.

```
<DstTzInfo 'America/Los_Angeles' LMT-1 day, 16:07:00 STD>
<DstTzInfo 'America/Los_Angeles' LMT-1 day, 16:07:00 STD>
True
```

However that's not the case for 0.18.1. Caveat is that we're using the new resample syntax: `b = a.resample('D').max()`

```
<DstTzInfo 'America/Los_Angeles' LMT-1 day, 16:07:00 STD>
<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>
False
```



Discussion:
rewriting your example like how we test

```
In [10]: s = pd.Series(index=pd.DatetimeIndex(['2013-01-01 06:00', '2013-01-01 07:00', '2013-01-02 06:00'],
                                     tz='America/Los_Angeles'),
              data=[1, 2, 3])

In [11]: s
Out[11]: 
2013-01-01 06:00:00-08:00    1
2013-01-01 07:00:00-08:00    2
2013-01-02 06:00:00-08:00    3
dtype: int64

In [12]: s.index
Out[12]: DatetimeIndex(['2013-01-01 06:00:00-08:00', '2013-01-01 07:00:00-08:00', '2013-01-02 06:00:00-08:00'], dtype='datetime64[ns, America/Los_Angeles]', freq=None)

In [13]: s.resample('D').max()
Out[13]: 
2013-01-01 00:00:00-08:00    2
2013-01-02 00:00:00-08:00    3
Freq: D, dtype: int64

In [14]: s.resample('D').max().index
Out[14]: DatetimeIndex(['2013-01-01 00:00:00-08:00', '2013-01-02 00:00:00-08:00'], dtype='datetime64[ns, America/Los_Angeles]', freq='D')
```

datetime w/tz and resampling are not tested very much. pull-requests to do more of this are welcome (the fix is very straightforward).

Thanks for the quick look!

Note that your re-written example doesn't display the problem. You need to look at `repr(...index.tz)` to see the difference (`LMT-1 day, 16:07:00 STD` vs `PST-1 day, 16:00:00 STD`)

have a look again at [12] vs [14]. the time changed. (look t the hour). the issue is the re-localization is off. The actual tz is correct.

I'm going to attempt fixing this one
Wouldn't we expect the index values to be at midnight on each day given that we're aggregating up to the day level?  If we run the code above without the timezones we get a similar output:

```
>>> s_notz = pd.Series(index=pd.DatetimeIndex(['2013-01-01 06:00', '2013-01-01 07:00', '2013-01-02 06:00']),
...               data=[1, 2, 3])
>>> s_notz.index
DatetimeIndex(['2013-01-01 06:00:00', '2013-01-01 07:00:00',
               '2013-01-02 06:00:00'],
              dtype='datetime64[ns]', freq=None)
>>> s_notz.resample("D").max()
2013-01-01    2
2013-01-02    3
Freq: D, dtype: int64
>>> s_notz.resample("D").max().index
DatetimeIndex(['2013-01-01', '2013-01-02'], dtype='datetime64[ns]', freq='D')
```

Any progress on this?  Had to spend about 20 minutes figuring this out and it impacts all join operations as the timezones are not treated as equal despite having the same zone name.  So all binary operations will result in losing timezone information ([13] below).  Using tz_convert brings the timezone back in line, but shouldn't be necessary.  Some more examples:

```python
In [11]: s = pd.Series([2], index=pd.date_range('2017-01-01', periods=48, freq="H", tz="US/Eastern"))

In [12]: ss = s / s.resample("D").mean()

In [13]: ss.dropna()
Out[13]: 
2017-01-01 05:00:00+00:00    1.0
2017-01-02 05:00:00+00:00    1.0
dtype: float64

In [14]: ss1 = s / s.resample("D").mean().tz_convert("US/Eastern")

In [15]: ss1.dropna()
Out[15]: 
2017-01-01 00:00:00-05:00    1.0
2017-01-02 00:00:00-05:00    1.0
dtype: float64

In [16]: s.resample("D").mean()
Out[16]: 
2017-01-01 00:00:00-05:00    2
2017-01-02 00:00:00-05:00    2

In [22]: s.resample("D").mean().index.tz
Out[22]: <DstTzInfo 'US/Eastern' EST-1 day, 19:00:00 STD>

In [23]: s.index.tz
Out[23]: <DstTzInfo 'US/Eastern' LMT-1 day, 19:04:00 STD>
```



i think #18596 will fix this if u want to give that a try 
Yes, that will fix the join issue.  Thanks.
ohh great

lmk rebase that tomorrow 

i suspect this will close a number of other issues as well
if you could have a look thru would be great
@rockg xref #19281, though I don't think this actually fixes it, the tz is still converted incorrectly. welcome an investigation.
Yeah, I agree it doesn't fix the resample, just the join issue ([13] in my example).  It works despite resample returning a different tzinfo, so half the battle. 

Resolving patch:
diff --git a/pandas/core/indexes/datetimelike.py b/pandas/core/indexes/datetimelike.py
--- a/pandas/core/indexes/datetimelike.py
+++ b/pandas/core/indexes/datetimelike.py
@@ -1005,7 +1005,7 @@ def shift(self, n, freq=None):
             result = self + offset
 
             if hasattr(self, 'tz'):
-                result.tz = self.tz
+                result._tz = self.tz
 
             return result
 
diff --git a/pandas/core/indexes/datetimes.py b/pandas/core/indexes/datetimes.py
--- a/pandas/core/indexes/datetimes.py
+++ b/pandas/core/indexes/datetimes.py
@@ -511,13 +511,7 @@ def _generate(cls, start, end, periods, name, offset,
                             'different timezones')
 
         inferred_tz = timezones.maybe_get_tz(inferred_tz)
-
-        # these may need to be localized
         tz = timezones.maybe_get_tz(tz)
-        if tz is not None:
-            date = start or end
-            if date.tzinfo is not None and hasattr(tz, 'localize'):
-                tz = tz.localize(date.replace(tzinfo=None)).tzinfo
 
         if tz is not None and inferred_tz is not None:
             if not timezones.tz_compare(inferred_tz, tz):
@@ -654,7 +648,8 @@ def _simple_new(cls, values, name=None, freq=None, tz=None,
         result._data = values
         result.name = name
         result.offset = freq
-        result.tz = timezones.maybe_get_tz(tz)
+        result._tz = timezones.maybe_get_tz(tz)
+        result._tz = timezones.tz_standardize(result._tz)
         result._reset_identity()
         return result
 
@@ -684,6 +679,17 @@ def _values(self):
         else:
             return self.values
 
+    @property
+    def tz(self):
+        # GH 18595
+        return self._tz
+
+    @tz.setter
+    def tz(self, value):
+        # GH 3746: Prevent localizing or converting the index by setting tz
+        raise AttributeError("Cannot directly set timezone. Use tz_localize() "
+                             "or tz_convert() as appropriate")
+
     @property
     def tzinfo(self):
         """
@@ -754,7 +760,7 @@ def _cached_range(cls, start=None, end=None, periods=None, offset=None,
 
             cachedRange = DatetimeIndex._simple_new(arr)
             cachedRange.offset = offset
-            cachedRange.tz = None
+            cachedRange = cachedRange.tz_localize(None)
             cachedRange.name = None
             drc[offset] = cachedRange
         else:
@@ -831,7 +837,7 @@ def __setstate__(self, state):
 
                 self.name = own_state[0]
                 self.offset = own_state[1]
-                self.tz = own_state[2]
+                self._tz = timezones.tz_standardize(own_state[2])
 
                 # provide numpy < 1.7 compat
                 if nd_state[2] == 'M8[us]':
@@ -1175,7 +1181,7 @@ def union(self, other):
         else:
             result = Index.union(this, other)
             if isinstance(result, DatetimeIndex):
-                result.tz = this.tz
+                result._tz = timezones.tz_standardize(this.tz)
                 if (result.freq is None and
                         (this.freq is not None or other.freq is not None)):
                     result.offset = to_offset(result.inferred_freq)
@@ -1223,7 +1229,7 @@ def union_many(self, others):
                 tz = this.tz
                 this = Index.union(this, other)
                 if isinstance(this, DatetimeIndex):
-                    this.tz = tz
+                    this._tz = timezones.tz_standardize(tz)
 
         if this.freq is None:
             this.offset = to_offset(this.inferred_freq)


Test patch:
diff --git a/pandas/tests/frame/test_alter_axes.py b/pandas/tests/frame/test_alter_axes.py
--- a/pandas/tests/frame/test_alter_axes.py
+++ b/pandas/tests/frame/test_alter_axes.py
@@ -249,8 +249,8 @@ def test_set_index_cast_datetimeindex(self):
         # convert to utc
         df['C'] = i.to_series().reset_index(drop=True)
         result = df['C']
-        comp = pd.DatetimeIndex(expected.values).copy()
-        comp.tz = None
+        comp = pd.DatetimeIndex(expected.values)
+        comp = comp.tz_localize(None)
         tm.assert_numpy_array_equal(result.values, comp.values)
 
         # list of datetimes with a tz
diff --git a/pandas/tests/indexes/datetimes/test_construction.py b/pandas/tests/indexes/datetimes/test_construction.py
--- a/pandas/tests/indexes/datetimes/test_construction.py
+++ b/pandas/tests/indexes/datetimes/test_construction.py
@@ -441,6 +441,34 @@ def test_000constructor_resolution(self):
 
         assert idx.nanosecond[0] == t1.nanosecond
 
+    def test_disallow_setting_tz(self):
+        # GH 3746
+        dti = DatetimeIndex(['2010'], tz='UTC')
+        with pytest.raises(AttributeError):
+            dti.tz = pytz.timezone('US/Pacific')
+
+    @pytest.mark.parametrize('tz', [
+        None, 'America/Los_Angeles', pytz.timezone('America/Los_Angeles'),
+        Timestamp('2000', tz='America/Los_Angeles').tz])
+    def test_constructor_start_end_with_tz(self, tz):
+        # GH 18595
+        start = Timestamp('2013-01-01 06:00:00', tz='America/Los_Angeles')
+        end = Timestamp('2013-01-02 06:00:00', tz='America/Los_Angeles')
+        result = DatetimeIndex(freq='D', start=start, end=end, tz=tz)
+        expected = DatetimeIndex(['2013-01-01 06:00:00',
+                                  '2013-01-02 06:00:00'],
+                                 tz='America/Los_Angeles')
+        tm.assert_index_equal(result, expected)
+        # Especially assert that the timezone is consistent for pytz
+        assert pytz.timezone('America/Los_Angeles') is result.tz
+
+    @pytest.mark.parametrize('tz', ['US/Pacific', 'US/Eastern', 'Asia/Tokyo'])
+    def test_constructor_with_non_normalized_pytz(self, tz):
+        # GH 18595
+        non_norm_tz = Timestamp('2010', tz=tz).tz
+        result = DatetimeIndex(['2010'], tz=non_norm_tz)
+        assert pytz.timezone(tz) is result.tz
+
 
 class TestTimeSeries(object):
 
diff --git a/pandas/tests/scalar/timestamp/test_timestamp.py b/pandas/tests/scalar/timestamp/test_timestamp.py
--- a/pandas/tests/scalar/timestamp/test_timestamp.py
+++ b/pandas/tests/scalar/timestamp/test_timestamp.py
@@ -521,6 +521,13 @@ def test_today(self):
         assert (abs(ts_from_string_tz.tz_localize(None) -
                     ts_from_method_tz.tz_localize(None)) < delta)
 
+    @pytest.mark.parametrize('tz', [None, pytz.timezone('US/Pacific')])
+    def test_disallow_setting_tz(self, tz):
+        # GH 3746
+        ts = Timestamp('2010')
+        with pytest.raises(AttributeError):
+            ts.tz = tz
+
 
 class TestTimestamp(object):
 
diff --git a/pandas/tests/test_resample.py b/pandas/tests/test_resample.py
--- a/pandas/tests/test_resample.py
+++ b/pandas/tests/test_resample.py
@@ -2532,6 +2532,18 @@ def test_with_local_timezone_pytz(self):
         expected = Series(1, index=expected_index)
         assert_series_equal(result, expected)
 
+    def test_resample_with_pytz(self):
+        # GH 13238
+        s = Series(2, index=pd.date_range('2017-01-01', periods=48, freq="H",
+                                          tz="US/Eastern"))
+        result = s.resample("D").mean()
+        expected = Series(2, index=pd.DatetimeIndex(['2017-01-01',
+                                                     '2017-01-02'],
+                                                    tz="US/Eastern"))
+        assert_series_equal(result, expected)
+        # Especially assert that the timezone is LMT for pytz
+        assert result.index.tz == pytz.timezone('US/Eastern')
+
     def test_with_local_timezone_dateutil(self):
         # see gh-5430
         local_timezone = 'dateutil/America/Los_Angeles'