Repository: pandas-dev/pandas

Issue:
'infer_freq' does not work with DST transition 
Consider the following index crossing a DST transition:

``` python
In [1]: import pandas as pd;pd.__version__
Out[1]: '0.15.0'
In [2]: index = pd.date_range(pd.Timestamp("2014-10-25 03:00", tz="Europe/Paris"), periods=10, freq="3H")

In [3]: index
Out[3]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2014-10-25 03:00:00+02:00, ..., 2014-10-26 05:00:00+01:00]
Length: 10, Freq: 3H, Timezone: Europe/Paris
```

Pandas cannot infer the index frequency:

``` python
In [4]: pd.infer_freq(index) is None
Out[4]: True
```

If we convert the index to UTC, the frequency can be inferred:

``` python
In [5]: pd.infer_freq(index.tz_convert("UTC"))
Out[5]: '3H'
```

If the index does not cross a DST boundary, the frequency can be inferred as well:

``` python
In [6]: index = pd.date_range(pd.Timestamp("2014-10-25 03:00", tz="Europe/Paris"), periods=6, freq="3H")

In [7]: index
Out[7]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2014-10-25 03:00:00+02:00, ..., 2014-10-25 18:00:00+02:00]
Length: 6, Freq: 3H, Timezone: Europe/Paris

In [8]: pd.infer_freq(index)
Out[8]: '3H'
```



Discussion:
cc @rockg
cc @ischwabacher

I am not sure this is a bug. The frequency inferer is looking for a constant difference between each of the times (essentially diffing them). So the freq by-definition is different around DST transitions. So `None` is correct (it is not a regular frequency). Though that _may_ be incorrect (e.g. maybe it should be inferring with UTC?)

I think this is a bug.  In terms of hours, there is a constant difference.  The code is converting the underlying UTC values to the local timezone so that week/month frequencies are calculated in local time (for example, 2010-12-25 00:00 is a Saturday in Paris but a Friday in UTC) but when that is done across timezones that shift becomes broken.  The issue is in `tseries.frequencies._FrequencyInferer`.

```
    def __init__(self, index, warn=True):
        self.index = index
        self.values = np.asarray(index).view('i8')

        if hasattr(index,'tz'):
            if index.tz is not None:
                self.values = tslib.tz_convert(self.values, 'UTC', index.tz)
```

Using the above example, we have:

```
index.asi8
[1414198800000000000 1414209600000000000 1414220400000000000
 1414231200000000000 1414242000000000000 1414252800000000000
 1414263600000000000 1414274400000000000 1414285200000000000
 1414296000000000000]
np.diff(index.asi8)
[10800000000000 10800000000000 10800000000000 10800000000000 10800000000000
 10800000000000 10800000000000 10800000000000 10800000000000]
self.values
[1414206000000000000 1414216800000000000 1414227600000000000
 1414238400000000000 1414249200000000000 1414260000000000000
 1414270800000000000 1414281600000000000 1414288800000000000
 1414299600000000000]
np.diff(self.values)
[10800000000000 10800000000000 10800000000000 10800000000000 10800000000000
 10800000000000 10800000000000  7200000000000 10800000000000]
```

Hopefully this is enough for somebody to go off on.  It's not immediately obvious what the fix is (besides treating the sub-daily and daily+ offsets differently--not converting in the former and converting in the latter).  


Resolving patch:
diff --git a/pandas/tseries/frequencies.py b/pandas/tseries/frequencies.py
--- a/pandas/tseries/frequencies.py
+++ b/pandas/tseries/frequencies.py
@@ -698,6 +698,8 @@ def __init__(self, index, warn=True):
         self.index = index
         self.values = np.asarray(index).view('i8')
 
+        # This moves the values, which are implicitly in UTC, to the
+        # the timezone so they are in local time
         if hasattr(index,'tz'):
             if index.tz is not None:
                 self.values = tslib.tz_convert(self.values, 'UTC', index.tz)
@@ -712,10 +714,18 @@ def __init__(self, index, warn=True):
     @cache_readonly
     def deltas(self):
         return tslib.unique_deltas(self.values)
+    
+    @cache_readonly
+    def deltas_asi8(self):
+        return tslib.unique_deltas(self.index.asi8)
 
     @cache_readonly
     def is_unique(self):
         return len(self.deltas) == 1
+    
+    @cache_readonly
+    def is_unique_asi8(self):
+        return len(self.deltas_asi8) == 1
 
     def get_freq(self):
         if not self.is_monotonic or not self.index.is_unique:
@@ -725,9 +735,12 @@ def get_freq(self):
         if _is_multiple(delta, _ONE_DAY):
             return self._infer_daily_rule()
         else:
-            # Possibly intraday frequency
-            if not self.is_unique:
+            # Possibly intraday frequency.  Here we use the 
+            # original .asi8 values as the modified values
+            # will not work around DST transitions.  See #8772
+            if not self.is_unique_asi8:
                 return None
+            delta = self.deltas_asi8[0]
             if _is_multiple(delta, _ONE_HOUR):
                 # Hours
                 return _maybe_add_count('H', delta / _ONE_HOUR)


Test patch:
diff --git a/pandas/tseries/tests/test_frequencies.py b/pandas/tseries/tests/test_frequencies.py
--- a/pandas/tseries/tests/test_frequencies.py
+++ b/pandas/tseries/tests/test_frequencies.py
@@ -268,6 +268,24 @@ def test_infer_freq_tz(self):
                 idx = DatetimeIndex(dates, tz=tz)
                 self.assertEqual(idx.inferred_freq, expected)
 
+    def test_infer_freq_tz_transition(self):
+        # Tests for #8772
+        date_pairs = [['2013-11-02', '2013-11-5'], #Fall DST
+                      ['2014-03-08', '2014-03-11'], #Spring DST
+                      ['2014-01-01', '2014-01-03']] #Regular Time
+        freqs = ['3H', '10T', '3601S', '3600001L', '3600000001U', '3600000000001N']
+
+        for tz in [None, 'Australia/Sydney', 'Asia/Tokyo', 'Europe/Paris',
+                   'US/Pacific', 'US/Eastern']:
+            for date_pair in date_pairs:
+                for freq in freqs:
+                    idx = date_range(date_pair[0], date_pair[1], freq=freq, tz=tz)
+                    print(idx)
+                    self.assertEqual(idx.inferred_freq, freq)
+                
+        index = date_range("2013-11-03", periods=5, freq="3H").tz_localize("America/Chicago")
+        self.assertIsNone(index.inferred_freq)
+
     def test_not_monotonic(self):
         rng = _dti(['1/31/2000', '1/31/2001', '1/31/2002'])
         rng = rng[::-1]