Repository: pandas-dev/pandas

Issue:
ValueError exception with pd.resample
When running `df.resample('2200L', how='sum', label='right')` with `df`:

```
2014-10-14 23:06:07.440000    6.44000
2014-10-14 23:06:07.761000    5.09600
2014-10-14 23:06:08.215000    6.44000
2014-10-14 23:06:08.486000    6.44000
2014-10-14 23:06:08.509000    5.20800
2014-10-14 23:06:08.789000    4.02842
2014-10-14 23:06:10.795000    5.65600
2014-10-14 23:06:11.618000    6.21600
2014-10-14 23:06:12.177000    6.21600
2014-10-14 23:06:14.620000    5.10720
2014-10-14 23:06:16.698000    5.95840
2014-10-14 23:06:16.745000    6.44000
2014-10-14 23:06:20.548000    6.21600
2014-10-14 23:06:20.549000    6.44000
2014-10-14 23:06:20.551000    5.95840
2014-10-14 23:06:23.206000    6.44000
2014-10-14 23:06:29.977000    6.44000
2014-10-14 23:06:35.307000    5.20800
2014-10-15 23:00:00               NaN
2014-10-15 23:00:02.200000        NaN
Name: spend, dtype: float64
```

I got:

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-20-71895ab1ef27> in <module>()
----> 1 R_inst = loaded_sorted.tail(200).head(20).resample('2200L', how='sum', label='right')

/Users/josh/anaconda3/envs/py34/lib/python3.4/site-packages/pandas/core/generic.py in resample(self, rule, how, axis, fill_method, closed, label, convention, kind, 
loffset, limit, base)                                                                                                                                                  
   2978                               fill_method=fill_method, convention=convention,
   2979                               limit=limit, base=base)
-> 2980         return sampler.resample(self).__finalize__(self)
   2981 
   2982     def first(self, offset):

/Users/josh/anaconda3/envs/py34/lib/python3.4/site-packages/pandas/tseries/resample.py in resample(self, obj)
     83 
     84         if isinstance(ax, DatetimeIndex):
---> 85             rs = self._resample_timestamps()
     86         elif isinstance(ax, PeriodIndex):
     87             offset = to_offset(self.freq)

/Users/josh/anaconda3/envs/py34/lib/python3.4/site-packages/pandas/tseries/resample.py in _resample_timestamps(self, kind)
    273         axlabels = self.ax
    274 
--> 275         self._get_binner_for_resample(kind=kind)
    276         grouper = self.grouper
    277         binner = self.binner

/Users/josh/anaconda3/envs/py34/lib/python3.4/site-packages/pandas/tseries/resample.py in _get_binner_for_resample(self, kind)
    121             kind = self.kind
    122         if kind is None or kind == 'timestamp':
--> 123             self.binner, bins, binlabels = self._get_time_bins(ax)
    124         elif kind == 'timedelta':
    125             self.binner, bins, binlabels = self._get_time_delta_bins(ax)

/Users/josh/anaconda3/envs/py34/lib/python3.4/site-packages/pandas/tseries/resample.py in _get_time_bins(self, ax)
    182 
    183         # general version, knowing nothing about relative frequencies
--> 184         bins = lib.generate_bins_dt64(ax_values, bin_edges, self.closed, hasnans=ax.hasnans)
    185 
    186         if self.closed == 'right':

/Users/josh/anaconda3/envs/py34/lib/python3.4/site-packages/pandas/lib.so in pandas.lib.generate_bins_dt64 (pandas/lib.c:17928)()

ValueError: Values falls after last bin
```

with `Python 3.4.1 :: Anaconda 2.1.0 (x86_64)` and:

```
Cython==0.21
DataShape==0.3.0
Flask==0.10.1
Jinja2==2.7.3
MarkupSafe==0.23
Pillow==2.5.1
PyYAML==3.11
Pygments==1.6
SQLAlchemy==0.9.7
Sphinx==1.2.3
Theano==0.6.0
Werkzeug==0.9.6
XlsxWriter==0.5.7
abstract-rendering==0.5.1
appnope==0.1.0
argcomplete==0.8.1
arrow==0.4.4
astropy==0.4.2
beautifulsoup4==4.3.2
binstar==0.7.1
bitarray==0.8.1
blaze==0.6.3
blz==0.6.2
bokeh==0.6.1
boto==2.34.0
cffi==0.8.6
colorama==0.3.1
configobj==5.0.6
cryptography==0.5.4
cytoolz==0.7.0
decorator==3.4.0
docutils==0.12
future==0.13.1
greenlet==0.4.4
h5py==2.3.1
ipython==2.2.0
itsdangerous==0.24
jdcal==1.0
jedi==0.8.1-final0
## FIXME: could not find svn URL in dependency_links for this package:
joblib==0.8.3-r1
llvmpy==0.12.7
lpsolve55==5.5.2.0
lxml==3.4.0
matplotlib==1.4.0
mock==1.0.1
multipledispatch==0.4.7
networkx==1.9.1
nltk==3.0.0
nose==1.3.4
numba==0.14.0
numexpr==2.3.1
numpy==1.9.0
openpyxl==1.8.5
pandas==0.15.0
parse==1.6.4
patsy==0.3.0
ply==3.4
psutil==2.1.1
psycopg2==2.5.4
py==1.4.25
pyOpenSSL==0.14
pycosat==0.6.1
pycparser==2.10
pycrypto==2.6.1
pycurl==7.19.5
pyflakes==0.8.1
pymc==3.0
pyparsing==2.0.1
pytest==2.6.3
python-dateutil==2.1
pytz==2014.7
pyzmq==14.3.1
qds-sdk==1.2.2
redis==2.9.1
requests==2.4.3
rope-py3k==0.9.4-1
runipy==0.1.1
scikit-image==0.10.1
scikit-learn==0.15.2
scipy==0.14.0
seaborn==0.5.dev
six==1.8.0
sockjs-tornado==1.0.1
spyder==2.3.1
statsmodels==0.5.0
sympy==0.7.5
tables==3.1.1
toolz==0.7.0
tornado==4.0.2
ujson==1.33
xlrd==0.9.3
```



Discussion:
I can reproduce this with your data (and the error message is in any case not very helpfull), but could you try to pin it down to which 'feature' of the data is causing it?  
E.g. filling the NaNs does not matter, resampling with eg `2000L` or `2200L` does.

@jorisvandenbossche You are right, it also fails with `fillna(0)`. The resample also fails with the following:

```
2014-10-14 23:06:23.206000    6.440
2014-10-14 23:06:29.977000    6.440
2014-10-14 23:06:35.307000    5.208
2014-10-15 23:00:00             0
2014-10-15 23:00:02.200000      0
Name: spend, dtype: float64
```

has to do with a bug I think in the edge detection (_adjust_bin_edges). This are quite a lot of edge cases.

BTW, a small code sample to reproduce it:

```
s = pd.Series(np.random.randn(5), index=pd.date_range('2014-10-14 23:06:23.206', periods=3, freq='400L')|pd.date_range('2014-10-15 23:00:00', periods=2, freq='2200L'))

s.resample('2200L', 'mean')
```


Resolving patch:
diff --git a/pandas/tseries/resample.py b/pandas/tseries/resample.py
--- a/pandas/tseries/resample.py
+++ b/pandas/tseries/resample.py
@@ -411,15 +411,19 @@ def _get_range_edges(first, last, offset, closed='left', base=0):
 def _adjust_dates_anchored(first, last, offset, closed='right', base=0):
     from pandas.tseries.tools import normalize_date
 
+    # First and last offsets should be calculated from the start day to fix an
+    # error cause by resampling across multiple days when a one day period is
+    # not a multiple of the frequency.
+    #
+    # See https://github.com/pydata/pandas/issues/8683
+
     start_day_nanos = Timestamp(normalize_date(first)).value
-    last_day_nanos = Timestamp(normalize_date(last)).value
 
     base_nanos = (base % offset.n) * offset.nanos // offset.n
     start_day_nanos += base_nanos
-    last_day_nanos += base_nanos
 
     foffset = (first.value - start_day_nanos) % offset.nanos
-    loffset = (last.value - last_day_nanos) % offset.nanos
+    loffset = (last.value - start_day_nanos) % offset.nanos
 
     if closed == 'right':
         if foffset > 0:


Test patch:
diff --git a/pandas/tseries/tests/test_resample.py b/pandas/tseries/tests/test_resample.py
--- a/pandas/tseries/tests/test_resample.py
+++ b/pandas/tseries/tests/test_resample.py
@@ -705,6 +705,29 @@ def test_resample_anchored_monthstart(self):
         for freq in freqs:
             result = ts.resample(freq, how='mean')
 
+    def test_resample_anchored_multiday(self):
+        # When resampling a range spanning multiple days, ensure that the
+        # start date gets used to determine the offset.  Fixes issue where
+        # a one day period is not a multiple of the frequency.
+        #
+        # See: https://github.com/pydata/pandas/issues/8683
+
+        s = pd.Series(np.random.randn(5),
+                      index=pd.date_range('2014-10-14 23:06:23.206',
+                                          periods=3, freq='400L')
+                      | pd.date_range('2014-10-15 23:00:00',
+                                      periods=2, freq='2200L'))
+
+        # Ensure left closing works
+        result = s.resample('2200L', 'mean')
+        self.assertEqual(result.index[-1],
+                         pd.Timestamp('2014-10-15 23:00:02.000'))
+
+        # Ensure right closing works
+        result = s.resample('2200L', 'mean', label='right')
+        self.assertEqual(result.index[-1],
+                         pd.Timestamp('2014-10-15 23:00:04.200'))
+
     def test_corner_cases(self):
         # miscellaneous test coverage