Repository: pandas-dev/pandas

Issue:
Multi-column grouping (e.g. OHLC) for DataFrame
This currently only works for single time series



Discussion:
Apparently the reason why 'ohlc' does not work for DataFrame.resample is that it only can create new column names `['open','close','high','low']` without prepending it with the original column name. Wouldn't the use of the column name as prefix already enable this function to be useable on DataFrames?

[Example on SO](http://stackoverflow.com/questions/18593427/how-to-convert-time-series-data-with-pandas-and-show-graph-via-highstock):

```
In [1]: df = pd.read_clipboard(sep='\s\s+')

In [2]: df
Out[2]: 
    TIMESTAMP  PRICE      VOLUME
0  1294311545  24990  1500000000
1  1294317813  25499  5000000000
2  1294318449  25499   100000000

In [3]: df.TIMESTAMP = pd.to_datetime(df.TIMESTAMP, unit='s')

In [4]: df.set_index('TIMESTAMP', inplace=True)

In [5]: vol_ohlc = df['VOLUME'].resample('H', how='ohlc')

In [6]: price_ohlc = df['PRICE'].resample('H', how='ohlc')

In [7]: pd.concat([vol_ohlc, price_ohlc], 1, keys=['VOLUME', 'PRICE'])
Out[7]: 
                         VOLUME                                      PRICE  \
                           open        high         low       close   open   
TIMESTAMP                                                                    
2011-01-06 10:00:00  1500000000  1500000000  1500000000  1500000000  24990   
2011-01-06 11:00:00         NaN         NaN         NaN         NaN    NaN   
2011-01-06 12:00:00  5000000000  5000000000   100000000   100000000  25499   


                      high    low  close  
TIMESTAMP                                 
2011-01-06 10:00:00  24990  24990  24990  
2011-01-06 11:00:00    NaN    NaN    NaN  
2011-01-06 12:00:00  25499  25499  25499 
```

So lazy implementation of this is just:

```
pd.concat((df.iloc[:, i].resample(rule, how='ohlc')
                    for i, _ in enumerate(df.columns)),
                keys=df.columns, 
                axis=1)
```

Not sure where to put that...

I think their is an issue about this already (can't find ATM)

@jreback not sure if this should go in groupby's ohlc function, if so was wondering if you know a way to iterate through columns SeriesGroupbys:

```
ipdb> self
<pandas.core.groupby.DataFrameGroupBy object at 0x101d9b090>
ipdb> for i in self._iterate_slices(): print i
('PRICE', 2011-01-06 10:59:05    24990
2011-01-06 12:43:33    25499
2011-01-06 12:54:09    25499
Name: PRICE, dtype: int64)
('VOLUME', 2011-01-06 10:59:05    1500000000
2011-01-06 12:43:33    5000000000
2011-01-06 12:54:09     100000000
Name: VOLUME, dtype: int64)
ipdb> self['PRICE']  # I want these
<pandas.core.groupby.SeriesGroupBy object at 0x10455ff10>
```

(Basically an iloc for DataFrameGroupbys)

could use column comes via self.obj.columns but that would fail for dupe names.  :(

`get_group` maybe? I would have to step thru...don't know otomh (off the top of my head)

Hmmm, not sure there is I created _iterate_column_groupbys to iterate with (name, SeriesGroupby). Then this drops out easy.

_Quite a few methods are broken for certain groupbys, have created issue before about it._

Also, resample doesn't work with dupe columns atm anyway. Maybe this method will help, atm seem to always be accessed via name (as no other choice).


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -1619,7 +1619,6 @@ def _iterate_slices(self):
         for val in slice_axis:
             if val in self.exclusions:
                 continue
-
             yield val, slicer(val)
 
     def _cython_agg_general(self, how, numeric_only=True):
@@ -2233,6 +2232,26 @@ def _wrap_agged_blocks(self, blocks):
 
         return result.convert_objects()
 
+    def _iterate_column_groupbys(self):
+        for i, colname in enumerate(self.obj.columns):
+            yield colname, SeriesGroupBy(self.obj.iloc[:, i], selection=colname,
+                                         grouper=self.grouper,
+                                         exclusions=self.exclusions)
+
+    def _apply_to_column_groupbys(self, func):
+        from pandas.tools.merge import concat
+        return concat((func(col_groupby)
+                            for _, col_groupby in self._iterate_column_groupbys()),
+                        keys=self.obj.columns,
+                        axis=1)
+
+    def ohlc(self):
+        """
+        Compute sum of values, excluding missing values
+
+        For multiple groupings, the result index will be a MultiIndex
+        """
+        return self._apply_to_column_groupbys(lambda x: x._cython_agg_general('ohlc'))
 
 from pandas.tools.plotting import boxplot_frame_groupby
 DataFrameGroupBy.boxplot = boxplot_frame_groupby


Test patch:
diff --git a/pandas/tseries/tests/test_resample.py b/pandas/tseries/tests/test_resample.py
--- a/pandas/tseries/tests/test_resample.py
+++ b/pandas/tseries/tests/test_resample.py
@@ -259,6 +259,31 @@ def test_resample_ohlc(self):
         self.assertEquals(xs['low'], s[:5].min())
         self.assertEquals(xs['close'], s[4])
 
+    def test_resample_ohlc_dataframe(self):
+        df = (pd.DataFrame({'PRICE': {Timestamp('2011-01-06 10:59:05', tz=None): 24990,
+                                     Timestamp('2011-01-06 12:43:33', tz=None): 25499,
+                                     Timestamp('2011-01-06 12:54:09', tz=None): 25499},
+                           'VOLUME': {Timestamp('2011-01-06 10:59:05', tz=None): 1500000000,
+                                      Timestamp('2011-01-06 12:43:33', tz=None): 5000000000,
+                                      Timestamp('2011-01-06 12:54:09', tz=None): 100000000}})
+            ).reindex_axis(['VOLUME', 'PRICE'], axis=1)
+        res = df.resample('H', how='ohlc')
+        exp = pd.concat([df['VOLUME'].resample('H', how='ohlc'),
+                          df['PRICE'].resample('H', how='ohlc')],
+                        axis=1,
+                        keys=['VOLUME', 'PRICE'])
+        assert_frame_equal(exp, res)
+
+        df.columns = [['a', 'b'], ['c', 'd']]
+        res = df.resample('H', how='ohlc')
+        exp.columns = pd.MultiIndex.from_tuples([('a', 'c', 'open'), ('a', 'c', 'high'),
+            ('a', 'c', 'low'), ('a', 'c', 'close'), ('b', 'd', 'open'),
+            ('b', 'd', 'high'), ('b', 'd', 'low'), ('b', 'd', 'close')])
+        assert_frame_equal(exp, res)
+
+        # dupe columns fail atm
+        # df.columns = ['PRICE', 'PRICE']
+
     def test_resample_reresample(self):
         dti = DatetimeIndex(
             start=datetime(2005, 1, 1), end=datetime(2005, 1, 10),