Repository: pandas-dev/pandas

Issue:
groupby filter can't access all columns
Unexpectedly? don't have access to the grouped columns.

Also seems to allow returning a boolean Series (and takes the first item as the condition).

```
In [11]: df = pd.DataFrame([[1, 2], [1, 3], [5, 6]], columns=['A', 'B'])

In [12]: g = df.groupby('A')  # same with as_index=False (which *correctly* has no effect)

In [13]: g.filter(lambda x: x['A'].sum() == 2)
KeyError: u'no item named A'

In [14]: g.filter(lambda x: x['B'].sum() == 5)  # works
Out[14]:
   A  B
0  1  2
1  1  3

named A'

In [14]: g.filter(lambda x: x['B'].sum() == 5)  # works
Out[14]:
   A  B
0  1  2
1  1  3

In [15]: g.filter(lambda x: x.sum() == 5)  # weird that this works (excepted raise)
Out[15]:
   A  B
0  1  2
1  1  3

In [16]: g = df.groupby(df['A'])  # hack/workaround

In [16]: g.filter(lambda x: x.sum() == 5)  # seems to look at first col
Out[16]:
   A  B
2  5  6

In [17]: g.filter(lambda x: x['A'].sum() == 5)  # works
Out[17]:
   A  B
2  5  6

In [18]: g.filter(lambda x: x['B'].sum() == 5)  # works
Out[18]:
   A  B
0  1  2
1  1  3
```

cc @danielballan



Discussion:
Woah, this _is_ bad. Very surprised the huge number of tests does not cover this. I'm too busy to dig into this this week, but I can get into it next week. Let me know if you make any progress in the meantime.

Not entirely sure, but this might be related: http://stackoverflow.com/questions/22139053/grouped-function-between-2-columns-in-a-pandas-dataframe

```
df = pandas.DataFrame({"Dummy":[1,2]*6, "X":[1,3,7]*4, 
                       "Y":[2,3,4]*4, "group":["A","B"]*6})
df.groupby('group')[:,['X', 'Y']].head(1)

         Dummy  X  Y group
group                     
A     0      1  1  2     A
B     1      2  3  3     B

[2 rows x 4 columns]
```

@naught101 I posted your find as #6524.

I suspect that it's because this syntax is not supported:

```
g[:,['X', 'Y']]
g[['X', 'Y']]
```

Trivial fix after #6570, use `selected_obj` rather than `objwithexclusions` on first line.


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -2529,7 +2529,7 @@ def filter(self, func, dropna=True, *args, **kwargs):
 
         indices = []
 
-        obj = self._obj_with_exclusions
+        obj = self._selected_obj
         gen = self.grouper.get_iterator(obj, axis=self.axis)
 
         fast_path, slow_path = self._define_paths(func, *args, **kwargs)


Test patch:
diff --git a/pandas/tests/test_groupby.py b/pandas/tests/test_groupby.py
--- a/pandas/tests/test_groupby.py
+++ b/pandas/tests/test_groupby.py
@@ -3438,6 +3438,13 @@ def test_filter_and_transform_with_non_unique_string_index(self):
         actual = grouped_df.pid.transform(len)
         assert_series_equal(actual, expected)
 
+    def test_filter_has_access_to_grouped_cols(self):
+        df = DataFrame([[1, 2], [1, 3], [5, 6]], columns=['A', 'B'])
+        g = df.groupby('A')
+        # previously didn't have access to col A #????
+        filt = g.filter(lambda x: x['A'].sum() == 2)
+        assert_frame_equal(filt, df.iloc[[0, 1]])
+
     def test_index_label_overlaps_location(self):
         # checking we don't have any label/location confusion in the
         # the wake of GH5375
@@ -3486,7 +3493,8 @@ def test_groupby_selection_with_methods(self):
                    'idxmin', 'idxmax',
                    'ffill', 'bfill',
                    'pct_change',
-                   'tshift'
+                   'tshift',
+                   #'ohlc'
                    ]
 
         for m in methods:
@@ -3501,8 +3509,11 @@ def test_groupby_selection_with_methods(self):
                            g_exp.apply(lambda x: x.sum()))
 
         assert_frame_equal(g.resample('D'), g_exp.resample('D'))
+        assert_frame_equal(g.resample('D', how='ohlc'),
+                           g_exp.resample('D', how='ohlc'))
 
-
+        assert_frame_equal(g.filter(lambda x: len(x) == 3),
+                           g_exp.filter(lambda x: len(x) == 3))
 
     def test_groupby_whitelist(self):
         from string import ascii_lowercase