Repository: pandas-dev/pandas

Issue:
groupby(group_keys=True) ignored when apply returns unsliced data
I ran into this unexplained behaviour with `groupby` when using  `group_keys=True` (the default), 
it's not clear why using `x` vs. `x[:]` causes the `group_keys` argument to be ignored.

``` python
In [86]: df = DataFrame({'key': [1, 1, 1, 2, 2, 2, 3, 3, 3],
    ...:                 'value': range(9)})
    ...: df
Out[86]: 
   key  value
0    1      0
1    1      1
2    1      2
3    2      3
4    2      4
5    2      5
6    3      6
7    3      7
8    3      8

In [87]: df.groupby('key', group_keys=True).apply(lambda x: x[:].key)
Out[87]: 
key   
1    0    1
     1    1
     2    1
2    3    2
     4    2
     5    2
3    6    3
     7    3
     8    3
Name: key, dtype: int64

In [88]: df.groupby('key', group_keys=True).apply(lambda x: x.key)
Out[88]: 
0    1
1    1
2    1
3    2
4    2
5    2
6    3
7    3
8    3
Name: key, dtype: int64
```



Discussion:
has nothing to do with `group_keys` (which is a very odd option anyhow).

Has to do with whether you are returning something that is exactly identical or not

`x[:].key` is NOT identical to `x.key`. They are equal, but the 2nd is the exact object, while the first is a copy. These aggregate differently because that's how groupby determines mutation. (e.g. even though you didn't actually mutate, it looks like you are).

Impossible to disambiguate. Nor even sure why you would.

What are you actually trying to do?

Actually. I asked why `x` cased doesn't behave like what you call the mutated case, not the other way round.
Also, I'm returning a series when I get in a dataframe in the apply function. How is `x[:2]` "mutation"
while `x.foo` is not?
If it's impossible to "disambiguate" when the return types are different, well. I don't know what you mean.

The docstring for group_keys is:

``` python
group_keys : boolean, default True
    When calling apply, add group keys to index to identify pieces
```

What I'm trying to do is to get the group keys included in the index, when I call apply.

I already have a workaround, It just seemed like a bug. If you're sure this is all just fine, feel free to close.


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -2851,8 +2851,9 @@ def _wrap_applied_output(self, keys, values, not_indexed_same=False):
                             return concat(values)
 
                     if not all_indexed_same:
+                        # GH 8467
                         return self._concat_objects(
-                            keys, values, not_indexed_same=not_indexed_same
+                            keys, values, not_indexed_same=True,
                         )
 
                 try:


Test patch:
diff --git a/pandas/tests/test_groupby.py b/pandas/tests/test_groupby.py
--- a/pandas/tests/test_groupby.py
+++ b/pandas/tests/test_groupby.py
@@ -2112,6 +2112,18 @@ def f_no_copy(x):
         grpby_no_copy = mydf.groupby('cat1').apply(f_no_copy)
         assert_series_equal(grpby_copy,grpby_no_copy)
 
+    def test_no_mutate_but_looks_like(self):
+
+        # GH 8467
+        # first show's mutation indicator
+        # second does not, but should yield the same results
+        df = DataFrame({'key': [1, 1, 1, 2, 2, 2, 3, 3, 3],
+                        'value': range(9)})
+
+        result1 = df.groupby('key', group_keys=True).apply(lambda x: x[:].key)
+        result2 = df.groupby('key', group_keys=True).apply(lambda x: x.key)
+        assert_series_equal(result1, result2)
+
     def test_apply_chunk_view(self):
         # Low level tinkering could be unsafe, make sure not
         df = DataFrame({'key': [1, 1, 1, 2, 2, 2, 3, 3, 3],