BUG: Fix groupby duplicate column error message
State: closed

closes #7511

Though having duplicated column names in a dataframe is never a good idea, it may happen, and that shouldn't confuse groupby() with a meaningless message. Currently

``` python
>>> obj=pd.DataFrame(columns=["A","B","A", "C"], data=[range(4), range(2,6), range(0, 8, 2)])
>>> obj.groupby("A").mean()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/core/generic.py", line 2846, in groupby
    sort=sort, group_keys=group_keys, squeeze=squeeze)
  File "pandas/core/groupby.py", line 1173, in groupby
    return klass(obj, by, **kwds)
  File "pandas/core/groupby.py", line 390, in __init__
    level=level, sort=sort)
  File "pandas/core/groupby.py", line 2086, in _get_grouper
    ping = Grouping(group_axis, gpr, obj=obj, name=name, level=level, sort=sort)
  File "pandas/core/groupby.py", line 1925, in __init__
    self.grouper = self.index.map(self.grouper)
  File "pandas/core/index.py", line 1524, in map
    return self._arrmap(self.values, mapper)
  File "generated.pyx", line 2203, in pandas.algos.arrmap_int64 (pandas/algos.c:72274)
TypeError: 'DataFrame' object is not callable
```

This patch fixes that.

``` python
>>> obj.groupby("A").mean()
     B  C
A A      
0 2  1  3
  4  2  6
2 4  3  5
```

Thanks.


Labels:
name: Groupby
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
this is issue #7511

This needs to raise if `getattr(self.grouper,'ndim',None) != 1`
https://github.com/pydata/pandas/blob/master/pandas/core/groupby.py#L1943

instead of the specific check you are doing

ok, i updated following your request
- The underlying ndim error in Grouping now properly raise (+ testcase) (though it's harder for end-user to trigger, as the problematic use-case itself is now correctly handled)
- all assert -> self.assertTrue

Well, if pandas allow dataframes to have duplicated columns, ability to group like this seems a pretty user-expectable consequence, to me. In fact, I'm not even sure of an easy (user-friendly) way to workaround that limitation without altering/duplicating the data...

But anyway, i don't mind much about that feature, and it's trivial to put back in the future if needed. So the current code now focuses only on the error message thing, as requested. See last commit.

Thanks

@bthyreau ok will merge this thanks.

In response to your comment, the ambiguity is that you are saying to pandas, hey I have multiple columns that I want to groupby on but they have the same NAME! (e.g. 'A','A') in your example. So the order if these is ambiguous.

If you wanted to group this way you could

`df.groupby([df.iloc[:,3],df.iloc[:,4]])` or whatever. I honestly don't think this is very common at all and most likely a mistake; that's why we are raising.

merged via https://github.com/pydata/pandas/commit/77d5f0403905544f576ce9298c42163dc5e35827

Ok Thanks
I noticed that you didn't merge my specific error-condition message before the Grouping() call; ie. instead of crashing with:

```
>>>df.groupby("A")
ValueError: Grouper result is not unique for 'A'
it would crashes with
ValueError: Grouper result with an ndim != 1
```

which, for the user point-of-view, is arguably no more helpful than before (#7511)

If that was not intended, i can reopen or create a new pr; just tell me.
Thanks

ps. code was:

```
+ if isinstance(gpr, DataFrame) and gpr.ndim > 1:
+   errmsg = "Grouper result is not unique for '%s'" % name
+   raise ValueError(errmsg)
```

I took that out as it is a very specific case; better to have a general message.

I understand, but then there is no way for the user to know what causes the error.
Would you accept some trivial PR with at least the following error-message change ?
raise ValueError("Grouper result with an ndim != 1")
to
raise ValueError("Grouper result with an ndim != 1 for '%s'" % self.name)

sure

how about showing tyoe(self.object)

can't assume it would have a name