ENH: Add DataFrameGroupBy.value_counts
State: closed

- [x] closes #43564
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Enhancement
name: Groupby

Comments:
Hello @johnzangwill! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:











There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-12-19 00:21:03 UTC
@rhshadrach
> df = pd.DataFrame({'a': [1, 1, 2, 2], 'b': [3, 3, 4, 4], 'c': [5, 5, 6, 7]})
> gb = df.groupby(df.index)
> res = gb.value_counts()

Awesome! Diabolical! Thanks for spotting that. It only happens when` rows= columns + 1`.
The "by" parameter of groupby is horribly overloaded and there is a tangle of heuristics to disambiguate the groupings. Grouping and Grouper are completely different but seem to be used interchangably...
My commit https://github.com/johnzangwill/pandas/commit/9b2869f5b4371b21dfc9402e0210a2b804db2a5e fixes the problem and does not seem to break anything. It is a tiny change, but if you want, it could be made into separate PR.



CI Doc build is failing (#44494) but it is nothing to do with my code.
Change with reset_index working with duplicate labels (Now a separate PR #44755)
I've taken out the fix for column duplicates in frame.py, and incuded the duplicates failure for `as_index=False` in the tests:
> https://github.com/johnzangwill/pandas/blob/6c0d7f840cb82fddac814a23df691b81764ab7cf/pandas/tests/groupby/test_frame_value_counts.py#L434

When #44755 gets resolved then I will fix the implementation and tests accordingly.
No progress on #44755, and I don't want that holding this up, so I am taking it out of Draft.
> lgtm pending #44755

The seemingly innocuous #44755 seems to have become stuck again. I would merge this PR now. I just needed #44755 to cover @rhshadrach's pathological case with as_index=False, It raises, and is covered in the tests, and I don't think that it matters.

Agree this case is not of sufficient severity to be of concern. But from https://github.com/pandas-dev/pandas/pull/44267#discussion_r762444074, can we check this upfront rather than go through the entire computation just to fail. I think this would just amount to checking if there are duplicate column labels in the case `as_index=False`.
@rhshadrach It is more complicated than just duplicates on the incoming. Your example had a column labelled "level_1" that happened to clash with a new column from a non-column grouper. The question is: do you want it to work and create duplicates, or raise?
Also, if the incomong data already has actual duplicates, then it seems bad to be unable to process it. But I could always fix that later in #44755, if and when it ever gets merged...
Ah, very true. Of course, this is already an issue with other ops, e.g.

```
df = pd.DataFrame({'level_1': [1, 1, 2]})
df.groupby(['level_1', [0, 0, 1]], as_index=False).size()
```

raises as well. I'd suggest raising an issue on this, and only specifically detecting duplicate input columns here.
```
> =================================== FAILURES ===================================
_____ [doctest] pandas.core.groupby.generic.DataFrameGroupBy.value_counts ______
1633         >>> df
1634             gender 	education 	country
1635         0 	male 	low 	    US
1636         1 	male 	medium 	    FR
1637         2 	female 	high 	    US
1638         3 	male 	low 	    FR
1639         4 	female 	high 	    FR
1640         5 	male 	low 	    FR
1641 
1642         >>> df.groupby('gender').value_counts()
Differences (unified diff with -expected +actual):
    @@ -5,3 +5,3 @@
                        US         1
             medium     FR         1
    -dtype: float64
    +dtype: int64

/home/runner/work/pandas/pandas/pandas/core/groupby/generic.py:1642: DocTestFailure
```

looks like a doc-test failure
All green
thanks @johnzangwill very nice!