Repository: pandas-dev/pandas

Issue:
BUG: ngroups and len(groups) do not equal when grouping with a list of Grouper and column label
#### Code

```python
m_index = pd.MultiIndex.from_product([['Mona', 'Dennis'], ['good', 'great']], names=['name', 'desc']) 
df = pd.DataFrame({'foo': [1, 2, 1, 2], 'bar': np.random.randn(4)},  index=m_index)
n1 = df.groupby([pd.Grouper(level='name'), 'foo']).ngroups
n2 = len(df.groupby([pd.Grouper(level='name'), 'foo'])) 
n3 = len(df.groupby([pd.Grouper(level='name'), 'foo']).groups) 
print(n1, n2, n3)
# 4 2 2

```
#### Problem description

I grouped a DataFrame by a list of pandas.Grouper object and column label, the length of groups or group keys should be 4, but I checked the length of GroupBy object as n2, it is 2, why? Thanks for your answer in advance.

<details>

ommit: None
python: 3.7.3.final.0
python-bits: 64
OS: Darwin
OS-release: 18.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: zh_CN.UTF-8
LOCALE: zh_CN.UTF-8

</details>


Discussion:
`len(<groupByObject>)` returns the number of groups as does calling the length on the `groups` property as in `n3`

`ngroups` on the other hand 

```
 def ngroups(self):
        return len(self.result_index)
```

returns the `result_index` which is a `MultiIndex` object. For the above example, the `result_index` output is the following `MultiIndex`

```
MultiIndex(levels=[['Dennis', 'Mona'], [1, 2]],
           codes=[[0, 0, 1, 1], [0, 1, 0, 1]],
           names=['name', 'foo'])
```

As of **pandas 0.24.2**, the `MultiIndex` `__len__` method is implemented as

```
def __len__(self):
        return len(self.codes[0])
```

so it returns the length of the first set of codes in the `MultiIndex`. Hence, `ngroups` outputs 4.'

---------

The groups that are formed in the example's `groupby` operation are

```
{('Dennis', 1): MultiIndex(levels=[['Dennis', 'Mona'], ['good', 'great']],
            codes=[[1], [0]],
            names=['name', 'desc']),
 ('Mona', 2): MultiIndex(levels=[['Dennis', 'Mona'], ['good', 'great']],
            codes=[[1], [1]],
            names=['name', 'desc'])}
```

It appears that the correct groups are not being formed correctly when you are passing a `pandas.Grouper` object

--------

Not sure if this is appropriate behaviour though.



This is pretty tricky. I'm surprised to see a difference between the below as well:

```python
>>> df.groupby([pd.Grouper(level=0), 'foo']).grouper.groups.keys()
dict_keys([('Dennis', 1), ('Mona', 2)])

>>> df.reset_index(level=1).groupby(['name', 'foo']).grouper.groups.keys()
dict_keys([('Dennis', 1), ('Dennis', 2), ('Mona', 1), ('Mona', 2)])

>>> df.groupby([pd.Grouper(['Mona', 'Mona', 'Dennis', 'Dennis']), 'foo']).grouper.groups.keys()
dict_keys([('Dennis', 1), ('Dennis', 2), ('Mona', 1), ('Mona', 2)])
```

I think the problem starts here:

https://github.com/pandas-dev/pandas/blob/ee6b1318ff07806ead79dac8119ad9ce580afe1c/pandas/core/groupby/ops.py#L261

Where the groups are getting constructed from the individual groupings (here the first grouping is the `pd.Grouper(level='name')` and the second grouping is the Series containing the values for `foo`).

The problem is that with the former iteration only goes over the unique values (here 'Dennis' and 'Mona') which is why your length calculations are getting truncated to 2:

https://github.com/pandas-dev/pandas/blob/ee6b1318ff07806ead79dac8119ad9ce580afe1c/pandas/core/groupby/grouper.py#L348

Not sure the best resolution off the top of my head but if you'd like to investigate further and try your hand at a PR would certainly be welcome!
When a `Grouping` is created using a `Grouper` object, the resultant `Grouping` consists of a list containing one object -> The `BaseGrouper` corresponding to the `Grouper` object initially passed. In OP's example, the first `Grouper` object is passed with `level='name'` parameter and the second is simply an index `foo`.  The `DataFrameGroupBy` finally constructed has a `BaseGrouper` object with the following `Grouping` objects

```
>>> dfGroupBy = df.grouby([pd.Grouper(level='name'), 'foo'])
>>> dfGroupBy.grouper.groupings
[Grouping(name), Grouping(foo)]
```
The `Grouping(name)` object is the one generated from the `Grouper` object. The `grouper` parameter of this `Grouping(name)` object is itself a `BaseGrouper`

```
>>> dfGroupBy.grouper.groupings[0].grouper
<pandas.core.groupby.groupby.BaseGrouper>
```

Meanwhile, if the `Grouping` is constructed simply using an `index` like `foo`

```
>> dfGroupby.grouper.groupings[1].grouper
array([1, 2, 1, 2], dtype=int64)
```

The `grouper` is a simple array.

_(Is this behavior intentional?)_

----

Also as @WillAyd mentioned, 

```
to_groupby = lzip(*(ping.grouper for ping in self.groupings))
```
when working with a `BaseGrouper` returns only the indices rather than an array of appropriate values.

-----

So, a possible fix might be either to handle the special case of a `BaseGrouper` while constructing the variable `to_groupby` or updating the `_get_grouper()` code to not have a `BaseGrouper` object as a member of a `Grouping`

Adding to the above discussion,
```
>>> dfGroupby.grouper.groupings[0].grouper.groupings[0].grouper
Index(['Mona', 'Mona', 'Dennis', 'Dennis'], dtype='object', name='name')
```
Thanks for the bug confirmation, will use combination of labels to groupby instead.
As a beginner of pandas and even Python, I am trying to understand the codes and logic you mentioned above.😂

Resolving patch:
diff --git a/pandas/core/groupby/grouper.py b/pandas/core/groupby/grouper.py
--- a/pandas/core/groupby/grouper.py
+++ b/pandas/core/groupby/grouper.py
@@ -280,7 +280,7 @@ def __init__(self, index, grouper=None, obj=None, name=None, level=None,
             if self.name is None:
                 self.name = grouper.result_index.name
             self.obj = self.grouper.obj
-            self.grouper = grouper
+            self.grouper = grouper._get_grouper()
 
         else:
             if self.grouper is None and self.name is not None:
diff --git a/pandas/core/groupby/ops.py b/pandas/core/groupby/ops.py
--- a/pandas/core/groupby/ops.py
+++ b/pandas/core/groupby/ops.py
@@ -150,6 +150,15 @@ def _get_splitter(self, data, axis=0):
         comp_ids, _, ngroups = self.group_info
         return get_splitter(data, comp_ids, ngroups, axis=axis)
 
+    def _get_grouper(self):
+        """
+        We are a grouper as part of another's groupings.
+
+        We have a specific method of grouping, so cannot
+        convert to a Index for our grouper.
+        """
+        return self.groupings[0].grouper
+
     def _get_group_keys(self):
         if len(self.groupings) == 1:
             return self.levels[0]
@@ -707,6 +716,15 @@ def groups(self):
     def nkeys(self):
         return 1
 
+    def _get_grouper(self):
+        """
+        We are a grouper as part of another's groupings.
+
+        We have a specific method of grouping, so cannot
+        convert to a Index for our grouper.
+        """
+        return self
+
     def get_iterator(self, data, axis=0):
         """
         Groupby iterator


Test patch:
diff --git a/pandas/tests/groupby/test_groupby.py b/pandas/tests/groupby/test_groupby.py
--- a/pandas/tests/groupby/test_groupby.py
+++ b/pandas/tests/groupby/test_groupby.py
@@ -1736,3 +1736,19 @@ def test_groupby_multiindex_series_keys_len_equal_group_axis():
     expected = pd.Series([3], index=ei)
 
     assert_series_equal(result, expected)
+
+
+def test_groupby_groups_in_BaseGrouper():
+    # GH 26326
+    # Test if DataFrame grouped with a pandas.Grouper has correct groups
+    mi = pd.MultiIndex.from_product([['A', 'B'],
+                                     ['C', 'D']], names=['alpha', 'beta'])
+    df = pd.DataFrame({'foo': [1, 2, 1, 2], 'bar': [1, 2, 3, 4]},
+                      index=mi)
+    result = df.groupby([pd.Grouper(level='alpha'), 'beta'])
+    expected = df.groupby(['alpha', 'beta'])
+    assert(result.groups == expected.groups)
+
+    result = df.groupby(['beta', pd.Grouper(level='alpha')])
+    expected = df.groupby(['beta', 'alpha'])
+    assert(result.groups == expected.groups)