Repository: pandas-dev/pandas
Pull request: ENH: truncate output of Groupby.groups
Resurrection of #24853. Have resolved conflicts and updated according to the last review it received. The original PR fixed some unrelated minor typos, so I've kept them in (and added a fix for another absolutely minor typo I'd noticed)

- [x] closes #1135
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
df.groupby('key').groups printed all: problem with large arrays
When you are working with a large array, it is not printed out in its entirety in the console, but when you have grouped them with `groupby('key')`, the groups are all printed out. Would it be possible to also restrict the output of `groupby('key').groups` to eg the first and last groups?

I was working with a rather large dataframe (around 80000 rows), and first it took a long time to print it all, and second the console got stuck for a while (but that could also be an issue with spyder).
I know it is not very useful to print out the groups, but I was curious to see what it would look like, but it was not a very good idea with such a large array.


```python
>>> import numpy as np
>>> import pandas as pd
>>> df = pd.DataFrame(np.random.randn(100000, 4), columns=list('abcd'))
>>> df['g'] = np.random.randint(0, 100, 100000)
>>> df.groupby('g').groups  # this repr should be truncated
```

Hints from issue discussion:
@jorisvandenbossche hah...this issue is from u 2 years ago! 

a good idea!

ah yes :-) long time ago. I think my first one!

I was hoping to take a shot at this. Does anyone have a recommended approach? `groups` gives a dict, so the dict repr is used. I'm not sure how to override that without defining a new class and having `groups` return that type instead.
i think u can use pprint: https://docs.python.org/3/library/pprint.html

use compact=True
I found `pandas.io.formats.printing.pprint_thing` which does something similar to dataframe truncation:
```python
>>> from pandas.io.formats.printing import pprint_thing
>>> z = [list('abc') for _ in range(100)]
>>> print(pprint_thing(z, max_seq_items=3))
[[a, b, c], [a, b, c], [a, b, c], ...]
```
However it truncates at the end rather than in the middle, which doesn't match df. Maybe `pprint_thing` could take another arg for where to truncate.

Also, `groups` currently returns a dict. I could subclass and override the `__repr__` to accomplish this, but the return type of `groups` would have to change. Perhaps there's a cleaner trick I'm not aware of? :smile: 
Would it be better to implement as I've described above and discuss further in the PR?
@jreback actually it appears [#2628](https://github.com/pandas-dev/pandas/issues/2628) is a duplicate. A similar approach has been suggested there.

Commit message: rewrite tests, typing

Patch to review:
diff --git a/doc/source/development/contributing.rst b/doc/source/development/contributing.rst
index a9237c239701b..f904781178656 100644
--- a/doc/source/development/contributing.rst
+++ b/doc/source/development/contributing.rst
@@ -32,7 +32,7 @@ check each issue individually, and it's not possible to find the unassigned ones
 
 For this reason, we implemented a workaround consisting of adding a comment with the exact
 text `take`. When you do it, a GitHub action will automatically assign you the issue
-(this will take seconds, and may require refreshint the page to see it).
+(this will take seconds, and may require refreshing the page to see it).
 By doing this, it's possible to filter the list of issues and find only the unassigned ones.
 
 So, a good way to find an issue to start contributing to pandas is to check the list of
diff --git a/doc/source/whatsnew/v1.1.0.rst b/doc/source/whatsnew/v1.1.0.rst
index 920919755dc23..327a099259b75 100644
--- a/doc/source/whatsnew/v1.1.0.rst
+++ b/doc/source/whatsnew/v1.1.0.rst
@@ -54,7 +54,7 @@ Other API changes
 
 - :meth:`Series.describe` will now show distribution percentiles for ``datetime`` dtypes, statistics ``first`` and ``last``
   will now be ``min`` and ``max`` to match with numeric dtypes in :meth:`DataFrame.describe` (:issue:`30164`)
--
+- :meth:`Groupby.groups` now returns an abbreviated representation when called on large dataframes (:issue:`1135`)
 
 Backwards incompatible API changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
index f3a0cf3841b5b..68c5809479b70 100644
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -607,7 +607,7 @@ def _repr_fits_horizontal_(self, ignore_width: bool = False) -> bool:
         Check if full repr fits in horizontal boundaries imposed by the display
         options width and max_columns.
 
-        In case off non-interactive session, no boundaries apply.
+        In case of non-interactive session, no boundaries apply.
 
         `ignore_width` is here so ipnb+HTML output can behave the way
         users expect. display.max_columns remains in effect.
diff --git a/pandas/core/groupby/groupby.py b/pandas/core/groupby/groupby.py
index aa21aa452be95..07967e5dad9ec 100644
--- a/pandas/core/groupby/groupby.py
+++ b/pandas/core/groupby/groupby.py
@@ -72,7 +72,7 @@ class providing the base-class of operations.
 
 _apply_docs = dict(
     template="""
-    Apply function `func`  group-wise and combine the results together.
+    Apply function `func` group-wise and combine the results together.
 
     The function passed to `apply` must take a {input} as its first
     argument and return a DataFrame, Series or scalar. `apply` will
diff --git a/pandas/core/groupby/ops.py b/pandas/core/groupby/ops.py
index 679d3668523c2..5e13779608dd6 100644
--- a/pandas/core/groupby/ops.py
+++ b/pandas/core/groupby/ops.py
@@ -350,7 +350,7 @@ def get_group_levels(self):
 
     def _is_builtin_func(self, arg):
         """
-        if we define an builtin function for this argument, return it,
+        if we define a builtin function for this argument, return it,
         otherwise return the arg
         """
         return SelectionMixin._builtin_table.get(arg, arg)
diff --git a/pandas/core/indexes/base.py b/pandas/core/indexes/base.py
index 10d9552e6f5a7..3f3684a5bb059 100644
--- a/pandas/core/indexes/base.py
+++ b/pandas/core/indexes/base.py
@@ -1,7 +1,7 @@
 from datetime import datetime
 import operator
 from textwrap import dedent
-from typing import Any, Dict, FrozenSet, Hashable, Optional, Union
+from typing import Any, FrozenSet, Hashable, Optional, Union
 import warnings
 
 import numpy as np
@@ -75,6 +75,7 @@
 from pandas.core.strings import StringMethods
 
 from pandas.io.formats.printing import (
+    PrettyDict,
     default_pprint,
     format_object_attrs,
     format_object_summary,
@@ -4765,7 +4766,7 @@ def _maybe_promote(self, other):
                 return self.astype("object"), other.astype("object")
         return self, other
 
-    def groupby(self, values) -> Dict[Hashable, np.ndarray]:
+    def groupby(self, values) -> PrettyDict:
         """
         Group the index labels by a given array of values.
 
@@ -4790,7 +4791,7 @@ def groupby(self, values) -> Dict[Hashable, np.ndarray]:
         # map to the label
         result = {k: self.take(v) for k, v in result.items()}
 
-        return result
+        return PrettyDict(result)
 
     def map(self, mapper, na_action=None):
         """
diff --git a/pandas/io/formats/printing.py b/pandas/io/formats/printing.py
index 4b5b5e9a0ce15..ff03c8127ee19 100644
--- a/pandas/io/formats/printing.py
+++ b/pandas/io/formats/printing.py
@@ -6,12 +6,14 @@
 from typing import (
     Any,
     Callable,
+    Dict,
     Iterable,
     List,
     Mapping,
     Optional,
     Sequence,
     Tuple,
+    TypeVar,
     Union,
 )
 
@@ -20,6 +22,8 @@
 from pandas.core.dtypes.inference import is_sequence
 
 EscapeChars = Union[Mapping[str, str], Iterable[str]]
+_KT = TypeVar("_KT")
+_VT = TypeVar("_VT")
 
 
 def adjoin(space: int, *lists: List[str], **kwargs) -> str:
@@ -98,7 +102,7 @@ def _pprint_seq(
 ) -> str:
     """
     internal. pprinter for iterables. you should probably use pprint_thing()
-    rather then calling this directly.
+    rather than calling this directly.
 
     bounds length of printed sequence, depending on options
     """
@@ -133,7 +137,7 @@ def _pprint_dict(
 ) -> str:
     """
     internal. pprinter for iterables. you should probably use pprint_thing()
-    rather then calling this directly.
+    rather than calling this directly.
     """
     fmt = "{{{things}}}"
     pairs = []
@@ -528,3 +532,10 @@ def format_object_attrs(
     if len(obj) > max_seq_items:
         attrs.append(("length", len(obj)))
     return attrs
+
+
+class PrettyDict(Dict[_KT, _VT]):
+    """Dict extension to support abbreviated __repr__"""
+
+    def __repr__(self) -> str:
+        return pprint_thing(self, max_seq_items=get_option("display.max_rows"))
diff --git a/pandas/tests/groupby/test_groupby.py b/pandas/tests/groupby/test_groupby.py
index eb9552fbbebc1..3921990d1b3ad 100644
--- a/pandas/tests/groupby/test_groupby.py
+++ b/pandas/tests/groupby/test_groupby.py
@@ -2037,3 +2037,16 @@ def test_groupby_list_level():
     expected = pd.DataFrame(np.arange(0, 9).reshape(3, 3))
     result = expected.groupby(level=[0]).mean()
     tm.assert_frame_equal(result, expected)
+
+
+def test_groups_repr_truncates():
+    # GH 1135
+    df = pd.DataFrame(np.random.randn(61, 1))
+    df["a"] = df.index
+
+    result = df.groupby("a").groups.__repr__()
+    expected = str({i: [i] for i in range(60)})[:-1] + ", ...}"
+    assert result == expected
+
+    result = df.groupby(np.array(df.a)).groups.__repr__()
+    assert result == expected


Code review comments:
diff_hunk: @@ -2037,3 +2037,16 @@ def test_groupby_list_level():
     expected = pd.DataFrame(np.arange(0, 9).reshape(3, 3))
     result = expected.groupby(level=[0]).mean()
     tm.assert_frame_equal(result, expected)
+
+
+def test_groups_repr_truncates():
+    # GH 1135
+    df = pd.DataFrame({"a": [1, 1, 1, 2, 2, 3], "b": [1, 2, 3, 4, 5, 6]})
+
+    with pd.option_context("display.max_rows", 2):
+        x = df.groupby("a").groups
path: pandas/tests/groupby/test_groupby.py
text: @user1:
use `with pd.option_context("display.max_seq_items", 2):` and just `return pprint_thing(self)`  from `repr` instead?




@author:
Thanks for your review - I've reread this sentence a few times

>  just return pprint_thing(self) from repr instead

but I don't understand what you mean. If I change the definition of `PrettyDict.__repr__` to
```
        return pprint_thing(self)
```
then won't the issue be not-solved? At least, with the test as it's currently written, it fails

@user1:
I thought pprint_thing called_pprint_dict which uses display.max_seq_items as the default. I'll check in a bit.

@user1:
> I thought pprint_thing called_pprint_dict which uses display.max_seq_items as the default. I'll check in a bit.

confirmed.

> then won't the issue be not-solved?

if the top level of the dict **must** follow display.max_rows for truncation, then maybe updating pprint_thing to accept, say, a tuple for max_seq_items to allow the nested sequences to follow display.max_seq_items could be an option. The tuple could be any length that specifies the max_seq_items for each level with the last value being used for deeper sequences.

so something like `return pprint_thing(self, max_seq_items=(get_option("display.max_rows"), get_option("display.max_seq_items"))` ?

or maybe alter the \_\_repr__ to be less dict-like

diff_hunk: @@ -4765,7 +4766,7 @@ def _maybe_promote(self, other):
                 return self.astype("object"), other.astype("object")
         return self, other
 
-    def groupby(self, values) -> Dict[Hashable, np.ndarray]:
+    def groupby(self, values) -> PrettyDict:
path: pandas/core/indexes/base.py
text: @user1:
now PrettyDict is generic can do
```suggestion
    def groupby(self, values) -> PrettyDict[Hashable, np.ndarray]:
```