Repository: Textualize/rich (python)

Issue:
[BUG] repr fails for pd.Series in Jupyter with pretty.install()
**Describe the bug**

![image](https://user-images.githubusercontent.com/1360812/143245956-2ef17723-ddd0-487e-ad2a-11402ba58c1d.png)

Any pd.Series does not appear in Jupyter (both Lab and Notebook) after `pretty.install()`, but appears with `pretty.pprint()`.
This is not the case with other pandas structures.

**To Reproduce**

For copy paste purposes:

```python
from rich import pretty
pretty.install()

import pandas as pd
df = pd.DataFrame.from_records([{'a': 1, 'b': 1}, {'a': 2, 'b': 2}])
df.a  # no output

pretty.pprint(df.a)  # outputs fine
```

```
python -m rich.diagnose
python -m rich._windows
pip freeze | grep rich
```

```
╭─────────────────────── <class 'rich.console.Console'> ───────────────────────╮
│ A high level console interface.                                              │
│                                                                              │
│ ╭──────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=80 ColorSystem.TRUECOLOR>                                 │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│     color_system = 'truecolor'                                               │
│         encoding = 'utf-8'                                                   │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w'               │
│                    encoding='utf-8'>                                         │
│           height = 24                                                        │
│    is_alt_screen = False                                                     │
│ is_dumb_terminal = False                                                     │
│   is_interactive = True                                                      │
│       is_jupyter = False                                                     │
│      is_terminal = True                                                      │
│   legacy_windows = False                                                     │
│         no_color = False                                                     │
│          options = ConsoleOptions(                                           │
│                        size=ConsoleDimensions(width=80, height=24),          │
│                        legacy_windows=False,                                 │
│                        min_width=1,                                          │
│                        max_width=80,                                         │
│                        is_terminal=True,                                     │
│                        encoding='utf-8',                                     │
│                        max_height=24,                                        │
│                        justify=None,                                         │
│                        overflow=None,                                        │
│                        no_wrap=False,                                        │
│                        highlight=None,                                       │
│                        markup=None,                                          │
│                        height=None                                           │
│                    )                                                         │
│            quiet = False                                                     │
│           record = False                                                     │
│         safe_box = True                                                      │
│             size = ConsoleDimensions(width=80, height=24)                    │
│        soft_wrap = False                                                     │
│           stderr = False                                                     │
│            style = None                                                      │
│         tab_size = 8                                                         │
│            width = 80                                                        │
╰──────────────────────────────────────────────────────────────────────────────╯
platform="Linux"
WindowsConsoleFeatures(vt=False, truecolor=False)
rich==10.14.0
```


Pull request:
pretty.install() fix for some objects with `_repr_*_`
## Type of changes

- [x] Bug fix - Fixes #1717 
- [ ] New feature
- [ ] Documentation / docstrings
- [ ] Tests
- [ ] Other

## Checklist

- [x] I've run the latest [black](https://github.com/psf/black) with default args on new code.
- [x] I've updated CHANGELOG.md and CONTRIBUTORS.md where appropriate.
- [ ] I've added tests for new code.
- [x] I accept that @willmcgugan may be pedantic in the code review.

## Description

The issue is that Rich assumes that if a `_repr_*_` method exists on an object, then notebook will definitely render that object. 

However, notebook only supports a subset of mimetypes, and even in the case of supported mimetypes, some libraries (e.g. `pandas`) support toggling off functionality, which results in the `_repr_*_` for that mimetype returning `None`. 

For example, `pd.Series` has a `_repr_latex_` method, but by default the config for it (`"display.latex.repr"`) is switched off and so `pd.Series._repr_latex_` returns `None`. 

We need something a little finer-grained and more aware of the notebook capabilities.

For all of the mimetypes that Jupyter notebook can display, we check if the corresponding `_repr_*_` special method exists and ensure it doesn't return `None`. If it returns `None`, Rich will print it, otherwise the notebook will render it.


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a5f50c90..d04e7b3c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,10 +5,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+
 ## [11.0.0] - Unreleased
 
 ### Added
 
+- Fixed issue with pretty repr in jupyter notebook https://github.com/Textualize/rich/issues/1717
 - Added max_depth arg to pretty printing
 
 ## [10.16.2] - 2021-01-02
@@ -1591,6 +1593,7 @@ Major version bump for a breaking change to `Text.stylize signature`, which corr
 
 - First official release, API still to be stabilized
 
+[11.0.0]: https://github.com/willmcgugan/rich/compare/v10.16.1...v11.0.0
 [10.16.1]: https://github.com/willmcgugan/rich/compare/v10.16.0...v10.16.1
 [10.16.0]: https://github.com/willmcgugan/rich/compare/v10.15.2...v10.16.0
 [10.15.2]: https://github.com/willmcgugan/rich/compare/v10.15.1...v10.15.2
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 5aaf8776..c29cd790 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -5,6 +5,7 @@ The following people have contributed to the development of Rich:
 <!-- Add your name below, sort alphabetically by surname. Link to Github profile / your home page. -->
 
 - [Gregory Beauregard](https://github.com/GBeauregard/pyffstream)
+- [Darren Burns](https://github.com/darrenburns)
 - [Pete Davison](https://github.com/pd93)
 - [James Estevez](https://github.com/jstvz)
 - [Oleksis Fraga](https://github.com/oleksis)
diff --git a/rich/jupyter.py b/rich/jupyter.py
index 7cdcc9ca..bedf5cb1 100644
--- a/rich/jupyter.py
+++ b/rich/jupyter.py
@@ -4,7 +4,6 @@ from . import get_console
 from .segment import Segment
 from .terminal_theme import DEFAULT_TERMINAL_THEME
 
-
 JUPYTER_HTML_FORMAT = """\
 <pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">{code}</pre>
 """
@@ -75,11 +74,16 @@ def _render_segments(segments: Iterable[Segment]) -> str:
 
 def display(segments: Iterable[Segment], text: str) -> None:
     """Render segments to Jupyter."""
-    from IPython.display import display as ipython_display
-
     html = _render_segments(segments)
     jupyter_renderable = JupyterRenderable(html, text)
-    ipython_display(jupyter_renderable)
+    try:
+        from IPython.display import display as ipython_display
+
+        ipython_display(jupyter_renderable)
+    except ModuleNotFoundError:
+        # Handle the case where the Console has force_jupyter=True,
+        # but IPython is not installed.
+        pass
 
 
 def print(*args: Any, **kwargs: Any) -> None:
diff --git a/rich/pretty.py b/rich/pretty.py
index 10cc2925..57e743df 100644
--- a/rich/pretty.py
+++ b/rich/pretty.py
@@ -1,28 +1,30 @@
 import builtins
+import dataclasses
+import inspect
 import os
-from rich.repr import RichReprResult
+import re
 import sys
 from array import array
-from collections import Counter, defaultdict, deque, UserDict, UserList
-import dataclasses
+from collections import Counter, UserDict, UserList, defaultdict, deque
 from dataclasses import dataclass, fields, is_dataclass
 from inspect import isclass
 from itertools import islice
-import re
+from types import MappingProxyType
 from typing import (
-    DefaultDict,
     TYPE_CHECKING,
     Any,
     Callable,
+    DefaultDict,
     Dict,
     Iterable,
     List,
     Optional,
     Set,
-    Union,
     Tuple,
+    Union,
 )
-from types import MappingProxyType
+
+from rich.repr import RichReprResult
 
 try:
     import attr as _attr_module
@@ -30,7 +32,6 @@ except ImportError:  # pragma: no cover
     _attr_module = None  # type: ignore
 
 
-from .highlighter import ReprHighlighter
 from . import get_console
 from ._loop import loop_last
 from ._pick import pick_bool
@@ -51,9 +52,6 @@ if TYPE_CHECKING:
         RenderResult,
     )
 
-# Matches Jupyter's special methods
-_re_jupyter_repr = re.compile(f"^_repr_.+_$")
-
 
 def _is_attr_object(obj: Any) -> bool:
     """Check if an object was created with attrs module."""
@@ -82,6 +80,70 @@ def _is_dataclass_repr(obj: object) -> bool:
         return False
 
 
+def _ipy_display_hook(
+    value: Any,
+    console: Optional["Console"] = None,
+    overflow: "OverflowMethod" = "ignore",
+    crop: bool = False,
+    indent_guides: bool = False,
+    max_length: Optional[int] = None,
+    max_string: Optional[int] = None,
+    expand_all: bool = False,
+) -> None:
+    from .console import ConsoleRenderable  # needed here to prevent circular import
+
+    # always skip rich generated jupyter renderables or None values
+    if isinstance(value, JupyterRenderable) or value is None:
+        return
+
+    console = console or get_console()
+    if console.is_jupyter:
+        # Delegate rendering to IPython if the object (and IPython) supports it
+        #  https://ipython.readthedocs.io/en/stable/config/integrating.html#rich-display
+        ipython_repr_methods = [
+            "_repr_html_",
+            "_repr_markdown_",
+            "_repr_json_",
+            "_repr_latex_",
+            "_repr_jpeg_",
+            "_repr_png_",
+            "_repr_svg_",
+            "_repr_mimebundle_",
+        ]
+        for repr_method in ipython_repr_methods:
+            method = getattr(value, repr_method, None)
+            if inspect.ismethod(method):
+                # Calling the method ourselves isn't ideal. The interface for the `_repr_*_` methods
+                #  specifies that if they return None, then they should not be rendered
+                #  by the notebook.
+                try:
+                    repr_result = method()
+                except Exception:
+                    continue  # If the method raises, treat it as if it doesn't exist, try any others
+                if repr_result is not None:
+                    return  # Delegate rendering to IPython
+
+    # certain renderables should start on a new line
+    if isinstance(value, ConsoleRenderable):
+        console.line()
+
+    console.print(
+        value
+        if isinstance(value, RichRenderable)
+        else Pretty(
+            value,
+            overflow=overflow,
+            indent_guides=indent_guides,
+            max_length=max_length,
+            max_string=max_string,
+            expand_all=expand_all,
+            margin=12,
+        ),
+        crop=crop,
+        new_line_start=True,
+    )
+
+
 def install(
     console: Optional["Console"] = None,
     overflow: "OverflowMethod" = "ignore",
@@ -106,8 +168,6 @@ def install(
     """
     from rich import get_console
 
-    from .console import ConsoleRenderable  # needed here to prevent circular import
-
     console = console or get_console()
     assert console is not None
 
@@ -131,37 +191,6 @@ def install(
             )
             builtins._ = value  # type: ignore
 
-    def ipy_display_hook(value: Any) -> None:  # pragma: no cover
-        assert console is not None
-        # always skip rich generated jupyter renderables or None values
-        if isinstance(value, JupyterRenderable) or value is None:
-            return
-        # on jupyter rich display, if using one of the special representations don't use rich
-        if console.is_jupyter and any(
-            _re_jupyter_repr.match(attr) for attr in dir(value)
-        ):
-            return
-
-        # certain renderables should start on a new line
-        if isinstance(value, ConsoleRenderable):
-            console.line()
-
-        console.print(
-            value
-            if isinstance(value, RichRenderable)
-            else Pretty(
-                value,
-                overflow=overflow,
-                indent_guides=indent_guides,
-                max_length=max_length,
-                max_string=max_string,
-                expand_all=expand_all,
-                margin=12,
-            ),
-            crop=crop,
-            new_line_start=True,
-        )
-
     try:  # pragma: no cover
         ip = get_ipython()  # type: ignore
         from IPython.core.formatters import BaseFormatter
@@ -171,7 +200,15 @@ def install(
 
             def __call__(self, value: Any) -> Any:
                 if self.pprint:
-                    return ipy_display_hook(value)
+                    return _ipy_display_hook(
+                        value,
+                        console=get_console(),
+                        overflow=overflow,
+                        indent_guides=indent_guides,
+                        max_length=max_length,
+                        max_string=max_string,
+                        expand_all=expand_all,
+                    )
                 else:
                     return repr(value)
 


Test patch:
diff --git a/tests/test_pretty.py b/tests/test_pretty.py
index f5753fd2..871f1397 100644
--- a/tests/test_pretty.py
+++ b/tests/test_pretty.py
@@ -1,16 +1,16 @@
-from array import array
-from collections import defaultdict, UserDict, UserList
-from dataclasses import dataclass, field
 import io
 import sys
+from array import array
+from collections import defaultdict, UserDict
 from typing import List
 
 import attr
 import pytest
+from dataclasses import dataclass, field
 
 from rich.console import Console
-from rich.pretty import install, Pretty, pprint, pretty_repr, Node
-
+from rich.pretty import install, Pretty, pprint, pretty_repr, Node, _ipy_display_hook
+from rich.text import Text
 
 skip_py36 = pytest.mark.skipif(
     sys.version_info.minor == 6 and sys.version_info.major == 3,
@@ -44,6 +44,85 @@ def test_install():
     assert sys.displayhook is not dh
 
 
+def test_ipy_display_hook__repr_html():
+    console = Console(file=io.StringIO(), force_jupyter=True)
+
+    class Thing:
+        def _repr_html_(self):
+            return "hello"
+
+    console.begin_capture()
+    _ipy_display_hook(Thing(), console=console)
+
+    # Rendering delegated to notebook because _repr_html_ method exists
+    assert console.end_capture() == ""
+
+
+def test_ipy_display_hook__multiple_special_reprs():
+    """
+    The case where there are multiple IPython special _repr_*_
+    methods on the object, and one of them returns None but another
+    one does not.
+    """
+    console = Console(file=io.StringIO(), force_jupyter=True)
+
+    class Thing:
+        def _repr_latex_(self):
+            return None
+
+        def _repr_html_(self):
+            return "hello"
+
+    console.begin_capture()
+    _ipy_display_hook(Thing(), console=console)
+
+    assert console.end_capture() == ""
+
+
+def test_ipy_display_hook__no_special_repr_methods():
+    console = Console(file=io.StringIO(), force_jupyter=True)
+
+    class Thing:
+        def __repr__(self) -> str:
+            return "hello"
+
+    console.begin_capture()
+    _ipy_display_hook(Thing(), console=console)
+
+    # No IPython special repr methods, so printed by Rich
+    assert console.end_capture() == "hello\n"
+
+
+def test_ipy_display_hook__special_repr_raises_exception():
+    """
+    When an IPython special repr method raises an exception,
+    we treat it as if it doesn't exist and look for the next.
+    """
+    console = Console(file=io.StringIO(), force_jupyter=True)
+
+    class Thing:
+        def _repr_markdown_(self):
+            raise Exception()
+
+        def _repr_latex_(self):
+            return None
+
+        def _repr_html_(self):
+            return "hello"
+
+    console.begin_capture()
+    _ipy_display_hook(Thing(), console=console)
+
+    assert console.end_capture() == ""
+
+
+def test_ipy_display_hook__console_renderables_on_newline():
+    console = Console(file=io.StringIO(), force_jupyter=True)
+    console.begin_capture()
+    _ipy_display_hook(Text("hello"), console=console)
+    assert console.end_capture() == "\nhello\n"
+
+
 def test_pretty():
     test = {
         "foo": [1, 2, 3, (4, 5, {6}, 7, 8, {9}), {}],
@@ -55,7 +134,6 @@ def test_pretty():
 
     result = pretty_repr(test, max_width=80)
     print(result)
-    # print(repr(result))
     expected = "{\n    'foo': [1, 2, 3, (4, 5, {6}, 7, 8, {9}), {}],\n    'bar': {\n        'egg': 'baz',\n        'words': [\n            'Hello World',\n            'Hello World',\n            'Hello World',\n            'Hello World',\n            'Hello World',\n            'Hello World',\n            'Hello World',\n            'Hello World',\n            'Hello World',\n            'Hello World'\n        ]\n    },\n    False: 'foo',\n    True: '',\n    'text': ('Hello World', 'foo bar baz egg')\n}"
     print(expected)
     assert result == expected