Repository: napari/napari

Issue:
[testing] test_qt_viewer_with_console breaks test_run_outside_ipython
## 🐛 Bug

Something about `napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console` is leaking and breaking `napari/_qt/_tests/test_app.py::test_run_outside_ipython`. I discovered this while trying to merge `main` into #4977, which has a new test that causes a similar problem. I think the issue goes unnoticed because it depends on execution order, which by default is sufficient to prevent the failure with existing tests.

This has something to do with IPython being imported/connected when opening the console and launching an in-process kernel. I have not found a way to shutdown the interactive IPython and clean up completely after this happens.

A workaround is to patch napari-console in tests such that toggling the widget does not start a new IPython kernel. This works in my testing and feels like a reasonable solution to me, but I would appreciate feedback. This relies on some internal implementation in napari-console so could be fragile if there are updates.

```python
@pytest.fixture()
def mock_console():
    """Mock the qtconsole to avoid starting an interactive IPython session.
    In-process IPython kernels can interfere with other tests and are difficult
    (impossible?) to shutdown.
    """
    from napari_console import QtConsole
    from qtconsole.rich_jupyter_widget import RichJupyterWidget

    class FakeQtConsole(RichJupyterWidget):
        def __init__(self, viewer: Viewer):
            super().__init__()
            self.viewer = viewer
            self.kernel_client = None
            self.kernel_manager = None

        _update_theme = Mock()
        push = Mock()
        closeEvent = QtConsole.closeEvent

    with patch("napari_console.QtConsole", FakeQtConsole):
        yield
```

## To Reproduce
Run `napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console` before `napari/_qt/_tests/test_app.py::test_run_outside_ipython` in the same pytest session.

#### Successful test run (`test_run_outside_ipython` before `test_qt_viewer_with_console`)

```
napari on  main [$?] via 🐍 v3.10.10 via 🅒 napari took 3s 
❯ pytest -W ignore -v napari/_qt/_tests/test_app.py::test_run_outside_ipython napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console 
======================================== test session starts ========================================
platform darwin -- Python 3.10.10, pytest-7.3.1, pluggy-1.0.0 -- /Users/aandersoniii/mf/envs/napari/bin/python3.10
cachedir: .pytest_cache
PyQt5 5.15.7 -- Qt runtime 5.15.8 -- Qt compiled 5.15.6
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/Users/aandersoniii/src/napari/.hypothesis/examples')
rootdir: /Users/aandersoniii/src/napari
configfile: pyproject.toml
plugins: npe2-0.7.0, qt-4.2.0, napari-plugin-engine-0.2.0, napari-0.5.0a2.dev112+g289367ca7.d20230420, cov-4.0.0, anyio-3.6.2, hypothesis-6.72.1, pretty-1.2.0
collected 2 items                                                                                   

napari/_qt/_tests/test_app.py::test_run_outside_ipython PASSED                                [ 50%]
napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console PASSED                       [100%]

======================================= slowest 10 durations ========================================
0.93s call     napari/_qt/_tests/test_app.py::test_run_outside_ipython
0.50s call     napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console
0.13s setup    napari/_qt/_tests/test_app.py::test_run_outside_ipython
0.04s setup    napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console
0.04s teardown napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console

(1 durations < 0.005s hidden.  Use -vv to show these durations.)
```

#### Unsuccessful test run (`test_run_outside_ipython` after `test_qt_viewer_with_console`)
  ```
  napari on  main [$?] via 🐍 v3.10.10 via 🅒 napari took 3s 
  ❯ pytest -W ignore -v napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console napari/_qt/_tests/test_app.py::test_run_outside_ipython
  ======================================== test session starts ========================================
  platform darwin -- Python 3.10.10, pytest-7.3.1, pluggy-1.0.0 -- /Users/aandersoniii/mf/envs/napari/bin/python3.10
  cachedir: .pytest_cache
  PyQt5 5.15.7 -- Qt runtime 5.15.8 -- Qt compiled 5.15.6
  hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/Users/aandersoniii/src/napari/.hypothesis/examples')
  rootdir: /Users/aandersoniii/src/napari
  configfile: pyproject.toml
  plugins: npe2-0.7.0, qt-4.2.0, napari-plugin-engine-0.2.0, napari-0.5.0a2.dev112+g289367ca7.d20230420, cov-4.0.0, anyio-3.6.2, hypothesis-6.72.1, pretty-1.2.0
  collected 2 items                                                                                   
  
  napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console PASSED                       [ 50%]
  napari/_qt/_tests/test_app.py::test_run_outside_ipython FAILED                                [100%] [100%]
  napari/_qt/_tests/test_app.py:33 test_run_outside_ipython - assert not True  +  where Tru…
  ============================================= FAILURES ==============================================
  _____________________________________ test_run_outside_ipython ______________________________________
  
  qapp = <PyQt5.QtWidgets.QApplication object at 0x137e48940>
  monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x1503ac220>
  
      def test_run_outside_ipython(qapp, monkeypatch):
          """Test that we don't incorrectly give ipython the event loop."""
          assert not _ipython_has_eventloop()
          v1 = Viewer(show=False)
  >       assert not _ipython_has_eventloop()
  E       assert not True
  E        +  where True = _ipython_has_eventloop()
  
  napari/_qt/_tests/test_app.py:38: AssertionError
  ======================================= slowest 10 durations ========================================
  0.97s call     napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console
  0.28s call     napari/_qt/_tests/test_app.py::test_run_outside_ipython
  0.17s setup    napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console
  0.03s teardown napari/_qt/_tests/test_qt_viewer.py::test_qt_viewer_with_console
  0.01s teardown napari/_qt/_tests/test_app.py::test_run_outside_ipython
  
  (1 durations < 0.005s hidden.  Use -vv to show these durations.)
                                           Summary of Failures                                         
  ┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
  ┃  File                 ┃  Function             ┃  Function Line  ┃  Error Line  ┃  Error           ┃
  ┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
  │  napari/_qt/_tests/…  │  test_run_outside_i…  │  34             │  38          │  AssertionError  │
  └───────────────────────┴───────────────────────┴─────────────────┴──────────────┴──────────────────┘
  Results (1.84s):
           1 failed
           1 passed
  ```

## Expected behavior

Tests should pass regardless of execution order.

## Environment
<details>
<summary>Environment info</summary>

```
napari: 0.5.0a2.dev138+g350e602de.d20230525
Platform: macOS-13.3.1-arm64-arm-64bit
System: MacOS 13.3.1
Python: 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Qt: 5.15.6
PyQt5: 5.15.7
NumPy: 1.23.5
SciPy: 1.10.1
Dask: 2023.4.1
VisPy: 0.13.0
magicgui: 0.7.2
superqt: unknown
in-n-out: 0.1.7
app-model: 0.1.4
npe2: 0.7.0

OpenGL:
- GL version: 2.1 Metal - 83.1
- MAX_TEXTURE_SIZE: 16384

Screens:
- screen 1: resolution 2560x1440, scale 1.0

Settings path:
- /Users/aandersoniii/Library/Application Support/napari/napari_3633aa9ab90a4ced489f2fa16034f61e4ad37f89/settings.yaml
```
</details>



Discussion:

Resolving patch:
diff --git a/napari/conftest.py b/napari/conftest.py
--- a/napari/conftest.py
+++ b/napari/conftest.py
@@ -38,7 +38,7 @@ def get_reader(path):
 from itertools import chain
 from multiprocessing.pool import ThreadPool
 from typing import TYPE_CHECKING
-from unittest.mock import patch
+from unittest.mock import Mock, patch
 from weakref import WeakKeyDictionary
 
 with suppress(ModuleNotFoundError):
@@ -53,6 +53,7 @@ def get_reader(path):
 from napari.components import LayerList
 from napari.layers import Image, Labels, Points, Shapes, Vectors
 from napari.utils.misc import ROOT_DIR
+from napari.viewer import Viewer
 
 if TYPE_CHECKING:
     from npe2._pytest_plugin import TestPluginManager
@@ -380,6 +381,38 @@ def single_threaded_executor():
     executor.shutdown()
 
 
+@pytest.fixture()
+def mock_console(request):
+    """Mock the qtconsole to avoid starting an interactive IPython session.
+    In-process IPython kernels can interfere with other tests and are difficult
+    (impossible?) to shutdown.
+
+    This fixture is configured to be applied automatically to tests unless they
+    use the `enable_console` marker. It's not autouse to avoid use on headless
+    tests (without Qt); instead it's enabled in `pytest_runtest_setup`.
+    """
+    if "enable_console" in request.keywords:
+        yield
+        return
+
+    from napari_console import QtConsole
+    from qtconsole.rich_jupyter_widget import RichJupyterWidget
+
+    class FakeQtConsole(RichJupyterWidget):
+        def __init__(self, viewer: Viewer):
+            super().__init__()
+            self.viewer = viewer
+            self.kernel_client = None
+            self.kernel_manager = None
+
+        _update_theme = Mock()
+        push = Mock()
+        closeEvent = QtConsole.closeEvent
+
+    with patch("napari_console.QtConsole", FakeQtConsole):
+        yield
+
+
 @pytest.fixture(autouse=True)
 def _mock_app():
     """Mock clean 'test_app' `NapariApplication` instance.
@@ -716,5 +749,6 @@ def pytest_runtest_setup(item):
                 "dangling_qanimations",
                 "dangling_qthreads",
                 "dangling_qtimers",
+                "mock_console",
             ]
         )


Test patch:
diff --git a/napari/_tests/test_advanced.py b/napari/_tests/test_advanced.py
--- a/napari/_tests/test_advanced.py
+++ b/napari/_tests/test_advanced.py
@@ -157,6 +157,7 @@ def test_range_one_images_and_points(make_napari_viewer):
     assert np.sum(view.dims._displayed_sliders) == 3
 
 
+@pytest.mark.enable_console
 @pytest.mark.filterwarnings("ignore::DeprecationWarning:jupyter_client")
 def test_update_console(make_napari_viewer):
     """Test updating the console with local variables."""
@@ -180,6 +181,7 @@ def test_update_console(make_napari_viewer):
         del viewer.window._qt_viewer.console.shell.user_ns[k]
 
 
+@pytest.mark.enable_console
 @pytest.mark.filterwarnings("ignore::DeprecationWarning:jupyter_client")
 def test_update_lazy_console(make_napari_viewer, capsys):
     """Test updating the console with local variables,
diff --git a/napari/_tests/test_conftest_fixtures.py b/napari/_tests/test_conftest_fixtures.py
--- a/napari/_tests/test_conftest_fixtures.py
+++ b/napari/_tests/test_conftest_fixtures.py
@@ -4,6 +4,9 @@
 from qtpy.QtCore import QMutex, QThread, QTimer
 from superqt.utils import qdebounced
 
+from napari._qt.qt_viewer import QtViewer
+from napari.viewer import ViewerModel
+
 
 class _TestThread(QThread):
     def __init__(self) -> None:
@@ -49,6 +52,11 @@ def test_disable_qtimer(qtbot):
     assert not th.isRunning()
 
 
+def test_console_mock(qapp):
+    qt_viewer = QtViewer(ViewerModel())
+    assert qt_viewer.console.__class__.__name__ == "FakeQtConsole"
+
+
 @pytest.mark.usefixtures("disable_throttling")
 @patch("qtpy.QtCore.QTimer.start")
 def test_disable_throttle(start_mock):