Repository: spyder-ide/spyder
Pull request: PR: Catch leaks in the test suite
<!--- Make sure to read the Contributing Guidelines:                   --->
<!--- https://github.com/spyder-ide/spyder/blob/master/CONTRIBUTING.md --->
<!--- and follow PEP 8, PEP 257 and Spyder's code style:               --->
<!--- https://github.com/spyder-ide/spyder/wiki/Dev:-Coding-Style      --->

## Description of Changes


* [ ] Wrote at least one-line docstrings (for any new functions)
* [ ] Added unit test(s) covering the changes (if testable)
<!--- Remember that an image/animation is worth a thousand words! --->
* [ ] Included a screenshot or animation (if affecting the UI, see [Licecap](https://www.cockos.com/licecap/))


<!--- Explain what you've done and why --->




### Issue(s) Resolved

<!--- List the issue(s) below, in the form "Fixes #1234"; one per line --->

Fixes https://github.com/spyder-ide/spyder/issues/17213


### Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the [Developer Certificate of Origin](https://developercertificate.org)
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

<!--- TYPE YOUR USER/NAME AFTER THE FOLLOWING: --->
I certify the above statement is true and correct:

<!--- Thanks for your help making Spyder better for everyone! --->


Problem statement:
Crash on bad IPython config
## Description
Crash when closing a console which shows an error. The restarter is not properly stopped.

### What steps will reproduce the problem?

 1- Create a console with an error at launch
```python
import os.path as osp
from IPython.core.application import get_ipython_dir
ipy_kernel_cfg = osp.join(get_ipython_dir(), 'profile_default',
                          'ipython_kernel_config.py')
with open(ipy_kernel_cfg, 'w') as f:
    # This option must be a string, not an int
    f.write("c.InteractiveShellApp.extra_extension = 1")
```
 2- close the console
 3- Crash

### Traceback
```python-traceback
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\qtconsole\manager.py", line 27, in poll
    super().poll()
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\restarter.py", line 143, in poll
    self.kernel_manager.restart_kernel(now=True, newports=newports)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\utils.py", line 26, in wrapped
    raise e
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\utils.py", line 23, in wrapped
    return loop.run_until_complete(future)
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line 70, in run_until_complete
    return f.result()
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 178, in result
    raise self._exception
  File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 280, in __step
    result = coro.send(None)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\manager.py", line 517, in _async_restart_kernel
    await ensure_async(self.start_kernel(**self._launch_args))
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\utils.py", line 26, in wrapped
    raise e
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\utils.py", line 23, in wrapped
    return loop.run_until_complete(future)
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line 70, in run_until_complete
    return f.result()
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 178, in result
    raise self._exception
  File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 280, in __step
    result = coro.send(None)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\manager.py", line 362, in _async_start_kernel
    raise e
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\manager.py", line 351, in _async_start_kernel
    await ensure_async(self._launch_kernel(kernel_cmd, **kw))
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\utils.py", line 26, in wrapped
    raise e
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\utils.py", line 23, in wrapped
    return loop.run_until_complete(future)
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line 70, in run_until_complete
    return f.result()
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 178, in result
    raise self._exception
  File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 280, in __step
    result = coro.send(None)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\manager.py", line 269, in _async_launch_kernel
    connection_info = await self.provisioner.launch_kernel(kernel_cmd, **kw)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\provisioning\local_provisioner.py", line 179, in launch_kernel
    self.process = launch_kernel(cmd, **scrubbed_kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\launcher.py", line 169, in launch_kernel
    raise ex
  File "C:\ProgramData\Anaconda3\lib\site-packages\jupyter_client\launcher.py", line 157, in launch_kernel
    proc = Popen(cmd, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 804, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 1174, in _get_handles
    c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
ValueError: I/O operation on closed file
```

## Versions

* Spyder version: 5.3.0.dev0 075349a769
* Python version: 3.8.5
* Qt version: 5.12.10
* PyQt5 version: 5.12.3
* Operating System: Windows 10

### Dependencies

```
# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.2 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0;<8.0.0        :  7.29.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.0 (OK)
jellyfish >=0.7               :  0.8.9 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  23.4.0 (OK)
nbconvert >=4.0               :  6.1.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
paramiko >=2.4.0              :  2.8.1 (OK)
parso >=0.7.0;<0.9.0          :  0.8.2 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.8.0 (OK)
pygments >=2.0                :  2.10.0 (OK)
pylint >=2.5.0                :  2.9.6 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.3.2;<1.4.0          :  1.3.2 (OK)
pylsp_black >=1.0.0           :  1.0.0 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.0 (OK)
qtawesome >=1.0.2             :  1.0.3 (OK)
qtconsole >=5.2.1;<5.3.0      :  5.2.1 (OK)
qtpy >=1.5.0                  :  1.10.0 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  58.2.0 (OK)
sphinx >=0.6.6                :  4.2.0 (OK)
spyder_kernels >=2.2.0;<2.3.0 :  2.3.0.dev0 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.6 (OK)
zmq >=17                      :  22.3.0 (OK)

# Optional:
cython >=0.21                 :  0.29.25 (OK)
matplotlib >=2.0.0            :  3.4.2 (OK)
numpy >=1.7                   :  1.20.3 (OK)
pandas >=1.1.1                :  1.3.4 (OK)
scipy >=0.17.0                :  1.7.1 (OK)
sympy >=0.7.3                 :  1.9 (OK)
```

Hints from issue discussion:

Commit message: fix test

Patch to review:
diff --git a/spyder/app/mainwindow.py b/spyder/app/mainwindow.py
index 938ea1e68bb..4f8908093fb 100644
--- a/spyder/app/mainwindow.py
+++ b/spyder/app/mainwindow.py
@@ -1553,7 +1553,8 @@ def closing(self, cancelable=False, close_immediately=False):
         # to show them in their previous locations in the next session.
         # Fixes spyder-ide/spyder#12139
         prefix = 'window' + '/'
-        self.layouts.save_current_window_settings(prefix)
+        if self.layouts is not None:
+            self.layouts.save_current_window_settings(prefix)
         PLUGIN_REGISTRY.delete_plugin(Plugins.Layout)
 
         self.already_closed = True
diff --git a/spyder/app/tests/test_mainwindow.py b/spyder/app/tests/test_mainwindow.py
index 300598fa4d2..85960ea4714 100644
--- a/spyder/app/tests/test_mainwindow.py
+++ b/spyder/app/tests/test_mainwindow.py
@@ -20,6 +20,9 @@
 from textwrap import dedent
 from unittest.mock import Mock
 import uuid
+import psutil
+import threading
+import traceback
 
 # Third party imports
 from flaky import flaky
@@ -191,7 +194,7 @@ def read_asset_file(filename):
 # ---- Fixtures
 # =============================================================================
 @pytest.fixture
-def main_window(request, tmpdir):
+def main_window(request, tmpdir, qtbot):
     """Main Window fixture"""
     if not running_in_ci():
         register_fake_entrypoints()
@@ -316,26 +319,25 @@ def main_window(request, tmpdir):
         main_window.window = window
     else:
         window = main_window.window
-        # Close everything we can think of
-        window.editor.close_file()
-        window.projects.close_project()
-
-        if window.console.error_dialog:
-            window.console.close_error_dialog()
-
-        window.switcher.close()
-        for client in window.ipyconsole.get_clients():
-            window.ipyconsole.close_client(client=client, ask_recursive=False)
-        window.outlineexplorer.stop_symbol_services('python')
-        # Reset cwd
-        window.explorer.chdir(get_home_dir())
-        spyder_boilerplate = window.get_plugin(
-            'spyder_boilerplate', error=False)
-        if spyder_boilerplate is not None:
-            window.unregister_plugin(spyder_boilerplate)
 
     # Remove Kite (In case it was registered via setup.py)
     window.completions.providers.pop('kite', None)
+    # Wait until console is up
+    shell = window.ipyconsole.get_current_shellwidget()
+    qtbot.waitUntil(lambda: shell._prompt_html is not None,
+                    timeout=SHELL_TIMEOUT)
+    # _DummyThread are created if current_thread() is called from them
+    # They will always leak (From python doc) so we ignore them.
+    init_threads = [
+        thread for thread in threading.enumerate()
+        if not isinstance(thread, threading._DummyThread)]
+    window._initial_number_threads = len(init_threads)
+    proc = psutil.Process()
+    window._initial_number_files = len(proc.open_files())
+    init_files = [repr(f) for f in proc.open_files()]
+    init_subprocesses = [repr(f) for f in proc.children()]
+    number_subprocesses = len(init_subprocesses)
+
     yield window
 
     # Print shell content if failed
@@ -352,6 +354,87 @@ def main_window(request, tmpdir):
                 print(client.info_page)
             window.close()
             del main_window.window
+        else:
+            # Close everything we can think of
+            window.editor.close_file()
+            window.projects.close_project()
+
+            if window.console.error_dialog:
+                window.console.close_error_dialog()
+
+            window.switcher.close()
+            for client in window.ipyconsole.get_clients():
+                window.ipyconsole.close_client(client=client, ask_recursive=False)
+            window.outlineexplorer.stop_symbol_services('python')
+            # Reset cwd
+            window.explorer.chdir(get_home_dir())
+            spyder_boilerplate = window.get_plugin(
+                'spyder_boilerplate', error=False)
+            if spyder_boilerplate is not None:
+                window.unregister_plugin(spyder_boilerplate)
+
+            known_leak = request.node.get_closest_marker(
+                'known_leak')
+            if known_leak:
+                # This test has a known leaks
+                return
+
+            # The test is not allowed to open new files or threads.
+            try:
+
+                def threads_condition():
+                    threads = [
+                        thread for thread in threading.enumerate()
+                        if not isinstance(thread, threading._DummyThread)]
+                    return (window._initial_number_threads
+                            >= len(threads))
+
+                qtbot.waitUntil(threads_condition, timeout=SHELL_TIMEOUT)
+            except Exception:
+                # print for debug purposes
+                sys.stderr.write("Initial threads:\n")
+                for thread in init_threads:
+                    sys.stderr.write(repr(thread) + "\n")
+                sys.stderr.write("Running threads:" + "\n")
+                for thread in threading.enumerate():
+                    if not isinstance(thread, threading._DummyThread):
+                        sys.stderr.write(repr(thread) + "\n")
+                sys.stderr.write("Running Threads stacks:\n")
+                for threadId, frame in sys._current_frames().items():
+                    sys.stderr.write("\nThread " + str(threadId) + ":\n")
+                    traceback.print_stack(frame)
+                raise
+            try:
+                qtbot.waitUntil(lambda: (number_subprocesses >=
+                                         len(proc.children())),
+                                timeout=SHELL_TIMEOUT)
+            except Exception:
+                # print for debug purposes
+                sys.stderr.write("Initially open processes:" + "\n")
+                for process in init_subprocesses:
+                    sys.stderr.write(repr(process) + "\n")
+                sys.stderr.write("Open processes:" + "\n")
+                for process in proc.children():
+                    sys.stderr.write(repr(process) + "\n")
+                raise
+
+            if os.name == 'nt':
+                # kernel stderr file leaks on windows
+                return
+            try:
+                qtbot.waitUntil(
+                    lambda: (window._initial_number_files
+                             >= len(proc.open_files())),
+                    timeout=SHELL_TIMEOUT)
+            except Exception:
+                # print for debug purposes
+                sys.stderr.write("Initially open files:" + "\n")
+                for file in init_files:
+                    sys.stderr.write(repr(file) + "\n")
+                sys.stderr.write("Open files:" + "\n")
+                for file in proc.open_files():
+                    sys.stderr.write(repr(file) + "\n")
+                raise
 
 
 @pytest.fixture(scope="session", autouse=True)
@@ -383,7 +466,8 @@ def test_single_instance_and_edit_magic(main_window, qtbot, tmpdir):
     """Test single instance mode and %edit magic."""
     editorstack = main_window.editor.get_current_editorstack()
     shell = main_window.ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
+    qtbot.waitUntil(
+        lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     spy_dir = osp.dirname(get_module_path('spyder'))
     lock_code = (
@@ -546,6 +630,7 @@ def test_filter_numpy_warning(main_window, qtbot):
 @flaky(max_runs=3)
 @pytest.mark.skipif(PY2 or not sys.platform == 'darwin',
                     reason="Times out in PY2 and fails on other than macOS")
+@pytest.mark.known_leak  # Opens Spyder/QtWebEngine/Default/Cookies
 def test_get_help_combo(main_window, qtbot):
     """
     Test that Help can display docstrings for names typed in its combobox.
@@ -599,6 +684,7 @@ def test_get_help_combo(main_window, qtbot):
 
 @pytest.mark.slow
 @pytest.mark.skipif(PY2, reason="Invalid definition of function in Python 2.")
+@pytest.mark.known_leak  # Opens Spyder/QtWebEngine/Default/Cookies
 def test_get_help_ipython_console_dot_notation(main_window, qtbot, tmpdir):
     """
     Test that Help works when called from the IPython console
@@ -3226,7 +3312,7 @@ def test_path_manager_updates_clients(qtbot, main_window, tmpdir):
             control = shell._control
             # `shell.executed` signal was not working so we use waitUntil
             qtbot.waitUntil(lambda: 'In [2]:' in control.toPlainText(),
-                            timeout=EVAL_TIMEOUT)
+                            timeout=10000)
             assert test_folder in control.toPlainText()
             count += 1
     assert count >= 1
@@ -3877,16 +3963,19 @@ def test_update_outline(main_window, qtbot, tmpdir):
         if editor.get_language() == 'Python'
     ]
 
+    def editor_filled():
+        return all(
+            [
+                len(treewidget.editor_tree_cache[editor.get_id()]) == 4
+                for editor in editors_py
+            ]
+        )
+
     # Wait a bit for trees to be filled
-    qtbot.wait(25000)
+    qtbot.waitUntil(editor_filled, timeout=25000)
 
     # Assert all Python editors are filled
-    assert all(
-        [
-            len(treewidget.editor_tree_cache[editor.get_id()]) == 4
-            for editor in editors_py
-        ]
-    )
+    assert editor_filled()
 
     # Split editor
     editorstack = main_window.editor.get_current_editorstack()
@@ -3938,16 +4027,19 @@ def test_update_outline(main_window, qtbot, tmpdir):
     # Show outline again
     outline_explorer.toggle_view_action.setChecked(True)
 
-    # Wait a bit for its tree to be filled
-    qtbot.wait(3000)
+    def editor_filled():
+        return all(
+            [
+                len(treewidget.editor_tree_cache[editor.get_id()]) == 4
+                for editor in treewidget.editor_ids.keys()
+            ]
+        )
 
-    # Assert the editors were filled
-    assert all(
-        [
-            len(treewidget.editor_tree_cache[editor.get_id()]) == 4
-            for editor in treewidget.editor_ids.keys()
-        ]
-    )
+    # Wait a bit for trees to be filled
+    qtbot.waitUntil(editor_filled, timeout=3000)
+
+    # Assert all Python editors are filled
+    assert editor_filled()
 
     # Remove test file from session
     CONF.set('editor', 'filenames', [])
@@ -4313,6 +4405,7 @@ def crash_func():
 
 @pytest.mark.slow
 @flaky(max_runs=3)
+@pytest.mark.skipif(os.name == 'nt', reason="Tour messes up focus on Windows")
 @pytest.mark.parametrize("focus_to_editor", [True, False])
 def test_focus_to_editor(main_window, qtbot, tmpdir, focus_to_editor):
     """Test that the focus_to_editor option works as expected."""
@@ -4343,6 +4436,7 @@ def foo(x):
 
     # Be sure the focus is on the editor before proceeding
     code_editor.setFocus()
+    assert QApplication.focusWidget() is code_editor
 
     # Select the run cell button to click it
     run_cell_action = main_window.run_toolbar_actions[1]
diff --git a/spyder/plugins/console/plugin.py b/spyder/plugins/console/plugin.py
index 37cd2162f94..0e9fa0d57fd 100644
--- a/spyder/plugins/console/plugin.py
+++ b/spyder/plugins/console/plugin.py
@@ -187,7 +187,7 @@ def get_sys_path(self):
         return self.get_widget().get_sys_path()
 
     @Slot(dict)
-    def handle_exception(self, error_data):
+    def handle_exception(self, error_data, sender=None):
         """
         Handle any exception that occurs during Spyder usage.
 
@@ -215,9 +215,11 @@ def handle_exception(self, error_data):
         The `label` and `steps` keys allow customizing the content of the
         error dialog.
         """
+        if sender is None:
+            sender = self.sender()
         self.get_widget().handle_exception(
             error_data,
-            sender=self.sender()
+            sender=sender
         )
 
     def quit(self):
diff --git a/spyder/plugins/console/tests/test_plugin.py b/spyder/plugins/console/tests/test_plugin.py
index 20d6bb1c3b8..f44f2e3337f 100644
--- a/spyder/plugins/console/tests/test_plugin.py
+++ b/spyder/plugins/console/tests/test_plugin.py
@@ -127,7 +127,8 @@ def test_handle_exception(console_plugin, mocker):
             text=error,
             is_traceback=True,
             title='Internal Python Language Server error',
-        )
+        ),
+        sender=console_plugin
     )
 
     # Make sure the error dialog was generated.
diff --git a/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py b/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
index 909452434f8..fdf3de3b7a1 100644
--- a/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
+++ b/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
@@ -11,11 +11,13 @@
 """
 
 # Standard library imports
+import threading
 import codecs
 import glob
 import os
 import os.path as osp
 import shutil
+import psutil
 import sys
 import tempfile
 from textwrap import dedent
@@ -206,6 +208,18 @@ def __getattr__(self, attr):
         qtbot.addWidget(window)
     window.resize(640, 480)
     window.show()
+    # Wait until the window is fully up
+    shell = console.get_current_shellwidget()
+    qtbot.waitUntil(lambda: shell._prompt_html is not None,
+                    timeout=SHELL_TIMEOUT)
+
+    threads_count = threading.active_count()
+    init_threads = [repr(t) for t in threading.enumerate()]
+    proc = psutil.Process()
+    init_subprocesses = [repr(f) for f in proc.children()]
+    number_subprocesses = len(init_subprocesses) - 1  # -1 from closed client
+    number_files = len(proc.open_files())
+    init_files = [repr(f) for f in proc.open_files()]
 
     yield console
 
@@ -226,6 +240,48 @@ def __getattr__(self, attr):
     os.environ.pop('IPYCONSOLE_TESTING')
     os.environ.pop('IPYCONSOLE_TEST_DIR')
     os.environ.pop('IPYCONSOLE_TEST_NO_STDERR')
+    known_leak = request.node.get_closest_marker(
+        'known_leak')
+    if known_leak:
+        return
+    try:
+        qtbot.waitUntil(
+            lambda: threads_count >= threading.active_count(),
+            timeout=SHELL_TIMEOUT)
+    except Exception:
+        # print for debug purposes
+        sys.stderr.write("Initial threads:" + "\n")
+        for thread in init_threads:
+            sys.stderr.write(repr(thread) + "\n")
+        sys.stderr.write("Running threads:" + "\n")
+        for thread in threading.enumerate():
+            sys.stderr.write(repr(thread) + "\n")
+        raise
+    try:
+        qtbot.waitUntil(lambda: number_files >= len(proc.open_files()),
+                        timeout=SHELL_TIMEOUT)
+    except Exception:
+        # print for debug purposes
+        sys.stderr.write("Initially open files:" + "\n")
+        for file in init_files:
+            sys.stderr.write(repr(file) + "\n")
+        sys.stderr.write("Open files:" + "\n")
+        for file in proc.open_files():
+            sys.stderr.write(repr(file) + "\n")
+        raise
+    try:
+        qtbot.waitUntil(lambda: (number_subprocesses >=
+                                 len(proc.children())),
+                        timeout=SHELL_TIMEOUT)
+    except Exception:
+        # print for debug purposes
+        sys.stderr.write("Initially open processes:" + "\n")
+        for process in init_subprocesses:
+            sys.stderr.write(repr(process) + "\n")
+        sys.stderr.write("Open processes" + "\n")
+        for process in proc.children():
+            sys.stderr.write(repr(process) + "\n")
+        raise
 
 
 # =============================================================================
@@ -237,8 +293,6 @@ def test_banners(ipyconsole, qtbot):
     """Test that console banners are generated correctly."""
     shell = ipyconsole.get_current_shellwidget()
     control = shell._control
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Long banner
     text = control.toPlainText().splitlines()
@@ -280,8 +334,6 @@ def test_get_calltips(ipyconsole, qtbot, function, signature, documentation):
     """Test that calltips show the documentation."""
     shell = ipyconsole.get_current_shellwidget()
     control = shell._control
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Import numpy
     with qtbot.waitSignal(shell.executed):
@@ -314,8 +366,6 @@ def test_auto_backend(ipyconsole, qtbot):
     """Test that the automatic backend was set correctly."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     with qtbot.waitSignal(shell.executed):
         shell.execute("ip = get_ipython(); ip.kernel.eventloop")
@@ -351,8 +401,6 @@ def test_pylab_client(ipyconsole, qtbot):
     """Test that the Pylab console is working correctly."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # This is here to generate further errors
     with qtbot.waitSignal(shell.executed):
@@ -383,8 +431,6 @@ def test_sympy_client(ipyconsole, qtbot):
     """Test that the SymPy console is working correctly."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # This is here to generate further errors
     with qtbot.waitSignal(shell.executed):
@@ -417,11 +463,9 @@ def test_cython_client(ipyconsole, qtbot):
     """Test that the Cython console is working correctly."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # This is here to generate further errors
-    with qtbot.waitSignal(shell.executed):
+    with qtbot.waitSignal(shell.executed, timeout=SHELL_TIMEOUT):
         shell.execute("%%cython\n"
                       "cdef int ctest(int x, int y):\n"
                       "    return x + y")
@@ -435,7 +479,7 @@ def test_cython_client(ipyconsole, qtbot):
     qtbot.wait(1000)
 
     # See that cython is still enabled after reset
-    with qtbot.waitSignal(shell.executed):
+    with qtbot.waitSignal(shell.executed, timeout=SHELL_TIMEOUT):
         shell.execute("%%cython\n"
                       "cdef int ctest(int x, int y):\n"
                       "    return x + y")
@@ -448,11 +492,6 @@ def test_cython_client(ipyconsole, qtbot):
 @flaky(max_runs=3)
 def test_tab_rename_for_slaves(ipyconsole, qtbot):
     """Test slave clients are renamed correctly."""
-    # Wait until the window is fully up
-    shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
-
     cf = ipyconsole.get_current_client().connection_file
     ipyconsole.get_widget()._create_client_for_kernel(cf, None, None, None)
     qtbot.waitUntil(lambda: len(ipyconsole.get_clients()) == 2)
@@ -468,9 +507,6 @@ def test_tab_rename_for_slaves(ipyconsole, qtbot):
 @flaky(max_runs=3)
 def test_no_repeated_tabs_name(ipyconsole, qtbot):
     """Test that tabs can't have repeated given names."""
-    shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
-
     # Rename first client
     ipyconsole.get_widget().rename_tabs_after_change('foo')
 
@@ -486,9 +522,6 @@ def test_no_repeated_tabs_name(ipyconsole, qtbot):
 @flaky(max_runs=3)
 def test_tabs_preserve_name_after_move(ipyconsole, qtbot):
     """Test that tabs preserve their names after they are moved."""
-    shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
-
     # Create a new client
     ipyconsole.create_new_client()
 
@@ -505,8 +538,6 @@ def test_conf_env_vars(ipyconsole, qtbot):
     """Test that kernels have env vars set by our kernel spec."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Get a CONF env var
     with qtbot.waitSignal(shell.executed):
@@ -522,8 +553,6 @@ def test_no_stderr_file(ipyconsole, qtbot):
     """Test that consoles can run without an stderr."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Execute a simple assignment
     with qtbot.waitSignal(shell.executed):
@@ -540,8 +569,6 @@ def test_non_ascii_stderr_file(ipyconsole, qtbot):
     """Test the creation of a console with a stderr file in a non-ascii dir."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Execute a simple assignment
     with qtbot.waitSignal(shell.executed):
@@ -558,8 +585,6 @@ def test_console_import_namespace(ipyconsole, qtbot):
     """Test an import of the form 'from foo import *'."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Import numpy
     with qtbot.waitSignal(shell.executed):
@@ -572,9 +597,6 @@ def test_console_import_namespace(ipyconsole, qtbot):
 @flaky(max_runs=3)
 def test_console_disambiguation(ipyconsole, qtbot):
     """Test the disambiguation of dedicated consoles."""
-    shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
-
     # Create directories and file for TEMP_DIRECTORY/a/b/c.py
     # and TEMP_DIRECTORY/a/d/c.py
     dir_b = osp.join(TEMP_DIRECTORY, 'a', 'b')
@@ -609,9 +631,6 @@ def test_console_disambiguation(ipyconsole, qtbot):
 @flaky(max_runs=3)
 def test_console_coloring(ipyconsole, qtbot):
     """Test that console gets the same coloring present in the Editor."""
-    shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
-
     config_options = ipyconsole.get_widget().config_options()
 
     syntax_style = config_options.JupyterWidget.syntax_style
@@ -639,8 +658,6 @@ def test_set_cwd(ipyconsole, qtbot, tmpdir):
     """Test kernel when changing cwd."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # spyder-ide/spyder#6451.
     savetemp = shell._cwd
@@ -663,8 +680,6 @@ def test_get_cwd(ipyconsole, qtbot, tmpdir):
     """Test current working directory."""
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # spyder-ide/spyder#6451.
     savetemp = shell._cwd
@@ -695,7 +710,6 @@ def test_get_cwd(ipyconsole, qtbot, tmpdir):
 def test_request_env(ipyconsole, qtbot):
     """Test that getting env vars from the kernel is working as expected."""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Add a new entry to os.environ
     with qtbot.waitSignal(shell.executed):
@@ -721,7 +735,6 @@ def test_request_syspath(ipyconsole, qtbot, tmpdir):
     expected.
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Add a new entry to sys.path
     with qtbot.waitSignal(shell.executed):
@@ -744,8 +757,6 @@ def test_request_syspath(ipyconsole, qtbot, tmpdir):
 def test_save_history_dbg(ipyconsole, qtbot):
     """Test that browsing command history is working while debugging."""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -823,8 +834,6 @@ def test_save_history_dbg(ipyconsole, qtbot):
 def test_dbg_input(ipyconsole, qtbot):
     """Test that spyder doesn't send pdb commands to unrelated input calls."""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -857,7 +866,6 @@ def test_unicode_vars(ipyconsole, qtbot):
     """
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Set value for a Unicode variable
     with qtbot.waitSignal(shell.executed):
@@ -877,9 +885,7 @@ def test_read_stderr(ipyconsole, qtbot):
     """
     Test the read operation of the stderr file of the kernel
     """
-    shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Set contents of the stderr file of the kernel
     content = 'Test text'
@@ -900,7 +906,6 @@ def test_values_dbg(ipyconsole, qtbot):
     debugging.
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -950,7 +955,6 @@ def test_execute_events_dbg(ipyconsole, qtbot):
     """Test execute events while debugging"""
 
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -1003,7 +1007,6 @@ def test_run_doctest(ipyconsole, qtbot):
     Test that doctests can be run without problems
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     code = dedent('''
     def add(x, y):
@@ -1041,7 +1044,6 @@ def test_mpl_backend_change(ipyconsole, qtbot):
     using the %matplotlib magic
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Import Matplotlib
     with qtbot.waitSignal(shell.executed):
@@ -1070,7 +1072,6 @@ def test_ctrl_c_dbg(ipyconsole, qtbot):
     Test that Ctrl+C works while debugging
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -1096,7 +1097,6 @@ def test_clear_and_reset_magics_dbg(ipyconsole, qtbot):
     Test that clear and reset magics are working while debugging
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -1135,7 +1135,6 @@ def test_restart_kernel(ipyconsole, mocker, qtbot):
     mocker.patch.object(ClientWidget, "_show_mpl_backend_errors")
 
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     # Do an assignment to verify that it's not there after restarting
     with qtbot.waitSignal(shell.executed):
@@ -1151,7 +1150,7 @@ def test_restart_kernel(ipyconsole, mocker, qtbot):
 
     # Check that we try to show Matplotlib backend errors at the beginning and
     # after the restart.
-    assert ClientWidget._show_mpl_backend_errors.call_count == 2
+    assert ClientWidget._show_mpl_backend_errors.call_count == 1
 
 
 @flaky(max_runs=3)
@@ -1159,9 +1158,7 @@ def test_load_kernel_file_from_id(ipyconsole, qtbot):
     """
     Test that a new client is created using its id
     """
-    shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     connection_file = osp.basename(client.connection_file)
     id_ = connection_file.split('kernel-')[-1].split('.json')[0]
@@ -1179,9 +1176,7 @@ def test_load_kernel_file_from_location(ipyconsole, qtbot, tmpdir):
     Test that a new client is created using a connection file
     placed in a different location from jupyter_runtime_dir
     """
-    shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     fname = osp.basename(client.connection_file)
     connection_file = to_text_string(tmpdir.join(fname))
@@ -1201,8 +1196,6 @@ def test_load_kernel_file(ipyconsole, qtbot, tmpdir):
     """
     shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     ipyconsole.get_widget()._create_client_for_kernel(
         client.connection_file, None, None, None)
@@ -1224,7 +1217,6 @@ def test_load_kernel_file(ipyconsole, qtbot, tmpdir):
 def test_sys_argv_clear(ipyconsole, qtbot):
     """Test that sys.argv is cleared up correctly"""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
 
     with qtbot.waitSignal(shell.executed):
         shell.execute('import sys; A = sys.argv')
@@ -1236,10 +1228,7 @@ def test_sys_argv_clear(ipyconsole, qtbot):
 @flaky(max_runs=5)
 def test_set_elapsed_time(ipyconsole, qtbot):
     """Test that the IPython console elapsed timer is set correctly."""
-    shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Show time label.
     ipyconsole.get_widget().set_show_elapsed_time_current_client(True)
@@ -1270,11 +1259,7 @@ def test_set_elapsed_time(ipyconsole, qtbot):
 @pytest.mark.skipif(os.name == 'nt', reason="Doesn't work on Windows")
 def test_stderr_file_is_removed_one_kernel(ipyconsole, qtbot, monkeypatch):
     """Test that consoles removes stderr when client is closed."""
-    # Wait until the window is fully up
-    shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # In a normal situation file should exist
     monkeypatch.setattr(QMessageBox, 'question',
@@ -1289,11 +1274,7 @@ def test_stderr_file_is_removed_one_kernel(ipyconsole, qtbot, monkeypatch):
 def test_stderr_file_is_removed_two_kernels(ipyconsole, qtbot, monkeypatch):
     """Test that console removes stderr when client and related clients
     are closed."""
-    # Wait until the window is fully up
-    shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # New client with the same kernel
     ipyconsole.get_widget()._create_client_for_kernel(
@@ -1315,11 +1296,7 @@ def test_stderr_file_is_removed_two_kernels(ipyconsole, qtbot, monkeypatch):
 def test_stderr_file_remains_two_kernels(ipyconsole, qtbot, monkeypatch):
     """Test that console doesn't remove stderr when a related client is not
     closed."""
-    # Wait until the window is fully up
-    shell = ipyconsole.get_current_shellwidget()
     client = ipyconsole.get_current_client()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # New client with the same kernel
     ipyconsole.get_widget()._create_client_for_kernel(
@@ -1405,8 +1382,6 @@ def test_remove_old_std_files(ipyconsole, qtbot):
 def test_console_working_directory(ipyconsole, qtbot):
     """Test for checking the working directory."""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
     with qtbot.waitSignal(shell.executed):
         shell.execute('import os; cwd = os.getcwd()')
 
@@ -1421,8 +1396,6 @@ def test_console_working_directory(ipyconsole, qtbot):
 def test_console_complete(ipyconsole, qtbot, tmpdir):
     """Test code completions in the console."""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -1553,8 +1526,6 @@ def check_value(name, value):
 def test_pdb_multiline(ipyconsole, qtbot):
     """Test entering a multiline statment into pdb"""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -1585,8 +1556,6 @@ def test_pdb_multiline(ipyconsole, qtbot):
 def test_pdb_ignore_lib(ipyconsole, qtbot, show_lib):
     """Test that pdb can avoid closed files."""
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -1622,8 +1591,6 @@ def test_calltip(ipyconsole, qtbot):
     See spyder-ide/spyder#10842
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Give focus to the widget that's going to receive clicks
     control = ipyconsole.get_widget().get_focus_widget()
@@ -1645,8 +1612,6 @@ def test_conda_env_activation(ipyconsole, qtbot):
     """
     # Wait until the window is fully up
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
 
     # Get conda activation environment variable
     with qtbot.waitSignal(shell.executed):
@@ -1666,8 +1631,6 @@ def test_kernel_kill(ipyconsole, qtbot):
     Test that the kernel correctly restarts after a kill.
     """
     shell = ipyconsole.get_current_shellwidget()
-    qtbot.waitUntil(lambda: shell._prompt_html is not None,
-                    timeout=SHELL_TIMEOUT)
     # Wait for the restarter to start
     qtbot.wait(3000)
     crash_string = 'import os, signal; os.kill(os.getpid(), signal.SIGTERM)'
diff --git a/spyder/plugins/ipythonconsole/widgets/client.py b/spyder/plugins/ipythonconsole/widgets/client.py
index 96171905e34..2bbb1710712 100644
--- a/spyder/plugins/ipythonconsole/widgets/client.py
+++ b/spyder/plugins/ipythonconsole/widgets/client.py
@@ -527,6 +527,10 @@ def show_kernel_error(self, error):
         # Tell the client we're in error mode
         self.is_error_shown = True
 
+        # Stop shellwidget
+        self.shellwidget.shutdown()
+        self.remove_std_files()
+
     def is_benign_error(self, error):
         """Decide if an error is benign in order to filter it."""
         benign_errors = [
@@ -598,11 +602,11 @@ def set_color_scheme(self, color_scheme, reset=True):
     def shutdown(self, is_last_client):
         """Shutdown connection and kernel if needed."""
         self.dialog_manager.close_all()
-        self.remove_std_files(is_last_client)
         shutdown_kernel = (
             is_last_client and not self.is_external_kernel
             and not self.is_error_shown)
         self.shellwidget.shutdown(shutdown_kernel)
+        self.remove_std_files(shutdown_kernel)
 
     def interrupt_kernel(self):
         """Interrupt the associanted Spyder kernel if it's running"""


Code review comments:
diff_hunk: @@ -352,6 +354,87 @@ def main_window(request, tmpdir):
                 print(client.info_page)
             window.close()
             del main_window.window
+        else:
+            # Close everything we can think of
+            window.editor.close_file()
+            window.projects.close_project()
+
+            if window.console.error_dialog:
+                window.console.close_error_dialog()
+
+            window.switcher.close()
+            for client in window.ipyconsole.get_clients():
+                window.ipyconsole.close_client(client=client, ask_recursive=False)
+            window.outlineexplorer.stop_symbol_services('python')
+            # Reset cwd
+            window.explorer.chdir(get_home_dir())
+            spyder_boilerplate = window.get_plugin(
path: spyder/app/tests/test_mainwindow.py
text: @user1:
```suggestion

            # Unregister boilerplate plugin
            spyder_boilerplate = window.get_plugin(
```

diff_hunk: @@ -352,6 +354,87 @@ def main_window(request, tmpdir):
                 print(client.info_page)
             window.close()
             del main_window.window
+        else:
+            # Close everything we can think of
+            window.editor.close_file()
+            window.projects.close_project()
+
+            if window.console.error_dialog:
+                window.console.close_error_dialog()
+
+            window.switcher.close()
+            for client in window.ipyconsole.get_clients():
+                window.ipyconsole.close_client(client=client, ask_recursive=False)
+            window.outlineexplorer.stop_symbol_services('python')
+            # Reset cwd
+            window.explorer.chdir(get_home_dir())
+            spyder_boilerplate = window.get_plugin(
+                'spyder_boilerplate', error=False)
+            if spyder_boilerplate is not None:
+                window.unregister_plugin(spyder_boilerplate)
+
+            known_leak = request.node.get_closest_marker(
+                'known_leak')
+            if known_leak:
+                # This test has a known leaks
+                return
+
+            # The test is not allowed to open new files or threads.
+            try:
+
path: spyder/app/tests/test_mainwindow.py
text: @user1:
```suggestion
```

diff_hunk: @@ -1151,17 +1150,15 @@ def test_restart_kernel(ipyconsole, mocker, qtbot):
 
     # Check that we try to show Matplotlib backend errors at the beginning and
     # after the restart.
-    assert ClientWidget._show_mpl_backend_errors.call_count == 2
+    assert ClientWidget._show_mpl_backend_errors.call_count == 1
path: spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
text: @user1:
Why did you changed this? This is what the test is test is precisely checking, as you can read in the comment above.

@author:
Because the test now wait for the console to be up, the first error is shown before the test starts, and is therefore not caught. I will change that to catch it again 

@author:
Fixed

diff_hunk: @@ -352,6 +354,87 @@ def main_window(request, tmpdir):
                 print(client.info_page)
             window.close()
             del main_window.window
+        else:
+            # Close everything we can think of
+            window.editor.close_file()
+            window.projects.close_project()
+
+            if window.console.error_dialog:
+                window.console.close_error_dialog()
+
+            window.switcher.close()
+            for client in window.ipyconsole.get_clients():
+                window.ipyconsole.close_client(client=client, ask_recursive=False)
+            window.outlineexplorer.stop_symbol_services('python')
+            # Reset cwd
+            window.explorer.chdir(get_home_dir())
+            spyder_boilerplate = window.get_plugin(
+                'spyder_boilerplate', error=False)
+            if spyder_boilerplate is not None:
+                window.unregister_plugin(spyder_boilerplate)
+
+            known_leak = request.node.get_closest_marker(
+                'known_leak')
+            if known_leak:
+                # This test has a known leaks
path: spyder/app/tests/test_mainwindow.py
text: @user1:
```suggestion
                # This test has a known leak
```