Repository: spyder-ide/spyder

Issue:
Spyder crashes at shutdown
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [ ] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [ ] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [ ] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [ ] Tried basic troubleshooting (if a bug/error)
    * [ ] Restarted Spyder
    * [ ] Reset preferences with ``spyder --reset``
    * [ ] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [ ] Tried the other applicable steps from the Troubleshooting Guide
* [ ] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description



### What steps reproduce the problem?
Run spyder, close it. I get good results with:
1. `pytest spyder/app/tests/test_mainwindow.py::test_window_title`

### What is the expected output? What do you see instead?



### Paste Traceback/Error Below (if applicable)
<!--- Copy from error dialog or View > Panes > Internal Console --->

```python-traceback
QThread: Destroyed while thread is still running
Fatal Python error: Aborted

Thread 0x00000002dd79f000 (most recent call first):
  File "/opt/homebrew/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/selectors.py", line 562 in select
  File "/opt/homebrew/lib/python3.10/site-packages/nest_asyncio.py", line 107 in _run_once
  File "/opt/homebrew/lib/python3.10/site-packages/nest_asyncio.py", line 84 in run_until_complete
  File "/opt/homebrew/lib/python3.10/site-packages/jupyter_client/utils.py", line 27 in wrapped
  File "/opt/homebrew/lib/python3.10/site-packages/jupyter_client/manager.py", line 504 in _async_shutdown_kernel
  File "/opt/homebrew/lib/python3.10/site-packages/jupyter_client/manager.py", line 82 in wrapper
  File "/opt/homebrew/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/tasks.py", line 232 in __step
  File "/opt/homebrew/lib/python3.10/site-packages/nest_asyncio.py", line 196 in step
  File "/opt/homebrew/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/events.py", line 80 in _run
  File "/opt/homebrew/lib/python3.10/site-packages/nest_asyncio.py", line 120 in _run_once
  File "/opt/homebrew/lib/python3.10/site-packages/nest_asyncio.py", line 84 in run_until_complete
  File "/opt/homebrew/lib/python3.10/site-packages/jupyter_client/utils.py", line 27 in wrapped
  File "/Users/quentin/pyscripts/spyder/spyder/plugins/ipythonconsole/utils/kernel_handler.py", line 541 in _thread_shutdown_kernel

Thread 0x00000002ccd7b000 (most recent call first):
  File "/Users/quentin/pyscripts/spyder/spyder/plugins/ipythonconsole/utils/kernel_handler.py", line 104 in run

Thread 0x00000002cccef000 (most recent call first):
  File "/Users/quentin/pyscripts/spyder/spyder/plugins/ipythonconsole/utils/kernel_handler.py", line 104 in run

Current thread 0x00000001dfad7a80 (most recent call first):
  <no Python frame>

```

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: master
* Python version: 
* Qt version:  Python 3.10.8 64-bit | Qt 5.15.5 | PyQt5 5.15.7 | Darwin 22.2.0 
* PyQt version: 
* Operating System name/version: 

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```
PASTE DEPENDENCIES HERE
```



Discussion:

Resolving patch:
diff --git a/spyder/plugins/ipythonconsole/utils/kernel_handler.py b/spyder/plugins/ipythonconsole/utils/kernel_handler.py
--- a/spyder/plugins/ipythonconsole/utils/kernel_handler.py
+++ b/spyder/plugins/ipythonconsole/utils/kernel_handler.py
@@ -137,6 +137,14 @@ class KernelHandler(QObject):
     The kernel raised an error while connecting.
     """
 
+    _shutdown_thread_list = []
+    """List of running shutdown threads"""
+
+    _shutdown_thread_list_lock = Lock()
+    """
+    Lock to add threads to _shutdown_thread_list or clear that list.
+    """
+
     def __init__(
         self,
         connection_file,
@@ -165,7 +173,6 @@ def __init__(
             self.handle_comm_ready)
 
         # Internal
-        self._shutdown_thread = None
         self._shutdown_lock = Lock()
         self._stdout_thread = None
         self._stderr_thread = None
@@ -489,7 +496,6 @@ def init_kernel_client(cls, connection_file, hostname, sshkey, password):
     def close(self, shutdown_kernel=True, now=False):
         """Close kernel"""
         self.close_comm()
-
         if shutdown_kernel and self.kernel_manager is not None:
             km = self.kernel_manager
             km.stop_restarter()
@@ -503,7 +509,8 @@ def close(self, shutdown_kernel=True, now=False):
                 shutdown_thread.run = self._thread_shutdown_kernel
                 shutdown_thread.start()
                 shutdown_thread.finished.connect(self.after_shutdown)
-                self._shutdown_thread = shutdown_thread
+                with self._shutdown_thread_list_lock:
+                    self._shutdown_thread_list.append(shutdown_thread)
 
         if (
             self.kernel_client is not None
@@ -515,7 +522,6 @@ def after_shutdown(self):
         """Cleanup after shutdown"""
         self.close_std_threads()
         self.kernel_comm.remove(only_closing=True)
-        self._shutdown_thread = None
 
     def _thread_shutdown_kernel(self):
         """Shutdown kernel."""
@@ -531,18 +537,19 @@ def _thread_shutdown_kernel(self):
             # kernel was externally killed
             pass
 
-    def wait_shutdown_thread(self):
+    @classmethod
+    def wait_all_shutdown_threads(cls):
         """Wait shutdown thread."""
-        thread = self._shutdown_thread
-        if thread is None:
-            return
-        if thread.isRunning():
-            try:
-                thread.kernel_manager._kill_kernel()
-            except Exception:
-                pass
-            thread.quit()
-            thread.wait()
+        with cls._shutdown_thread_list_lock:
+            for thread in cls._shutdown_thread_list:
+                if thread.isRunning():
+                    try:
+                        thread.kernel_manager._kill_kernel()
+                    except Exception:
+                        pass
+                    thread.quit()
+                    thread.wait()
+            cls._shutdown_thread_list = []
 
     def copy(self):
         """Copy kernel."""
diff --git a/spyder/plugins/ipythonconsole/widgets/main_widget.py b/spyder/plugins/ipythonconsole/widgets/main_widget.py
--- a/spyder/plugins/ipythonconsole/widgets/main_widget.py
+++ b/spyder/plugins/ipythonconsole/widgets/main_widget.py
@@ -1644,10 +1644,8 @@ def close_all_clients(self):
             client.close_client(is_last_client)
             open_clients.remove(client)
 
-        # Wait for all KernelHandler's to shutdown.
-        for client in self.clients:
-            if client.kernel_handler:
-                client.kernel_handler.wait_shutdown_thread()
+        # Wait for all KernelHandler threads to shutdown.
+        KernelHandler.wait_all_shutdown_threads()
 
         # Close cached kernel
         self.close_cached_kernel()


Test patch:
diff --git a/spyder/app/tests/test_mainwindow.py b/spyder/app/tests/test_mainwindow.py
--- a/spyder/app/tests/test_mainwindow.py
+++ b/spyder/app/tests/test_mainwindow.py
@@ -129,11 +129,6 @@ def test_leaks(main_window, qtbot):
 
     Many other ways of leaking exist but are not covered here.
     """
-    def wait_all_shutdown():
-        objects = gc.get_objects()
-        for o in objects:
-            if isinstance(o, KernelHandler):
-                o.wait_shutdown_thread()
 
     def ns_fun(main_window, qtbot):
         # Wait until the window is fully up
@@ -148,7 +143,7 @@ def ns_fun(main_window, qtbot):
         # Count initial objects
         # Only one of each should be present, but because of many leaks,
         # this is most likely not the case. Here only closing is tested
-        wait_all_shutdown()
+        KernelHandler.wait_all_shutdown_threads()
         gc.collect()
         objects = gc.get_objects()
         n_code_editor_init = 0
@@ -183,7 +178,7 @@ def ns_fun(main_window, qtbot):
         main_window.ipyconsole.restart()
 
         # Wait until the shells are closed
-        wait_all_shutdown()
+        KernelHandler.wait_all_shutdown_threads()
         return n_shell_init, n_code_editor_init
 
     n_shell_init, n_code_editor_init = ns_fun(main_window, qtbot)