Repository: spyder-ide/spyder

Issue:
Kernel is dead message when starting Spyder
## Description

### What steps will reproduce the problem?

<!--- You can use Markdown here --->

Error triggered while testing macOS installer build as part of the Spyder 6 beta2 release (rc0). The error shows up after Spyder starts when the installation process finishes

### Traceback
```python-traceback
Traceback (most recent call last):
  File "/Users/daniel/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/spyder/plugins/ipythonconsole/widgets/status.py", line 106, in <lambda>
    lambda sw=shellwidget: self.on_kernel_start(sw)
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daniel/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/spyder/plugins/ipythonconsole/widgets/status.py", line 136, in on_kernel_start
    mpl_backend = shellwidget.get_matplotlib_backend()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daniel/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/spyder/plugins/ipythonconsole/widgets/shell.py", line 734, in get_matplotlib_backend
    blocking=True).get_matplotlib_backend()
                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daniel/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/spyder_kernels/comms/commbase.py", line 610, in __call__
    self._comms_wrapper._send_call(call_dict, self._comm_id, buffers)
  File "/Users/daniel/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/spyder/plugins/ipythonconsole/comms/kernelcomm.py", line 146, in _send_call
    raise RuntimeError("Kernel is dead")
RuntimeError: Kernel is dead
```

## Versions

* Spyder version: 6.0.0b2.dev174  (standalone)
* Python version: 3.11.9 64-bit
* Qt version: 5.15.8
* PyQt5 version: 5.15.9
* Operating System: macOS-11.6.8-x86_64-i386-64bit

### Dependencies

```
# Mandatory:
aiohttp >=3.9.3                   :  3.9.5 (OK)
asyncssh >=2.0.0,<3.0.0           :  2.14.1 (OK)
atomicwrites >=1.2.0              :  1.4.1 (OK)
chardet >=2.0.0                   :  5.2.0 (OK)
cloudpickle >=0.5.0               :  3.0.0 (OK)
cookiecutter >=1.6.0              :  2.6.0 (OK)
diff_match_patch >=20181111       :  20230430 (OK)
github >=2.3.0                    :  2.3.0 (OK)
intervaltree >=3.0.2              :  3.1.0 (OK)
IPython >=8.13.0,<9.0.0,!=8.17.1  :  8.25.0 (OK)
jedi >=0.17.2,<0.20.0             :  0.19.1 (OK)
jellyfish >=0.7                   :  1.0.4 (OK)
jsonschema >=3.2.0                :  4.22.0 (OK)
keyring >=17.0.0                  :  25.2.1 (OK)
nbconvert >=4.0                   :  7.16.4 (OK)
numpydoc >=0.6.0                  :  1.7.0 (OK)
parso >=0.7.0,<0.9.0              :  0.8.4 (OK)
pexpect >=4.4.0                   :  4.9.0 (OK)
pickleshare >=0.4                 :  0.7.5 (OK)
psutil >=5.3                      :  5.9.8 (OK)
pygments >=2.0                    :  2.18.0 (OK)
pylint >=3.1,<4                   :  3.2.3 (OK)
pylint_venv >=3.0.2               :  3.0.3 (OK)
pyls_spyder >=0.4.0               :  0.4.0 (OK)
pylsp >=1.11.0,<1.12.0            :  1.11.0 (OK)
pylsp_black >=2.0.0,<3.0.0        :  2.0.0 (OK)
pyuca >=1.2                       :  1.2 (OK)
qdarkstyle >=3.2.0,<3.3.0         :  3.2.3 (OK)
qstylizer >=0.2.2                 :  0.2.3 (OK)
qtawesome >=1.3.1,<1.4.0          :  1.3.1 (OK)
qtconsole >=5.5.1,<5.6.0          :  5.5.2 (OK)
qtpy >=2.4.0                      :  2.4.1 (OK)
rtree >=0.9.7                     :  1.2.0 (OK)
setuptools >=49.6.0               :  70.0.0 (OK)
sphinx >=0.6.6                    :  7.3.7 (OK)
spyder_kernels >=3.0.0b7,<3.0.0b8 :  3.0.0b7 (OK)
superqt >=0.6.2,<1.0.0            :  0.6.7 (OK)
textdistance >=4.2.0              :  4.6.2 (OK)
three_merge >=0.1.1               :  0.1.1 (OK)
watchdog >=0.10.3                 :  4.0.1 (OK)
yarl >=1.9.4                      :  1.9.4 (OK)
zmq >=24.0.0                      :  26.0.3 (OK)

# Optional:
cython >=0.21                     :  3.0.10 (OK)
matplotlib >=3.0.0                :  3.8.4 (OK)
numpy >=1.7                       :  2.0.0 (OK)
pandas >=1.1.1                    :  2.2.2 (OK)
scipy >=0.17.0                    :  1.13.1 (OK)
sympy >=0.7.3                     :  1.12.1 (OK)
```



Discussion:
> Operating System: macOS-11.6.8-x86_64-i386-64bit

I see that you're using Mac 11, but the installers require Mac 12 or newer now. So, this could be the cause of the error.

Let me check on my VM (which has Mac 14) to see what I find.
Could it be worthy to somehow validate the OS version and prevent installing from the installer if it is not supported?
> Could it be worthy to somehow validate the OS version and prevent installing from the installer if it is not supported?

Good idea but I don't know if that's possible. @mrclary, what do you think?
I think this is possible. Either using our `pre-install.sh` script which is run by constructor at install time, or using `__osx>=12` virtual package in the base environment specs. I'll test them to see what happens.
Hmm...
Well, `__osx>=12` in the base environment specs did not seem to do anything.
Using `pre-install.sh` to check macOS version successfully aborts the install process.

@jaimergp, do you know if there is a more elegant way for constructor to verify os version at the beginning of the install process?
Two issues with using `pre-install.sh`:
* The script does not raise the error until _after_ packages are already unzipped, i.e. there is some install time (and cleanup) before the user is made aware of any error. Ideally, the os version should be checked at the same time that the install location is checked, avoiding unnecessary install actions.
* The user is alerted to an error but not the reason for the error. The reason can only be determined by inspecting the install.log.
> Well, __osx>=12 in the base environment specs did not seem to do anything.

That's because the solver runs on the build machine, not the target installation machine.

That said we should detect that and add it as a check at installation time, but right now we don't have that logic.

Also, I think the [PKG installers have a way to check this natively](https://developer.apple.com/library/archive/documentation/DeveloperTools/Reference/DistributionDefinitionRef/Chapters/Distribution_XML_Ref.html#//apple_ref/doc/uid/TP40005370-CH100-SW34), so maybe you can hack something together while we come up with something on the constructor side.
Giving it a try at https://github.com/conda/constructor/pull/809

Resolving patch:
diff --git a/spyder/plugins/ipythonconsole/utils/kernel_handler.py b/spyder/plugins/ipythonconsole/utils/kernel_handler.py
--- a/spyder/plugins/ipythonconsole/utils/kernel_handler.py
+++ b/spyder/plugins/ipythonconsole/utils/kernel_handler.py
@@ -201,8 +201,14 @@ def connect_(self):
         self._shellwidget_connected = True
         # Emit signal in case the connection is already made
         if self.connection_state in [
-                KernelConnectionState.IpykernelReady,
-                KernelConnectionState.SpyderKernelReady]:
+            KernelConnectionState.IpykernelReady,
+            KernelConnectionState.SpyderKernelReady
+        ]:
+            # This is necessary for systems in which the kernel takes too much
+            # time to start because in that case its heartbeat is not detected
+            # as beating at this point.
+            # Fixes spyder-ide/spyder#22179
+            self.kernel_client.hb_channel._beating = True
             self.sig_kernel_is_ready.emit()
         elif self.connection_state == KernelConnectionState.Error:
             self.sig_kernel_connection_error.emit()
@@ -274,6 +280,11 @@ def handle_comm_ready(self):
             KernelConnectionState.SpyderKernelWaitComm,
             KernelConnectionState.Crashed
         ]:
+            # This is necessary for systems in which the kernel takes too much
+            # time to start because in that case its heartbeat is not detected
+            # as beating at this point.
+            # Fixes spyder-ide/spyder#22179
+            self.kernel_client.hb_channel._beating = True
             self.connection_state = KernelConnectionState.SpyderKernelReady
             self.sig_kernel_is_ready.emit()
 
diff --git a/spyder/plugins/ipythonconsole/widgets/status.py b/spyder/plugins/ipythonconsole/widgets/status.py
--- a/spyder/plugins/ipythonconsole/widgets/status.py
+++ b/spyder/plugins/ipythonconsole/widgets/status.py
@@ -7,9 +7,13 @@
 """Status bar widgets."""
 
 # Standard library imports
+import functools
 import sys
 import textwrap
 
+# Third-party imports
+from spyder_kernels.comms.frontendcomm import CommError
+
 # Local imports
 from spyder.api.shellconnect.mixins import ShellConnectMixin
 from spyder.api.translations import _
@@ -100,9 +104,11 @@ def update_status(self, gui):
     def add_shellwidget(self, shellwidget):
         """Add shellwidget."""
         shellwidget.sig_config_spyder_kernel.connect(
-            lambda sw=shellwidget: self.config_spyder_kernel(sw)
+            functools.partial(self.config_spyder_kernel, shellwidget)
         )
         shellwidget.kernel_handler.sig_kernel_is_ready.connect(
+            # We can't use functools.partial here because it gives memory leaks
+            # in Python versions older than 3.10
             lambda sw=shellwidget: self.on_kernel_start(sw)
         )
 
@@ -117,8 +123,9 @@ def add_shellwidget(self, shellwidget):
     def config_spyder_kernel(self, shellwidget):
         shellwidget.kernel_handler.kernel_comm.register_call_handler(
             "update_matplotlib_gui",
-            lambda gui, sid=id(shellwidget):
-                self.update_matplotlib_gui(gui, sid)
+            functools.partial(
+                self.update_matplotlib_gui, shellwidget_id=id(shellwidget)
+            )
         )
         shellwidget.set_kernel_configuration("update_gui", True)
 
@@ -133,7 +140,12 @@ def on_kernel_start(self, shellwidget):
         if running_in_ci() and not sys.platform.startswith("linux"):
             mpl_backend = "inline"
         else:
-            mpl_backend = shellwidget.get_matplotlib_backend()
+            # Needed when the comm is not connected.
+            # Fixes spyder-ide/spyder#22194
+            try:
+                mpl_backend = shellwidget.get_matplotlib_backend()
+            except CommError:
+                mpl_backend = None
 
         # Hide widget if Matplotlib is not available or failed to import
         if mpl_backend is None:


Test patch:
diff --git a/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py b/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
--- a/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
+++ b/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
@@ -1923,7 +1923,6 @@ def test_pdb_comprehension_namespace(ipyconsole, qtbot, tmpdir):
 
 @flaky(max_runs=10)
 @pytest.mark.auto_backend
-@pytest.mark.skipif(os.name == 'nt', reason="Fails on windows")
 def test_restart_interactive_backend(ipyconsole, qtbot):
     """
     Test that we ask for a restart or not after switching to different
@@ -1936,7 +1935,8 @@ def test_restart_interactive_backend(ipyconsole, qtbot):
 
     # This is necessary to test no spurious messages are printed to the console
     shell.clear_console()
-    qtbot.waitUntil(lambda: '\nIn [2]: ' == shell._control.toPlainText())
+    empty_console_text = '\n\nIn [2]: ' if os.name == "nt" else '\nIn [2]: '
+    qtbot.waitUntil(lambda: empty_console_text == shell._control.toPlainText())
 
     # Switch to the tk backend
     ipyconsole.set_conf('pylab/backend', 'tk')
@@ -1967,7 +1967,7 @@ def test_restart_interactive_backend(ipyconsole, qtbot):
     assert bool(os.environ.get('BACKEND_REQUIRE_RESTART'))
 
     # Check we no spurious messages are shown before the restart below
-    assert "\nIn [2]: " == shell._control.toPlainText()
+    assert empty_console_text == shell._control.toPlainText()
 
     # Restart kernel to check if the new interactive backend is set
     ipyconsole.restart_kernel()