Repository: jupyter/jupyter_client (python)

Issue:
When no kernel is found, jupter-run crashes with a traceback
Hello,
in Fedora, we use Automatic Bug Reporting Tool https://github.com/abrt/abrt

When a packaged software "crashes", the tool records that and encourages users to report it as a bug.

It also detects Python software that ends with a Traceback.

We have encountered an user report that running `/usr/bin/jupyter-run` with no kernel installed results in a Traceback and it is reported as a crash.

I tried reproducing this with pip-installed jupyter-client.

When I create a Python virtual environment on my machine and install jupyter-client in it, I cannot reproduce, because (presumably) it finds my kernels in /usr/share/jupyter/kernels/ and/or ~/.local/share/jupyter/kernels/.

However, if I use an empty podman container, I can reproduce:

```
$ podman run --rm -ti fedora:38 /usr/bin/bash
...
[root@cd03926c2bc1 /]# python3.11 -m venv venv
[root@cd03926c2bc1 /]# . venv/bin/activate
(venv) [root@cd03926c2bc1 /]# pip install jupyter-client
...
Successfully installed jupyter-client-8.4.0 jupyter-core-5.4.0 platformdirs-3.11.0 python-dateutil-2.8.2 pyzmq-25.1.1 six-1.16.0 tornado-6.3.3 traitlets-5.11.2

(venv) [root@cd03926c2bc1 /]# jupyter-run
[RunApp] WARNING | Kernelspec name python3 cannot be found!
[RunApp] ERROR | No such kernel named python3
Traceback (most recent call last):
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 85, in wrapper
    out = await method(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 401, in _async_start_kernel
    kernel_cmd, kw = await self._async_pre_start_kernel(**kw)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 363, in _async_pre_start_kernel
    self.kernel_spec,
    ^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 184, in kernel_spec
    self._kernel_spec = self.kernel_spec_manager.get_kernel_spec(self.kernel_name)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/kernelspec.py", line 280, in get_kernel_spec
    raise NoSuchKernel(kernel_name)
jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3
Traceback (most recent call last):
  File "/venv/bin/jupyter-run", line 8, in <module>
    sys.exit(RunApp.launch_instance())
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_core/application.py", line 280, in launch_instance
    super().launch_instance(argv=argv, **kwargs)
  File "/venv/lib64/python3.11/site-packages/traitlets/config/application.py", line 1052, in launch_instance
    app.initialize(argv)
  File "/venv/lib64/python3.11/site-packages/traitlets/config/application.py", line 117, in inner
    return method(app, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/runapp.py", line 71, in initialize
    JupyterConsoleApp.initialize(self)
  File "/venv/lib64/python3.11/site-packages/jupyter_client/consoleapp.py", line 366, in initialize
    self.init_kernel_manager()
  File "/venv/lib64/python3.11/site-packages/jupyter_client/consoleapp.py", line 319, in init_kernel_manager
    self.kernel_manager.start_kernel(**kwargs)
  File "/venv/lib64/python3.11/site-packages/jupyter_core/utils/__init__.py", line 173, in wrapped
    return loop.run_until_complete(inner)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 94, in wrapper
    raise e
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 85, in wrapper
    out = await method(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 401, in _async_start_kernel
    kernel_cmd, kw = await self._async_pre_start_kernel(**kw)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 363, in _async_pre_start_kernel
    self.kernel_spec,
    ^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/manager.py", line 184, in kernel_spec
    self._kernel_spec = self.kernel_spec_manager.get_kernel_spec(self.kernel_name)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib64/python3.11/site-packages/jupyter_client/kernelspec.py", line 280, in get_kernel_spec
    raise NoSuchKernel(kernel_name)
```

Would it be possible to not show the traceback in this case and simply end the app when `[RunApp] ERROR | No such kernel named python3` is displayed, with a non-zero exit code?

Pull request description:
jupyter-run: avoid traceback for NoSuchKernel
NoSuchKernel used to raise during KernelManager instantiation, but it is now delayed, so our `except NoSuchKernel` isn't catching the error when it's raised anymore.

Access kernel_spec to ensure it's raised where it will be caught.

Also removes a redundant warning log immediately before raising, which prevents complete handling of NoSuchError and produces unavoidable duplicate logs.

Test added for runapp, both regression for #993 and simple exercise for everything working correctly.

closes #993

Patch:
diff --git a/jupyter_client/consoleapp.py b/jupyter_client/consoleapp.py
index 4e9ad35..434fdf0 100644
--- a/jupyter_client/consoleapp.py
+++ b/jupyter_client/consoleapp.py
@@ -306,8 +306,11 @@ class JupyterConsoleApp(ConnectionFileMixin):
                 parent=self,
                 data_dir=self.data_dir,
             )
+            # access kernel_spec to ensure the NoSuchKernel error is raised
+            # if it's going to be
+            kernel_spec = self.kernel_manager.kernel_spec  # noqa: F841
         except NoSuchKernel:
-            self.log.critical("Could not find kernel %s", self.kernel_name)
+            self.log.critical("Could not find kernel %r", self.kernel_name)
             self.exit(1)  # type:ignore[attr-defined]
 
         self.kernel_manager = t.cast(KernelManager, self.kernel_manager)
diff --git a/jupyter_client/kernelspec.py b/jupyter_client/kernelspec.py
index 41ed2ba..b903ee6 100644
--- a/jupyter_client/kernelspec.py
+++ b/jupyter_client/kernelspec.py
@@ -281,7 +281,6 @@ class KernelSpecManager(LoggingConfigurable):
 
         resource_dir = self._find_spec_directory(kernel_name.lower())
         if resource_dir is None:
-            self.log.warning("Kernelspec name %s cannot be found!", kernel_name)
             raise NoSuchKernel(kernel_name)
 
         return self._get_kernel_spec_by_name(kernel_name, resource_dir)
diff --git a/jupyter_client/localinterfaces.py b/jupyter_client/localinterfaces.py
index 8e622f1..e257b26 100644
--- a/jupyter_client/localinterfaces.py
+++ b/jupyter_client/localinterfaces.py
@@ -8,11 +8,11 @@ import re
 import socket
 import subprocess
 from subprocess import PIPE, Popen
-from typing import Any, Callable, Iterable, Sequence
+from typing import Any, Callable, Iterable, Mapping, Sequence
 from warnings import warn
 
-LOCAL_IPS: list = []
-PUBLIC_IPS: list = []
+LOCAL_IPS: list[str] = []
+PUBLIC_IPS: list[str] = []
 
 LOCALHOST: str = ""
 
@@ -75,27 +75,34 @@ class NoIPAddresses(Exception):  # noqa
     pass
 
 
-def _populate_from_list(addrs: Sequence[str] | None) -> None:
+def _populate_from_list(addrs: Sequence[str]) -> None:
     """populate local and public IPs from flat list of all IPs"""
+    _populate_from_dict({"all": addrs})
+
+
+def _populate_from_dict(addrs: Mapping[str, Sequence[str]]) -> None:
+    """populate local and public IPs from dict of {'en0': 'ip'}"""
     if not addrs:
-        raise NoIPAddresses
+        raise NoIPAddresses()
 
     global LOCALHOST
     public_ips = []
     local_ips = []
 
-    for ip in addrs:
-        local_ips.append(ip)
-        if not ip.startswith("127."):
-            public_ips.append(ip)
-        elif not LOCALHOST:
-            LOCALHOST = ip
+    for iface, ip_list in addrs.items():
+        for ip in ip_list:
+            local_ips.append(ip)
+            if not LOCALHOST and (iface.startswith("lo") or ip.startswith("127.")):
+                LOCALHOST = ip
+            if not iface.startswith("lo") and not ip.startswith(("127.", "169.254.")):
+                # don't include link-local address in public_ips
+                public_ips.append(ip)
 
     if not LOCALHOST or LOCALHOST == "127.0.0.1":
         LOCALHOST = "127.0.0.1"
         local_ips.insert(0, LOCALHOST)
 
-    local_ips.extend(["0.0.0.0", ""])  # noqa
+    local_ips.extend(["0.0.0.0", ""])  # noqa: S104
 
     LOCAL_IPS[:] = _uniq_stable(local_ips)
     PUBLIC_IPS[:] = _uniq_stable(public_ips)
@@ -155,59 +162,38 @@ def _load_ips_psutil() -> None:
     """load ip addresses with netifaces"""
     import psutil
 
-    global LOCALHOST
-    local_ips = []
-    public_ips = []
+    addr_dict: dict[str, list[str]] = {}
 
     # dict of iface_name: address_list, eg
     # {"lo": [snicaddr(family=<AddressFamily.AF_INET>, address="127.0.0.1",
     #   ...), snicaddr(family=<AddressFamily.AF_INET6>, ...)]}
     for iface, ifaddresses in psutil.net_if_addrs().items():
-        for address_data in ifaddresses:
-            if address_data.family == socket.AF_INET:
-                addr = address_data.address
-                if not (iface.startswith("lo") or addr.startswith("127.")):
-                    public_ips.append(addr)
-                elif not LOCALHOST:
-                    LOCALHOST = addr
-                local_ips.append(addr)
-    if not LOCALHOST:
-        # we never found a loopback interface (can this ever happen?), assume common default
-        LOCALHOST = "127.0.0.1"
-        local_ips.insert(0, LOCALHOST)
-    local_ips.extend(["0.0.0.0", ""])  # noqa
-    LOCAL_IPS[:] = _uniq_stable(local_ips)
-    PUBLIC_IPS[:] = _uniq_stable(public_ips)
+        addr_dict[iface] = [
+            address_data.address
+            for address_data in ifaddresses
+            if address_data.family == socket.AF_INET
+        ]
+
+    _populate_from_dict(addr_dict)
 
 
 def _load_ips_netifaces() -> None:
     """load ip addresses with netifaces"""
     import netifaces  # type: ignore[import-not-found]
 
-    global LOCALHOST
-    local_ips = []
-    public_ips = []
+    addr_dict: dict[str, list[str]] = {}
 
     # list of iface names, 'lo0', 'eth0', etc.
     for iface in netifaces.interfaces():
         # list of ipv4 addrinfo dicts
+        addr_dict[iface] = []
+
         ipv4s = netifaces.ifaddresses(iface).get(netifaces.AF_INET, [])
         for entry in ipv4s:
             addr = entry.get("addr")
-            if not addr:
-                continue
-            if not (iface.startswith("lo") or addr.startswith("127.")):
-                public_ips.append(addr)
-            elif not LOCALHOST:
-                LOCALHOST = addr
-            local_ips.append(addr)
-    if not LOCALHOST:
-        # we never found a loopback interface (can this ever happen?), assume common default
-        LOCALHOST = "127.0.0.1"
-        local_ips.insert(0, LOCALHOST)
-    local_ips.extend(["0.0.0.0", ""])  # noqa
-    LOCAL_IPS[:] = _uniq_stable(local_ips)
-    PUBLIC_IPS[:] = _uniq_stable(public_ips)
+            if addr:
+                addr_dict[iface].append(addr)
+    _populate_from_dict(addr_dict)
 
 
 def _load_ips_gethostbyname() -> None:
diff --git a/jupyter_client/runapp.py b/jupyter_client/runapp.py
index 9ed4b15..bea5d5c 100644
--- a/jupyter_client/runapp.py
+++ b/jupyter_client/runapp.py
@@ -3,10 +3,9 @@
 # Distributed under the terms of the Modified BSD License.
 from __future__ import annotations
 
-import queue
+import atexit
 import signal
 import sys
-import time
 import typing as t
 
 from jupyter_core.application import JupyterApp, base_aliases, base_flags
@@ -73,7 +72,8 @@ class RunApp(JupyterApp, JupyterConsoleApp):  # type:ignore[misc]
         super().initialize(argv)
         JupyterConsoleApp.initialize(self)
         signal.signal(signal.SIGINT, self.handle_sigint)
-        self.init_kernel_info()
+        if self.kernel_manager:
+            atexit.register(self.kernel_manager.shutdown_kernel)
 
     def handle_sigint(self, *args: t.Any) -> None:
         """Handle SIGINT."""
@@ -82,28 +82,11 @@ class RunApp(JupyterApp, JupyterConsoleApp):  # type:ignore[misc]
         else:
             self.log.error("Cannot interrupt kernels we didn't start.\n")
 
-    def init_kernel_info(self) -> None:
-        """Wait for a kernel to be ready, and store kernel info"""
-        timeout = self.kernel_timeout
-        tic = time.time()
-        self.kernel_client.hb_channel.unpause()
-        msg_id = self.kernel_client.kernel_info()
-        while True:
-            try:
-                reply = self.kernel_client.get_shell_msg(timeout=1)
-            except queue.Empty as e:
-                if (time.time() - tic) > timeout:
-                    msg = "Kernel didn't respond to kernel_info_request"
-                    raise RuntimeError(msg) from e
-            else:
-                if reply["parent_header"].get("msg_id") == msg_id:
-                    self.kernel_info = reply["content"]
-                    return
-
     def start(self) -> None:
         """Start the application."""
         self.log.debug("jupyter run: starting...")
         super().start()
+        self.kernel_client.wait_for_ready(timeout=self.kernel_timeout)
         if self.filenames_to_run:
             for filename in self.filenames_to_run:
                 self.log.debug("jupyter run: executing `%s`", filename)
@@ -112,8 +95,10 @@ class RunApp(JupyterApp, JupyterConsoleApp):  # type:ignore[misc]
                     reply = self.kernel_client.execute_interactive(code, timeout=OUTPUT_TIMEOUT)
                     return_code = 0 if reply["content"]["status"] == "ok" else 1
                     if return_code:
-                        raise Exception("jupyter-run error running '%s'" % filename)
+                        msg = f"jupyter-run error running '{filename}'"
+                        raise Exception(msg)
         else:
+            self.log.debug("jupyter run: executing from stdin")
             code = sys.stdin.read()
             reply = self.kernel_client.execute_interactive(code, timeout=OUTPUT_TIMEOUT)
             return_code = 0 if reply["content"]["status"] == "ok" else 1


Test patch:
diff --git a/tests/test_runapp.py b/tests/test_runapp.py
new file mode 100644
index 0000000..7f295ba
--- /dev/null
+++ b/tests/test_runapp.py
@@ -0,0 +1,34 @@
+import sys
+from subprocess import run
+
+
+def test_runapp(tmp_path):
+    test_py = tmp_path / "test.py"
+    with test_py.open("w") as f:
+        f.write("print('hello')")
+
+    p = run(
+        [sys.executable, "-m", "jupyter_client.runapp", str(test_py)],
+        capture_output=True,
+        text=True,
+        check=True,
+    )
+    assert p.stdout.strip() == "hello"
+
+
+def test_no_such_kernel(tmp_path):
+    test_py = tmp_path / "test.py"
+    with test_py.open("w") as f:
+        f.write("print('hello')")
+    kernel_name = "nosuchkernel"
+    p = run(
+        [sys.executable, "-m", "jupyter_client.runapp", "--kernel", kernel_name, str(test_py)],
+        capture_output=True,
+        text=True,
+        check=False,
+    )
+    assert p.returncode
+    assert "Could not find kernel" in p.stderr
+    assert kernel_name in p.stderr
+    # shouldn't show a traceback
+    assert "Traceback" not in p.stderr