Repository: ipython/ipython (2024-10-21 ~ 2025-01-30)

problem_statement: autocall full behavior change in 8.27 (not present in 8.26.)
In 8.26 autocall of full checked for `__call__` on the object (I believe). Seems in 8.27 it is much more aggressive and will autocall for **everything**


```
In [17]: foo = dict()

In [18]: foo
-------> foo()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[18], line 1
----> 1 foo()

TypeError: 'dict' object is not callable

In [19]:
```

Not sure if perhaps it is related to this PR #14486 or not

date: 2024-10-21
patch: diff --git a/IPython/core/prefilter.py b/IPython/core/prefilter.py
index a29df0c27ad..fc7b1c34161 100644
--- a/IPython/core/prefilter.py
+++ b/IPython/core/prefilter.py
@@ -512,8 +512,10 @@ def check(self, line_info):
             callable(oinfo.obj)
             and (not self.exclude_regexp.match(line_info.the_rest))
             and self.function_name_regexp.match(line_info.ifun)
-            and line_info.raw_the_rest.startswith(" ")
-            or not line_info.raw_the_rest.strip()
+            and (
+                line_info.raw_the_rest.startswith(" ")
+                or not line_info.raw_the_rest.strip()
+            )
         ):
             return self.prefilter_manager.get_handler_by_name("auto")
         else:

test_patch: diff --git a/IPython/core/tests/test_prefilter.py b/IPython/core/tests/test_prefilter.py
index 999cd43e6e8..379a530e8e5 100644
--- a/IPython/core/tests/test_prefilter.py
+++ b/IPython/core/tests/test_prefilter.py
@@ -137,3 +137,13 @@ def test_autocall_should_support_unicode():
     finally:
         ip.run_line_magic("autocall", "0")
         del ip.user_ns["π"]
+
+
+def test_autocall_regression_gh_14513():
+    ip.run_line_magic("autocall", "2")
+    ip.user_ns["foo"] = dict()
+    try:
+        assert ip.prefilter("foo") == "foo"
+    finally:
+        ip.run_line_magic("autocall", "0")
+        del ip.user_ns["foo"]


----

problem_statement: Jupyter shebang cells ignore exit code on SIGINT
When a shebang cell (i.e., %%sh, %%bash, etc.) in a Jupyter notebook is manually interrupted, the cell promptly completes successfully, regardless of the exit code of the program. In the attached screenshot, I interrupted the kernel while the first cell was running, yet it proceeds run the second cell. By contrast, the synchronous failure in the third cell raises an error and prevents the fourth cell from running.

![Image](https://github.com/user-attachments/assets/b11648e8-e09d-424b-b545-90c0a0920fd2)

I believe this is due the following logic in IPython/core/magics/script.py, where a CalledProcessError is never raised when a shebang cell is interrupted:
```
try:
    in_thread(_stream_communicate(p, cell))
except KeyboardInterrupt:
    try:
        p.send_signal(signal.SIGINT)
        in_thread(asyncio.wait_for(p.wait(), timeout=0.1))
        if p.returncode is not None:
            print("Process is interrupted.")
            return
        p.terminate()
        in_thread(asyncio.wait_for(p.wait(), timeout=0.1))
        if p.returncode is not None:
            print("Process is terminated.")
            return
        p.kill()
        print("Process is killed.")
    except OSError:
        pass
    except Exception as e:
        print("Error while terminating subprocess (pid=%i): %s" % (p.pid, e))
    return

if args.raise_error and p.returncode != 0:
    # If we get here and p.returncode is still None, we must have
    # killed it but not yet seen its return code. We don't wait for it,
    # in case it's stuck in uninterruptible sleep. -9 = SIGKILL
    rc = p.returncode or -9
    raise CalledProcessError(rc, cell)
```
It seems like this would have the intended behavior were it not for the early returns, but this has been present since https://github.com/ipython/ipython/commit/40a92dc6cbb782eb330480f1a802d911476e3972.


date: 2025-01-29
patch: diff --git a/IPython/core/magics/execution.py b/IPython/core/magics/execution.py
index 7387e212f3..dd745249d3 100644
--- a/IPython/core/magics/execution.py
+++ b/IPython/core/magics/execution.py
@@ -977,7 +977,21 @@ def _run_with_debugger(
                     break
                 finally:
                     sys.settrace(trace)
-            
+
+            # Perform proper cleanup of the session in case if
+            # it exited with "continue" and not "quit" command
+            if hasattr(deb, "rcLines"):
+                # Run this code defensively in case if custom debugger
+                # class does not implement rcLines, which although public
+                # is an implementation detail of `pdb.Pdb` and not part of
+                # the more generic basic debugger framework (`bdb.Bdb`).
+                deb.set_quit()
+                deb.rcLines.extend(["q"])
+                try:
+                    deb.run("", code_ns, local_ns)
+                except StopIteration:
+                    # Stop iteration is raised on quit command
+                    pass
 
         except:
             etype, value, tb = sys.exc_info()
diff --git a/IPython/core/magics/script.py b/IPython/core/magics/script.py
index 0c405ef420..8abc22efe6 100644
--- a/IPython/core/magics/script.py
+++ b/IPython/core/magics/script.py
@@ -67,6 +67,10 @@ def script_args(f):
     return f
 
 
+class RaiseAfterInterrupt(Exception):
+    pass
+
+
 @magics_class
 class ScriptMagics(Magics):
     """Magics for talking to scripts
@@ -176,6 +180,10 @@ def shebang(self, line, cell):
 
         The rest of the cell is run by that program.
 
+        .. versionchanged:: 9.0
+          Interrupting the script executed without `--bg` will end in
+          raising an exception (unless `--no-raise-error` is passed).
+
         Examples
         --------
         ::
@@ -292,20 +300,33 @@ async def _stream_communicate(process, cell):
                 p.send_signal(signal.SIGINT)
                 in_thread(asyncio.wait_for(p.wait(), timeout=0.1))
                 if p.returncode is not None:
-                    print("Process is interrupted.")
-                    return
+                    print("Process was interrupted.")
+                    if args.raise_error:
+                        raise RaiseAfterInterrupt()
+                    else:
+                        return
                 p.terminate()
                 in_thread(asyncio.wait_for(p.wait(), timeout=0.1))
                 if p.returncode is not None:
-                    print("Process is terminated.")
-                    return
+                    print("Process was terminated.")
+                    if args.raise_error:
+                        raise RaiseAfterInterrupt()
+                    else:
+                        return
                 p.kill()
-                print("Process is killed.")
+                print("Process was killed.")
+                if args.raise_error:
+                    raise RaiseAfterInterrupt()
+            except RaiseAfterInterrupt:
+                pass
             except OSError:
                 pass
             except Exception as e:
                 print("Error while terminating subprocess (pid=%i): %s" % (p.pid, e))
-            return
+            if args.raise_error:
+                raise CalledProcessError(p.returncode, cell) from None
+            else:
+                return
 
         if args.raise_error and p.returncode != 0:
             # If we get here and p.returncode is still None, we must have

test_patch: diff --git a/IPython/core/tests/test_magic.py b/IPython/core/tests/test_magic.py
index 036f250d34..a107da0c6c 100644
--- a/IPython/core/tests/test_magic.py
+++ b/IPython/core/tests/test_magic.py
@@ -6,12 +6,16 @@
 import os
 import re
 import shlex
+import signal
 import sys
 import warnings
 from importlib import invalidate_caches
 from io import StringIO
 from pathlib import Path
+from subprocess import CalledProcessError
 from textwrap import dedent
+from time import sleep
+from threading import Thread
 from unittest import TestCase, mock
 
 import pytest
@@ -1121,6 +1125,36 @@ def test_script_config():
     assert "whoda" in sm.magics["cell"]
 
 
+def _interrupt_after_1s():
+    sleep(1)
+    signal.raise_signal(signal.SIGINT)
+
+
+def test_script_raise_on_interrupt():
+    ip = get_ipython()
+
+    with pytest.raises(CalledProcessError):
+        thread = Thread(target=_interrupt_after_1s)
+        thread.start()
+        ip.run_cell_magic(
+            "script", f"{sys.executable}", "from time import sleep; sleep(2)"
+        )
+        thread.join()
+
+
+def test_script_do_not_raise_on_interrupt():
+    ip = get_ipython()
+
+    thread = Thread(target=_interrupt_after_1s)
+    thread.start()
+    ip.run_cell_magic(
+        "script",
+        f"--no-raise-error {sys.executable}",
+        "from time import sleep; sleep(2)",
+    )
+    thread.join()
+
+
 def test_script_out():
     ip = get_ipython()
     ip.run_cell_magic("script", f"--out output {sys.executable}", "print('hi')")


----

problem_statement: Jupyter shebang cells won't show partially written lines
When a shebang cell (i.e., %%sh, %%bash, etc.) in a Jupyter notebook writes/flushes output, the output doesn't show up until the script ends or a newline is written. This hurts the usability of programs that indicate progress by writing periodically on the same line.

Changing this ipython logic to read whatever text is available (rather than waiting for a newline) would fix the issue: https://github.com/ipython/ipython/blob/18e45295c06eb9/IPython/core/magics/script.py#L215. (I'm not familiar with asyncio's streams, so it might also be necessary to ensure the fd is in non-blocking mode.)

Below is a GIF demonstrating the issue. Observe that:

1. The output regular Python cell updates in realtime, even if it doesn't include a newline.
2. The output an equivalent shebang cell does not update in realtime.
3. The output of a shebang cell that includes newlines updates in realtime.

![Image](https://github.com/user-attachments/assets/fd25968f-0400-4722-af4a-63e190413c27)

date: 2025-01-30
patch: diff --git a/IPython/core/magics/script.py b/IPython/core/magics/script.py
index 8abc22efe6..3bfc4d8d67 100644
--- a/IPython/core/magics/script.py
+++ b/IPython/core/magics/script.py
@@ -220,7 +220,7 @@ def in_thread(coro):
 
         async def _readchunk(stream):
             try:
-                return await stream.readuntil(b"\n")
+                return await stream.read(100)
             except asyncio.exceptions.IncompleteReadError as e:
                 return e.partial
             except asyncio.exceptions.LimitOverrunError as e:

test_patch: diff --git a/IPython/core/tests/test_magic.py b/IPython/core/tests/test_magic.py
index a107da0c6c..4e167007c5 100644
--- a/IPython/core/tests/test_magic.py
+++ b/IPython/core/tests/test_magic.py
@@ -12,6 +12,8 @@
 from importlib import invalidate_caches
 from io import StringIO
 from pathlib import Path
+from time import sleep
+from threading import Thread
 from subprocess import CalledProcessError
 from textwrap import dedent
 from time import sleep
@@ -1259,6 +1261,37 @@ def test_script_defaults():
             assert cmd in ip.magics_manager.magics["cell"]
 
 
+async def test_script_streams_continiously(capsys):
+    ip = get_ipython()
+    # Windows is slow to start up a thread on CI
+    is_windows = os.name == "nt"
+    step = 3 if is_windows else 1
+    code = dedent(
+        f"""\
+    import time
+    for _ in range(3):
+        time.sleep({step})
+        print(".", flush=True, end="")
+    """
+    )
+
+    def print_numbers():
+        for i in range(3):
+            sleep(step)
+            print(i, flush=True, end="")
+
+    thread = Thread(target=print_numbers)
+    thread.start()
+    sleep(step / 2)
+    ip.run_cell_magic("script", f"{sys.executable}", code)
+    thread.join()
+
+    captured = capsys.readouterr()
+    # If the streaming was line-wise or broken
+    # we would get `012...`
+    assert captured.out == "0.1.2."
+
+
 @magics_class
 class FooFoo(Magics):
     """class with both %foo and %%foo magics"""