Repository: ipython/ipython

Issue:
Auto suggestions cannot be completed when in the middle of multi-line input
<!-- This is the repository for IPython command line, if you can try to make sure this question/bug/feature belong here and not on one of the Jupyter repositories. 

If it's a generic Python/Jupyter question, try other forums or discourse.jupyter.org.

If you are unsure, it's ok to post here, though, there are few maintainer so you might not get a fast response. 

-->

Completing an ipython 8 [autosuggestion](https://ipython.readthedocs.io/en/stable/whatsnew/version8.html#autosuggestions) by pressing `ctrl-f` or `ctrl-e` works perfectly for code on the final line of multi-line input (the following two screenshots show before and after pressing `ctrl-f` or `ctrl-e`):

<img width="357" alt="Screenshot 2023-03-12 at 14 09 48" src="https://user-images.githubusercontent.com/19657652/224570634-fb856132-4b73-46b4-bdf0-576f9601633f.png">

<img width="363" alt="Screenshot 2023-03-12 at 14 10 00" src="https://user-images.githubusercontent.com/19657652/224570647-40ccf5c1-00d6-4d5e-b965-691ca12ffe51.png">

However pressing `ctrl-f` or `ctrl-e` fails when trying to complete the suggestion in the middle of multi-line input -- instead, the only thing that works is holding down right arrow until the suggestion is completed:

<img width="361" alt="Screenshot 2023-03-12 at 14 11 27" src="https://user-images.githubusercontent.com/19657652/224570743-37756db5-989a-46d4-8447-bb23649d7309.png">

Note this problem persists across different terminals (apple Terminal and iTerm2) and after disabling custom `bashrc` and `inputrc`. Is there any way suggestions can be completed with one keystroke in the middle of multi-line input?


Discussion:
Thank you for the report, I can reproduce it. Here are two workarounds for now:
- you can use <kbd>End</kbd>
- you can update to 8.11 where <kbd>right</kbd> was changed back to complete full line (whereas <kbd>ctrl</kbd> + <kbd>right</kbd> completes token-by-token and <kbd>alt</kbd> + <kbd>right</kbd> completes character-by- character).
Thanks for the quick response. Unfortunately I tried hitting <kbd>Right</kbd> in ipython 8.11 and it doesn't work (<kbd>Ctrl</kbd>+<kbd>Right</kbd> and <kbd>Alt</kbd>+<kbd>Right</kbd> also don't work)... ~~so now there seems to be no way to complete a middle line~~.

Edit: Actually while <kbd>Right</kbd> and <kbd>Alt</kbd>+<kbd>Right</kbd> don't work, <kbd>Ctrl</kbd>+<kbd>Right</kbd> works in Terminal but not iTerm2 (on my setup). However <kbd>End</kbd> does work in both ipython 8.10 and ipython 8.11. I'll use that for the time being.
This is not strictly related to this bug, but I just wanted to ask if you happen to have an opinion on https://github.com/ipython/ipython/issues/13987? Please leave :+1: or :-1:  over there if you do - thank you :)

Resolving patch:
diff --git a/IPython/terminal/shortcuts/__init__.py b/IPython/terminal/shortcuts/__init__.py
--- a/IPython/terminal/shortcuts/__init__.py
+++ b/IPython/terminal/shortcuts/__init__.py
@@ -181,30 +181,45 @@ def create_identifier(handler: Callable):
 ]
 
 AUTO_SUGGEST_BINDINGS = [
+    # there are two reasons for re-defining bindings defined upstream:
+    # 1) prompt-toolkit does not execute autosuggestion bindings in vi mode,
+    # 2) prompt-toolkit checks if we are at the end of text, not end of line
+    #    hence it does not work in multi-line mode of navigable provider
     Binding(
-        auto_suggest.accept_in_vi_insert_mode,
+        auto_suggest.accept_or_jump_to_end,
         ["end"],
-        "default_buffer_focused & (ebivim | ~vi_insert_mode)",
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
     ),
     Binding(
-        auto_suggest.accept_in_vi_insert_mode,
+        auto_suggest.accept_or_jump_to_end,
         ["c-e"],
-        "vi_insert_mode & default_buffer_focused & ebivim",
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
+    ),
+    Binding(
+        auto_suggest.accept,
+        ["c-f"],
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
+    ),
+    Binding(
+        auto_suggest.accept,
+        ["right"],
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
     ),
-    Binding(auto_suggest.accept, ["c-f"], "vi_insert_mode & default_buffer_focused"),
     Binding(
         auto_suggest.accept_word,
         ["escape", "f"],
-        "vi_insert_mode & default_buffer_focused & ebivim",
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
     ),
     Binding(
         auto_suggest.accept_token,
         ["c-right"],
-        "has_suggestion & default_buffer_focused",
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
     ),
     Binding(
         auto_suggest.discard,
         ["escape"],
+        # note this one is using `emacs_insert_mode`, not `emacs_like_insert_mode`
+        # as in `vi_insert_mode` we do not want `escape` to be shadowed (ever).
         "has_suggestion & default_buffer_focused & emacs_insert_mode",
     ),
     Binding(
@@ -236,22 +251,23 @@ def create_identifier(handler: Callable):
     Binding(
         auto_suggest.accept_character,
         ["escape", "right"],
-        "has_suggestion & default_buffer_focused",
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
     ),
     Binding(
         auto_suggest.accept_and_move_cursor_left,
         ["c-left"],
-        "has_suggestion & default_buffer_focused",
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
     ),
     Binding(
         auto_suggest.accept_and_keep_cursor,
         ["c-down"],
-        "has_suggestion & default_buffer_focused",
+        "has_suggestion & default_buffer_focused & emacs_like_insert_mode",
     ),
     Binding(
         auto_suggest.backspace_and_resume_hint,
         ["backspace"],
-        "has_suggestion & default_buffer_focused",
+        # no `has_suggestion` here to allow resuming if no suggestion
+        "default_buffer_focused & emacs_like_insert_mode",
     ),
 ]
 
diff --git a/IPython/terminal/shortcuts/auto_suggest.py b/IPython/terminal/shortcuts/auto_suggest.py
--- a/IPython/terminal/shortcuts/auto_suggest.py
+++ b/IPython/terminal/shortcuts/auto_suggest.py
@@ -2,6 +2,7 @@
 import tokenize
 from io import StringIO
 from typing import Callable, List, Optional, Union, Generator, Tuple
+import warnings
 
 from prompt_toolkit.buffer import Buffer
 from prompt_toolkit.key_binding import KeyPressEvent
@@ -178,9 +179,8 @@ def down(self, query: str, other_than: str, history: History) -> None:
                     break
 
 
-# Needed for to accept autosuggestions in vi insert mode
-def accept_in_vi_insert_mode(event: KeyPressEvent):
-    """Apply autosuggestion if at end of line."""
+def accept_or_jump_to_end(event: KeyPressEvent):
+    """Apply autosuggestion or jump to end of line."""
     buffer = event.current_buffer
     d = buffer.document
     after_cursor = d.text[d.cursor_position :]
@@ -193,6 +193,15 @@ def accept_in_vi_insert_mode(event: KeyPressEvent):
         nc.end_of_line(event)
 
 
+def _deprected_accept_in_vi_insert_mode(event: KeyPressEvent):
+    """Accept autosuggestion or jump to end of line.
+
+    .. deprecated:: 8.12
+        Use `accept_or_jump_to_end` instead.
+    """
+    return accept_or_jump_to_end(event)
+
+
 def accept(event: KeyPressEvent):
     """Accept autosuggestion"""
     buffer = event.current_buffer
@@ -373,3 +382,16 @@ def swap_autosuggestion_down(event: KeyPressEvent):
         provider=provider,
         direction_method=provider.down,
     )
+
+
+def __getattr__(key):
+    if key == "accept_in_vi_insert_mode":
+        warnings.warn(
+            "`accept_in_vi_insert_mode` is deprecated since IPython 8.12 and "
+            "renamed to `accept_or_jump_to_end`. Please update your configuration "
+            "accordingly",
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        return _deprected_accept_in_vi_insert_mode
+    raise AttributeError
diff --git a/IPython/terminal/shortcuts/filters.py b/IPython/terminal/shortcuts/filters.py
--- a/IPython/terminal/shortcuts/filters.py
+++ b/IPython/terminal/shortcuts/filters.py
@@ -181,10 +181,33 @@ def is_windows_os():
     "vi_mode": vi_mode,
     "vi_insert_mode": vi_insert_mode,
     "emacs_insert_mode": emacs_insert_mode,
+    # https://github.com/ipython/ipython/pull/12603 argued for inclusion of
+    # emacs key bindings with a configurable `emacs_bindings_in_vi_insert_mode`
+    # toggle; when the toggle is on user can access keybindigns like `ctrl + e`
+    # in vi insert mode. Because some of the emacs bindings involve `escape`
+    # followed by another key, e.g. `escape` followed by `f`, prompt-toolkit
+    # needs to wait to see if there will be another character typed in before
+    # executing pure `escape` keybinding; in vi insert mode `escape` switches to
+    # command mode which is common and performance critical action for vi users.
+    # To avoid the delay users employ a workaround:
+    # https://github.com/ipython/ipython/issues/13443#issuecomment-1032753703
+    # which involves switching `emacs_bindings_in_vi_insert_mode` off.
+    #
+    # For the workaround to work:
+    # 1) end users need to toggle `emacs_bindings_in_vi_insert_mode` off
+    # 2) all keybindings which would involve `escape` need to respect that
+    #    toggle by including either:
+    #      - `vi_insert_mode & ebivim` for actions which have emacs keybindings
+    #        predefined upstream in prompt-toolkit, or
+    #      - `emacs_like_insert_mode` for actions which do not have existing
+    #        emacs keybindings predefined upstream (or need overriding of the
+    #        upstream bindings to modify behaviour), defined below.
+    "emacs_like_insert_mode": (vi_insert_mode & ebivim) | emacs_insert_mode,
     "has_completions": has_completions,
     "insert_mode": vi_insert_mode | emacs_insert_mode,
     "default_buffer_focused": default_buffer_focused,
     "search_buffer_focused": has_focus(SEARCH_BUFFER),
+    # `ebivim` stands for emacs bindings in vi insert mode
     "ebivim": ebivim,
     "supports_suspend": supports_suspend,
     "is_windows_os": is_windows_os,


Test patch:
diff --git a/IPython/terminal/tests/test_shortcuts.py b/IPython/terminal/tests/test_shortcuts.py
--- a/IPython/terminal/tests/test_shortcuts.py
+++ b/IPython/terminal/tests/test_shortcuts.py
@@ -1,7 +1,7 @@
 import pytest
 from IPython.terminal.shortcuts.auto_suggest import (
     accept,
-    accept_in_vi_insert_mode,
+    accept_or_jump_to_end,
     accept_token,
     accept_character,
     accept_word,
@@ -22,6 +22,13 @@
 from unittest.mock import patch, Mock
 
 
+def test_deprected():
+    import IPython.terminal.shortcuts.auto_suggest as iptsa
+
+    with pytest.warns(DeprecationWarning, match=r"8\.12.+accept_or_jump_to_end"):
+        iptsa.accept_in_vi_insert_mode
+
+
 def make_event(text, cursor, suggestion):
     event = Mock()
     event.current_buffer = Mock()
@@ -80,7 +87,7 @@ def test_autosuggest_at_EOL(text, cursor, suggestion, called):
 
     event = make_event(text, cursor, suggestion)
     event.current_buffer.insert_text = Mock()
-    accept_in_vi_insert_mode(event)
+    accept_or_jump_to_end(event)
     if called:
         event.current_buffer.insert_text.assert_called()
     else: