Repository: xonsh/xonsh

Issue:
abbrevs should only be expanded at the start of the command
<!--- Provide a general summary of the issue in the Title above -->
<!--- If you have a question along the lines of "How do I do this Bash command in xonsh"
please first look over the Bash to Xonsh translation guide: https://xon.sh/bash_to_xsh.html
If you don't find an answer there, please do open an issue! -->

## xonfig

<details>

```
+------------------+---------------------+
| xonsh            | 0.9.18              |
| Git SHA          | 26475146            |
| Commit Date      | May 5 06:42:28 2020 |
| Python           | 3.8.3               |
| PLY              | 3.11                |
| have readline    | True                |
| prompt toolkit   | 3.0.5               |
| shell type       | prompt_toolkit      |
| pygments         | 2.6.1               |
| on posix         | True                |
| on linux         | True                |
| distro           | manjaro             |
| on darwin        | False               |
| on windows       | False               |
| on cygwin        | False               |
| on msys2         | False               |
| is superuser     | False               |
| default encoding | utf-8               |
| xonsh encoding   | utf-8               |
| encoding errors  | surrogateescape     |
+------------------+---------------------+
```

</details>

<!--- Tell us what should happen -->
as of abbrevs expands in all places. So it may clash will subcommands. For example, if we have 

~~`aliases['ps'] = 'procs'`~~
```
abbrevs['ps'] = 'procs'
``` 

and do type `docker ps` then it will get expanded to `docker procs`. 

So the default behaviour should be to expand only at the start of the command. special handling for sudo can be added. 
Also we could have special symbols in aliases to mark it as expandable in all places like `aliases['*ps*']` like the glob pattern. 






Discussion:
I think that it does only expand at the beginning.  In this example, docker has a `ps` subcommand.

```
$ docker ps --help                                                                                                                                                                

Usage:	docker ps [OPTIONS]

List containers

Options:
  -a, --all             Show all containers (default shows just running)
  -f, --filter filter   Filter output based on conditions provided
      --format string   Pretty-print containers using a Go template
  -n, --last int        Show n last created containers (includes all states) (default -1)
  -l, --latest          Show the latest created container (includes all states)
      --no-trunc        Don't truncate output
  -q, --quiet           Only display numeric IDs
  -s, --size            Display total file sizes
```
This should show the issue
![image](https://user-images.githubusercontent.com/6702219/88026340-5bbc1b80-cb53-11ea-81da-bb77b9984a97.png)

In the description, it was a typo. I will change. The definition is in `abbrevs`
What is abbrevs?
https://xon.sh/xontribs.html#abbrevs
Oh OK! I am not fmailiar with that part of the code
This is a feature rather than a bug. I specifically wanted the abbrevs to be able to expand one or more subprocess arguments and Python expressions. You would also lose the ability to expand abbreviated commands in logical subprocess expressions (`foo and bar`) and pipelines (`foo | bar`) by limiting the expansion to the start of the command line. The expansion can be avoided by quoting the argument string (e.g. `docker "ps"`). So, I'm against adding such a limitation.
What about adding a possibility to "revert" the expansion for instance by pressing the space key once again after the expansion takes place?
I think the default workflow should not interfere with how things work usually. There are two things that will make it more correct. 

1. make it context aware. if it founds within subprocess/sudo then it could expand. we could add more cases as we find them and with full correctness.
2. have the shortcut that forces a word expansion even if it fails. as of now simple `space/enter` triggers it. we can make it to be `shift+space`

> 1. make it context aware. if it founds within subprocess/sudo then it could expand. we could add more cases as we find them and with full correctness.

Such a dictionary based rule is a no-go to me, simply because the number of command prefixes is infinite (`sudo`, `super`, `su -c`, `sh -c`, `env`, `ssh`, ...). We would also lose the Python mode expansion. Now we even can expand Xonsh/Python code in multiline...

> 2. have the shortcut that forces a word expansion even if it fails. as of now simple `space/enter` triggers it. we can make it to be `shift+space`

We are limited by what VT100/VT220 terminals and their emulators can do. Combinations of space key and modifiers are either non-existent (<kbd>Shift</kbd>+<kbd>Space</kbd>) or ambiguous (<kbd>Ctrl</kbd>+<kbd>Space</kbd>).
@laloch so you don't want to handle such clashes without disrupting how things usually work? please check the fish shell's way of doing. https://fishshell.com/docs/current/cmds/abbr.html

Atleast we need a way to restrict abbrevs that only expand at the start. Can we have something like `^abbrev` definitions that expand only at the start? 

also there must be some way to detect the context other than having a list of dictionary right?

Resolving patch:
diff --git a/xonsh/xontribs_meta.py b/xonsh/xontribs_meta.py
--- a/xonsh/xontribs_meta.py
+++ b/xonsh/xontribs_meta.py
@@ -2,9 +2,14 @@
 This modules is the place where one would define the xontribs.
 """
 
+import ast
 import functools
+import importlib.util
+from pathlib import Path
 import typing as tp
 
+from xonsh.lazyasd import LazyObject, lazyobject
+
 
 class _XontribPkg(tp.NamedTuple):
     """Class to define package information of a xontrib.
@@ -43,11 +48,20 @@ class Xontrib(tp.NamedTuple):
     """
 
     url: str = ""
-    description: str = ""
+    description: tp.Union[str, LazyObject] = ""
     package: tp.Optional[_XontribPkg] = None
     tags: tp.Tuple[str, ...] = ()
 
 
+def get_module_docstring(module: str) -> str:
+    """Find the module and return its docstring without actual import"""
+
+    spec = importlib.util.find_spec(module)
+    if spec and spec.has_location and spec.origin:
+        return ast.get_docstring(ast.parse(Path(spec.origin).read_text()))
+    return ""
+
+
 @functools.lru_cache()
 def get_xontribs() -> tp.Dict[str, Xontrib]:
     """Return xontrib definitions lazily."""
@@ -70,20 +84,7 @@ def define_xontribs():
     return {
         "abbrevs": Xontrib(
             url="http://xon.sh",
-            description="Adds ``abbrevs`` dictionary to hold user-defined "
-            "command abbreviations. The dictionary is searched "
-            "as you type and the matching words are replaced "
-            "at the command line by the corresponding "
-            "dictionary contents once you hit 'Space' or "
-            "'Return' key. For instance a frequently used "
-            "command such as ``git status`` can be abbreviated "
-            "to ``gst`` as follows::\n"
-            "\n"
-            "    $ xontrib load abbrevs\n"
-            "    $ abbrevs['gst'] = 'git status'\n"
-            "    $ gst # Once you hit <space> or <return>, "
-            "'gst' gets expanded to 'git status'.\n"
-            "\n",
+            description=lazyobject(lambda: get_module_docstring("xontrib.abbrevs")),
             package=core_pkg,
         ),
         "apt_tabcomplete": Xontrib(
diff --git a/xontrib/abbrevs.py b/xontrib/abbrevs.py
--- a/xontrib/abbrevs.py
+++ b/xontrib/abbrevs.py
@@ -2,12 +2,36 @@
 Command abbreviations.
 
 This expands input words from `abbrevs` disctionary as you type.
+Adds ``abbrevs`` dictionary to hold user-defined "command abbreviations.
+The dictionary is searched as you type the matching words are replaced
+at the command line by the corresponding dictionary contents once you hit
+'Space' or 'Return' key.
+
+For instance a frequently used command such as ``git status`` can be abbreviated to ``gst`` as follows::
+
+    $ xontrib load abbrevs
+    $ abbrevs['gst'] = 'git status'
+    $ gst # Once you hit <space> or <return> 'gst' gets expanded to 'git status'.
+
+one can set a callback function that receives current buffer and word to customize the expanded word based on context
+
+.. code-block:: python
+
+    $ abbrevs['ps'] = lambda buffer, word: "procs" if buffer.text.startswith(word) else word
+
+
+It is also possible to set the cursor position after expansion with,
+
+    $ abbrevs['gp'] = "git push <edit> --force"
 """
 
 import builtins
+import typing as tp
+
 from prompt_toolkit.filters import completion_is_selected, IsMultiline
 from prompt_toolkit.keys import Keys
 from xonsh.built_ins import DynamicAccessProxy
+from xonsh.events import events
 from xonsh.tools import check_for_partial_string
 
 __all__ = ()
@@ -16,59 +40,70 @@
 proxy = DynamicAccessProxy("abbrevs", "__xonsh__.abbrevs")
 setattr(builtins, "abbrevs", proxy)
 
-last_expanded = None
+
+class _LastExpanded(tp.NamedTuple):
+    word: str
+    expanded: str
+
+
+last_expanded: tp.Optional[_LastExpanded] = None
+EDIT_SYMBOL = "<edit>"
 
 
-def expand_abbrev(buffer):
+def get_abbreviated(key: str, buffer) -> str:
+    abbrevs = getattr(builtins, "abbrevs", None)
+    abbr = abbrevs[key]
+    if callable(abbr):
+        text = abbr(buffer=buffer, word=key)
+    else:
+        text = abbr
+    return text
+
+
+def expand_abbrev(buffer) -> bool:
+    """expand the given abbr text. Return true if cursor position changed."""
     global last_expanded
     last_expanded = None
     abbrevs = getattr(builtins, "abbrevs", None)
     if abbrevs is None:
-        return
+        return False
     document = buffer.document
     word = document.get_word_before_cursor(WORD=True)
     if word in abbrevs.keys():
         partial = document.text[: document.cursor_position]
         startix, endix, quote = check_for_partial_string(partial)
         if startix is not None and endix is None:
-            return
+            return False
         buffer.delete_before_cursor(count=len(word))
-        buffer.insert_text(abbrevs[word])
-        last_expanded = word
+        text = get_abbreviated(word, buffer)
+        buffer.insert_text(text)
+        last_expanded = _LastExpanded(word, text)
+        if EDIT_SYMBOL in text:
+            set_cursor_position(buffer, text)
+            return True
+    return False
 
 
-def revert_abbrev(buffer):
+def revert_abbrev(buffer) -> bool:
     global last_expanded
     if last_expanded is None:
         return False
-    abbrevs = getattr(builtins, "abbrevs", None)
-    if abbrevs is None:
-        return False
-    if last_expanded not in abbrevs.keys():
-        return False
     document = buffer.document
-    expansion = abbrevs[last_expanded] + " "
+    expansion = last_expanded.expanded + " "
     if not document.text_before_cursor.endswith(expansion):
         return False
     buffer.delete_before_cursor(count=len(expansion))
-    buffer.insert_text(last_expanded)
+    buffer.insert_text(last_expanded.word)
     last_expanded = None
     return True
 
 
-def set_cursor_position(buffer):
-    abbrevs = getattr(builtins, "abbrevs", None)
-    if abbrevs is None:
-        return False
-    global last_expanded
-    abbr = abbrevs[last_expanded]
-    pos = abbr.rfind("<edit>")
+def set_cursor_position(buffer, expanded: str) -> None:
+    pos = expanded.rfind(EDIT_SYMBOL)
     if pos == -1:
-        return False
-    buffer.cursor_position = buffer.cursor_position - (len(abbr) - pos)
-    buffer.delete(6)
-    last_expanded = None
-    return True
+        return
+    buffer.cursor_position = buffer.cursor_position - (len(expanded) - pos)
+    buffer.delete(len(EDIT_SYMBOL))
 
 
 @events.on_ptk_create
@@ -83,9 +118,13 @@ def custom_keybindings(bindings, **kw):
     @handler(" ", filter=IsMultiline() & insert_mode)
     def handle_space(event):
         buffer = event.app.current_buffer
+
+        add_space = True
         if not revert_abbrev(buffer):
-            expand_abbrev(buffer)
-        if last_expanded is None or not set_cursor_position(buffer):
+            position_changed = expand_abbrev(buffer)
+            if position_changed:
+                add_space = False
+        if add_space:
             buffer.insert_text(" ")
 
     @handler(


Test patch:
diff --git a/tests/xontribs/test_abbrevs.py b/tests/xontribs/test_abbrevs.py
new file mode 100644
--- /dev/null
+++ b/tests/xontribs/test_abbrevs.py
@@ -0,0 +1,47 @@
+"""test xontrib.abbrevs"""
+
+import importlib
+import sys
+
+from prompt_toolkit.buffer import Buffer
+from pytest import fixture, mark
+from xonsh.xontribs import find_xontrib
+import builtins
+
+
+@fixture
+def _buffer():
+    def _wrapper(text):
+        buf = Buffer()
+        buf.insert_text(text)
+        return buf
+
+    return _wrapper
+
+
+@fixture
+def abbrevs_xontrib(monkeypatch, source_path):
+    monkeypatch.syspath_prepend(source_path)
+    spec = find_xontrib("abbrevs")
+    yield importlib.import_module(spec.name)
+    del sys.modules[spec.name]
+
+
+@mark.parametrize(
+    "abbr,val,expanded,cur",
+    [
+        ("ps", "procs", "procs", None),
+        ("ps", lambda **kw: "callback", "callback", None),
+        ("kill", "kill <edit> -9", "kill  -9", 5),
+        ("pt", "poe<edit>try", "poetry", 3),
+    ],
+)
+def test_gets_expanded(abbr, val, expanded, cur, abbrevs_xontrib, _buffer):
+    builtins.abbrevs[abbr] = val
+    from xontrib.abbrevs import expand_abbrev
+
+    buf = _buffer(abbr)
+    expand_abbrev(buf)
+    assert buf.text == expanded
+    if cur is not None:
+        assert buf.cursor_position == cur