Repository: xonsh/xonsh

Issue:
Can not add completion function
I'm having trouble registering completion function for an alias. Here's a minimal example:

    def _f():
        def j():
            pass
    
        global aliases
        aliases['j'] = j
    
        def completions(pref, *args):
            return set(['hello', 'world'])
    
        completer add j completions "start"
    
    _f()
    del _f
    

## xonfig
+------------------+----------------------+
| xonsh            | 0.7.5.dev3           |
| Git SHA          | 8faf3a4c             |
| Commit Date      | Aug 18 22:45:11 2018 |
| Python           | 3.7.0                |
| PLY              | 3.9                  |
| have readline    | True                 |
| prompt toolkit   | 1.0.15               |
| shell type       | prompt_toolkit1      |
| pygments         | 2.2.0                |
| on posix         | True                 |
| on linux         | True                 |
| distro           | arch                 |
| on darwin        | False                |
| on windows       | False                |
| on cygwin        | False                |
| on msys2         | False                |
| is superuser     | False                |
| default encoding | utf-8                |
| xonsh encoding   | utf-8                |
| encoding errors  | surrogateescape      |
+------------------+----------------------+

## Expected Behavior
The function `completions` should be added as the completion function for command `j`

## Current Behavior
sourcing the above code emits the following error:


[FrameInfo(frame=<frame at 0x7fda763c0be8, file '/home/saga/.local/lib/python3.7/site-packages/xonsh/completers/_aliases.py', line 89, code _register_completer>, filename='/home/saga/.local/lib/python3.7/site-packages/xonsh/completers/_aliases.py', lineno=89, function='_register_completer', code_context=None, index=None), FrameInfo(frame=<frame at 0x7fda76334b88, file '/home/saga/.local/lib/python3.7/site-packages/xonsh/completers/_aliases.py', line 132, code completer_alias>, filename='/home/saga/.local/lib/python3.7/site-packages/xonsh/completers/_aliases.py', lineno=132, function='completer_alias', code_context=None, index=None), FrameInfo(frame=<frame at 0x7fda763efbb8, file '/home/saga/.local/lib/python3.7/site-packages/xonsh/proc.py', line 1172, code proxy_two>, filename='/home/saga/.local/lib/python3.7/site-packages/xonsh/proc.py', lineno=1172, function='proxy_two', code_context=None, index=None), FrameInfo(frame=<frame at 0x55ece10312d8, file '/home/saga/.local/lib/python3.7/site-packages/xonsh/proc.py', line 1351, code run>, filename='/home/saga/.local/lib/python3.7/site-packages/xonsh/proc.py', lineno=1351, function='run', code_context=None, index=None), FrameInfo(frame=<frame at 0x7fda7001a5d8, file '/usr/lib/python3.7/threading.py', line 917, code _bootstrap_inner>, filename='/usr/lib/python3.7/threading.py', lineno=917, function='_bootstrap_inner', code_context=None, index=None), FrameInfo(frame=<frame at 0x7fda76330980, file '/usr/lib/python3.7/threading.py', line 885, code _bootstrap>, filename='/usr/lib/python3.7/threading.py', lineno=885, function='_bootstrap', code_context=None, index=None)]
No such function: completions

## Steps to Reproduce
Execute the above mentioned snippet.


Discussion:
Hey @gsaga -- thanks for reporting!

A couple of things to note here --
First, if you move the definition of the `completions` function outside of your `_f` block you will be able to register it using the `completer add j completions "start"` command. 
I'm not sure why it's throwing an error in the way you've set it up, but that's a workaround for now.

Second, the name of the completer is just a name and it won't be default be associated with your alias `j`.  In fact, since you are adding it to the start of the completer list, you will get `("Hello", "World")` back as the completion results for nearly any command that you run.

If you want a completion to only fire for a certain command, you can add in some logic to check which function is being executed and if it isn't the command you want, then you can just return an empty set and the completers will continue firing in order. You can see an example of how that's done in the [`xompletions`](https://github.com/xonsh/xonsh/blob/master/xonsh/completers/xompletions.py#L7-L19) completer, where we only fire if the command is `xonfig` and have further checks for the secondary arguments.
>If you want a completion to only fire for a certain command, you can add in some logic to check which function is being executed and if it isn't the command you want, then you can just return an empty set and the completers will continue firing in order.

Shouldn't that logic be a part of the xonsh shell. It's different from any other shell I've used. What is the reason behind xonsh completion system being the way it is? It looks computationally expensive as well. If I have 1000 completion functions registered, the shell might possibly have to call all 1000 of them.
Hey @gsaga -- it could be made part of the xonsh shell but would require some rethinking of the completion framework.  It can be computationally expensive, although we take advantage of bash completions where available and that has removed the need for a lot of built-in completions.  I believe, too, that the prompt_toolkit version handles the completion calls in a non-blocking fashion so it shouldn't slow down user experience in a noticeable way.  

One of the hurdles to clear if we do something more akin to dynamic dispatch for completions is that the mixed context nature of xonsh can make it unclear which completion functions should be run in which situations.

 We're certainly open to a discussion of other options and any contributions that people want to make towards improving things!

Resolving patch:
diff --git a/xonsh/built_ins.py b/xonsh/built_ins.py
--- a/xonsh/built_ins.py
+++ b/xonsh/built_ins.py
@@ -397,6 +397,9 @@ def __init__(self, cmd, *, cls=subprocess.Popen, stdin=None, stdout=None,
             Handle to captured stdin
         captured_stderr : file-like
             Handle to captured stderr
+        stack : list of FrameInfo namedtuples or None
+            The stack of the call-site of alias, if the alias requires it.
+            None otherwise.
         """
         self._stdin = self._stdout = self._stderr = None
         # args
@@ -417,6 +420,7 @@ def __init__(self, cmd, *, cls=subprocess.Popen, stdin=None, stdout=None,
         self.last_in_pipeline = False
         self.captured_stdout = None
         self.captured_stderr = None
+        self.stack = None
 
     def __str__(self):
         s = self.__class__.__name__ + '(' + str(self.cmd) + ', '
@@ -577,6 +581,7 @@ def build(kls, cmd, *, cls=subprocess.Popen, **kwargs):
         spec.resolve_auto_cd()
         spec.resolve_executable_commands()
         spec.resolve_alias_cls()
+        spec.resolve_stack()
         return spec
 
     def redirect_leading(self):
@@ -666,6 +671,23 @@ def resolve_alias_cls(self):
         cpable = getattr(alias, '__xonsh_capturable__', self.captured)
         self.captured = cpable
 
+    def resolve_stack(self):
+        """Computes the stack for a callable alias's call-site, if needed."""
+        if not callable(self.alias):
+            return
+        # check that we actual need the stack
+        sig = inspect.signature(self.alias)
+        if len(sig.parameters) <= 5 and 'stack' not in sig.parameters:
+            return
+        # compute the stack, and filter out these build methods
+        # run_subproc() is the 4th command in the stack
+        # we want to filter out one up, e.g. subproc_captured_hiddenobject()
+        # after that the stack from the call site starts.
+        stack = inspect.stack(context=0)
+        assert stack[3][3] == 'run_subproc', 'xonsh stack has changed!'
+        del stack[:5]
+        self.stack = stack
+
 
 def _safe_pipe_properties(fd, use_tty=False):
     """Makes sure that a pipe file descriptor properties are sane."""
diff --git a/xonsh/completers/_aliases.py b/xonsh/completers/_aliases.py
--- a/xonsh/completers/_aliases.py
+++ b/xonsh/completers/_aliases.py
@@ -1,4 +1,3 @@
-import inspect
 import builtins
 import collections
 
@@ -38,7 +37,7 @@ def _add_one_completer(name, func, loc='end'):
     builtins.__xonsh_completers__.update(new)
 
 
-def _list_completers(args, stdin=None):
+def _list_completers(args, stdin=None, stack=None):
     o = "Registered Completer Functions: \n"
     _comp = builtins.__xonsh_completers__
     ml = max((len(i) for i in _comp), default=0)
@@ -53,7 +52,7 @@ def _list_completers(args, stdin=None):
     return o + '\n'.join(_strs) + '\n'
 
 
-def _remove_completer(args, stdin=None):
+def _remove_completer(args, stdin=None, stack=None):
     err = None
     if len(args) != 1:
         err = "completer remove takes exactly 1 argument."
@@ -69,7 +68,7 @@ def _remove_completer(args, stdin=None):
         return None, err + '\n', 1
 
 
-def _register_completer(args, stdin=None):
+def _register_completer(args, stdin=None, stack=None):
     err = None
     if len(args) not in {2, 3}:
         err = ("completer add takes either 2 or 3 arguments.\n"
@@ -86,8 +85,7 @@ def _register_completer(args, stdin=None):
                 if not callable(func):
                     err = "%s is not callable" % func_name
             else:
-                print(inspect.stack(context=0))
-                for frame_info in inspect.stack(context=0):
+                for frame_info in stack:
                     frame = frame_info[0]
                     if func_name in frame.f_locals:
                         func = frame.f_locals[func_name]
@@ -104,7 +102,8 @@ def _register_completer(args, stdin=None):
         return None, err + '\n', 1
 
 
-def completer_alias(args, stdin=None):
+def completer_alias(args, stdin=None, stdout=None, stderr=None, spec=None,
+                    stack=None):
     err = None
     if len(args) == 0 or args[0] not in (VALID_ACTIONS | {'help'}):
         err = ('Please specify an action.  Valid actions are: '
@@ -129,7 +128,7 @@ def completer_alias(args, stdin=None):
         func = _remove_completer
     elif args[0] == 'list':
         func = _list_completers
-    return func(args[1:], stdin=stdin)
+    return func(args[1:], stdin=stdin, stack=stack)
 
 COMPLETER_LIST_HELP_STR = """completer list: ordered list the active completers
 
diff --git a/xonsh/proc.py b/xonsh/proc.py
--- a/xonsh/proc.py
+++ b/xonsh/proc.py
@@ -1157,36 +1157,44 @@ def parse_proxy_return(r, stdout, stderr):
     return cmd_result
 
 
-def proxy_zero(f, args, stdin, stdout, stderr, spec):
+def proxy_zero(f, args, stdin, stdout, stderr, spec, stack):
     """Calls a proxy function which takes no parameters."""
     return f()
 
 
-def proxy_one(f, args, stdin, stdout, stderr, spec):
+def proxy_one(f, args, stdin, stdout, stderr, spec, stack):
     """Calls a proxy function which takes one parameter: args"""
     return f(args)
 
 
-def proxy_two(f, args, stdin, stdout, stderr, spec):
+def proxy_two(f, args, stdin, stdout, stderr, spec, stack):
     """Calls a proxy function which takes two parameter: args and stdin."""
     return f(args, stdin)
 
 
-def proxy_three(f, args, stdin, stdout, stderr, spec):
+def proxy_three(f, args, stdin, stdout, stderr, spec, stack):
     """Calls a proxy function which takes three parameter: args, stdin, stdout.
     """
     return f(args, stdin, stdout)
 
 
-def proxy_four(f, args, stdin, stdout, stderr, spec):
+def proxy_four(f, args, stdin, stdout, stderr, spec, stack):
     """Calls a proxy function which takes four parameter: args, stdin, stdout,
     and stderr.
     """
     return f(args, stdin, stdout, stderr)
 
 
-PROXIES = (proxy_zero, proxy_one, proxy_two, proxy_three, proxy_four)
-PROXY_KWARG_NAMES = frozenset(['args', 'stdin', 'stdout', 'stderr', 'spec'])
+def proxy_five(f, args, stdin, stdout, stderr, spec, stack):
+    """Calls a proxy function which takes four parameter: args, stdin, stdout,
+    stderr, and spec.
+    """
+    return f(args, stdin, stdout, stderr, spec)
+
+
+PROXIES = (proxy_zero, proxy_one, proxy_two, proxy_three, proxy_four, proxy_five)
+PROXY_KWARG_NAMES = frozenset(['args', 'stdin', 'stdout', 'stderr', 'spec',
+                               'stack'])
 
 
 def partial_proxy(f):
@@ -1198,13 +1206,13 @@ def partial_proxy(f):
             numargs += 1
         elif name in PROXY_KWARG_NAMES and param.kind == param.KEYWORD_ONLY:
             numargs += 1
-    if numargs < 5:
+    if numargs < 6:
         return functools.partial(PROXIES[numargs], f)
-    elif numargs == 5:
+    elif numargs == 6:
         # don't need to partial.
         return f
     else:
-        e = 'Expected proxy with 5 or fewer arguments for {}, not {}'
+        e = 'Expected proxy with 6 or fewer arguments for {}, not {}'
         raise XonshError(e.format(', '.join(PROXY_KWARG_NAMES), numargs))
 
 
@@ -1348,7 +1356,8 @@ def run(self):
             STDERR_DISPATCHER.register(sp_stderr), \
             redirect_stdout(STDOUT_DISPATCHER), \
             redirect_stderr(STDERR_DISPATCHER):
-                r = self.f(self.args, sp_stdin, sp_stdout, sp_stderr, spec)
+                r = self.f(self.args, sp_stdin, sp_stdout, sp_stderr, spec,
+                           spec.stack)
         except SystemExit as e:
             r = e.code if isinstance(e.code, int) else int(bool(e.code))
         except OSError as e:
@@ -1628,7 +1637,7 @@ def wait(self, timeout=None):
         stderr = self._pick_buf(self.stderr, sys.stderr, enc, err)
         # run the actual function
         try:
-            r = self.f(self.args, stdin, stdout, stderr, spec)
+            r = self.f(self.args, stdin, stdout, stderr, spec, spec.stack)
         except Exception:
             print_exception()
             r = 1


Test patch:
diff --git a/tests/test_integrations.py b/tests/test_integrations.py
--- a/tests/test_integrations.py
+++ b/tests/test_integrations.py
@@ -240,6 +240,28 @@ def _echo(args):
 echo Just the place for a snark. >tttt
 cat tttt
 """, 'Just the place for a snark.\n', 0),
+#
+# Test completion registration and subproc stack
+#
+("""
+def _f():
+    def j():
+        pass
+
+    global aliases
+    aliases['j'] = j
+
+    def completions(pref, *args):
+        return set(['hello', 'world'])
+
+    completer add j completions "start"
+
+
+_f()
+del _f
+
+"""
+, '', 0),
 ]