Repository: ipython/ipython

Issue:
set_custom_completer seems to be a no-op
In [interactive_shell.py](https://github.com/ipython/ipython/blob/a2685d78f2403f84e6cb915ae11a4f6033ccc5f6/IPython/core/interactiveshell.py#L2034), `set_custom_completer` does

    self.Completer.matchers.insert(pos,newcomp)

Looking at the source code of IPCompleter, though, [matchers](https://github.com/ipython/ipython/blob/eed56ba34aa3856b5726d60eedfe60a17e1c26c5/IPython/core/completer.py#L1114) seems to be a property returning an ephemeral list, so the insert is a no-op.

To reproduce:

    import IPython

    def foo(*args, **kwargs):
        pass

    print(len(IPython.get_ipython().Completer.matchers))
    IPython.get_ipython().set_custom_completer(foo, 0)
    print(len(IPython.get_ipython().Completer.matchers))

This should print the same number twice (in my version, 5, 5)



Discussion:
Solving this is *almost* straightforward, but I think the interaction with `use_jedi` may cause some headaches. I see three reasonable ways to solve this:

1. Maintain two separate lists for the `use_jedi` and `not use_jedi` cases:

```python
    _matchers = {}
    @property
    def matchers(self):
        """All active matcher routines for completion"""
        if self.use_jedi:
            return self._matchers.setdefault('jedi', [
                self.file_matches,
                self.magic_matches,
                self.dict_key_matches,
            ])
        else:
            return self._matchers.setdefault('default', [
                self.python_matches,
                self.file_matches,
                self.magic_matches,
                self.python_func_kw_matches,
                self.dict_key_matches,
            ])
```

This will fix the issue as reported by making the lists persistent and mutable, *but* it means that if you use `set_custom_completer`, then change the value of `self.Completer.use_jedi`, your custom completers will be wiped out.

2. Make custom completers always come before the default completers and use a separate `custom_matchers` attribute. In this, you'd set `custom_matchers` in the `IPCompleter` constructor to `[]`, then `matchers` becomes:

```python
    @property
    def matchers(self):
        """All active matcher routines for completion"""
        if self.use_jedi:
            return [*self.custom_matchers,
                self.file_matches,
                self.magic_matches,
                self.dict_key_matches,
            ]
        else:
            return [*self.custom_matchers,
                self.python_matches,
                self.file_matches,
                self.magic_matches,
                self.python_func_kw_matches,
                self.dict_key_matches,
            ]
```

An alternate version of this is to allow custom matches to come either before or after the other matchers, but never be interpolated between them.

3. Change `use_jedi` to be a property that dynamically inserts or removes the python matchers when toggled. This would be somewhat complicated to implement, but once done the worst side effect would be that order would not be preserved if you inserted a custom matcher between one of the non-jedi matchers and one of the jedi-matchers, then turned jedi on and off again.
Thanks!

I really like your

> 
> An alternate version of this is to allow custom matches to come either before or after the other matchers, but never be interpolated between them.
> 

in your second suggestion, FWIW.  

If needed, I'd be happy to submit a pull request for it.
I think adding `self.custom_matchers` to the front of the list makes the most sense to me. Thanks for digging into this!
This is still an issue, isn't it? Shalll I submit a PR?  I'll use (2) from @pganssle's suggestions.
@Madhu94 I can reproduce with the latest version, so it looks like it hasn't been solved yet, so I suggest making a PR.
Looks like this is still an issue. Let's see if Paul's suggestion still works :)

Resolving patch:
diff --git a/IPython/core/completer.py b/IPython/core/completer.py
--- a/IPython/core/completer.py
+++ b/IPython/core/completer.py
@@ -626,6 +626,8 @@ def __init__(self, namespace=None, global_namespace=None, **kwargs):
         else:
             self.global_namespace = global_namespace
 
+        self.custom_matchers = []
+
         super(Completer, self).__init__(**kwargs)
 
     def complete(self, text, state):
@@ -1122,12 +1124,14 @@ def matchers(self):
 
         if self.use_jedi:
             return [
+                *self.custom_matchers,
                 self.file_matches,
                 self.magic_matches,
                 self.dict_key_matches,
             ]
         else:
             return [
+                *self.custom_matchers,
                 self.python_matches,
                 self.file_matches,
                 self.magic_matches,
diff --git a/IPython/core/interactiveshell.py b/IPython/core/interactiveshell.py
--- a/IPython/core/interactiveshell.py
+++ b/IPython/core/interactiveshell.py
@@ -2218,7 +2218,7 @@ def set_custom_completer(self, completer, pos=0):
         list where you want the completer to be inserted."""
 
         newcomp = types.MethodType(completer,self.Completer)
-        self.Completer.matchers.insert(pos,newcomp)
+        self.Completer.custom_matchers.insert(pos,newcomp)
 
     def set_completer_frame(self, frame=None):
         """Set the frame of the completer."""


Test patch:
diff --git a/IPython/core/tests/test_interactiveshell.py b/IPython/core/tests/test_interactiveshell.py
--- a/IPython/core/tests/test_interactiveshell.py
+++ b/IPython/core/tests/test_interactiveshell.py
@@ -998,3 +998,21 @@ def test_should_run_async():
     assert not ip.should_run_async("a = 5")
     assert ip.should_run_async("await x")
     assert ip.should_run_async("import asyncio; await asyncio.sleep(1)")
+
+
+def test_set_custom_completer():
+    num_completers = len(ip.Completer.matchers)
+
+    def foo(*args, **kwargs):
+        return "I'm a completer!"
+
+    ip.set_custom_completer(foo, 0)
+
+    # check that we've really added a new completer
+    assert len(ip.Completer.matchers) == num_completers + 1
+
+    # check that the first completer is the function we defined
+    assert ip.Completer.matchers[0]() == "I'm a completer!"
+
+    # clean up
+    ip.Completer.custom_matchers.pop()