Repository: ipython/ipython

Issue:
Autocall fails if first function argument begins with "-" or "+
Original Launchpad bug 315706: https://bugs.launchpad.net/ipython/+bug/315706
Reported by: dmuders (Dirk Muders).

Auto parentheses fail if the first argument of a function begins with "-" or "+". Thus these examples fail:

```
%autocall 2
def f(a):
    print a

f -1

f +1
```

The reason is the re_exclude_auto in prefilter.py which reads

```
re_exclude_auto = re.compile(r'^[,&^\|\*/\+-]'
                                                   r'|^is |^not |^in |^and |^or ')
```

in v0.9.1. It appears that this line had been changed at some point after v0.6.2 to also catch operators. However, "-" and "+" can be used with numbers as shown above. Changing the line to

```
re_exclude_auto = re.compile(r'^[,&^\|\*/]'
                                                   r'|^is |^not |^in |^and |^or ')
```

fixes the problem but I do not know if something else was intended by adding "+-". If so, then this would need to be caught in a different way.



Discussion:
[ LP comment 1 by: Fernando Perez, on 2010-04-25 01:48:04.569902+00:00 ]

No, see this comment in the code:

```
# RegExp to exclude strings with this start from autocalling.  In
# particular, all binary operators should be excluded, so that if foo is
# callable, foo OP bar doesn't become foo(OP bar), which is invalid.
```

Since any callable X could potentially also implement arithmetic, so that

```
X+1
```

is as valid as (and different from!):

```
X(1)
```

We can't autocall on _any_ binary operator.

[ LP comment 2 by: Dirk Muders, on 2010-04-26 07:54:38.604015+00:00 ]

I understand that argument. In our application we have created a user CLI based on IPython, partially because of being able to omit the parentheses. This makes it easier for users who are unfamiliar with Python but who have used other interfaces that do not need the parentheses. The case with signs (of course mostly the "-" sign) as first characters often occurs in our interface, hence the default IPython behavior breaks the general behavior of our interface and is thus undesirable. We are working with a locally modified version of IPython, but it would be nice to have an "official" switch to choose between the two options. 

[ LP comment 3 by: Fernando Perez, on 2010-04-26 19:21:31.005203+00:00 ]

Well, if you are embedding IPython yourselves, you can try doing something like (this is code for trunk, the imports will be slightly different for 0.10):

```
import re
from IPython.core import prefilter

prefilter.re_exclude_auto = re.compile('your regexp here')

# rest of your code...
```

Let me know if this works for you.  If it does what you need OK, you have a workaround for now, and I'll then refactor the code to have an official API for this.

I've reopened the bug so it stays tracked.

[ LP comment 4 by: Dirk Muders, on 2010-04-27 07:23:54.060105+00:00 ]

We are using IPython directly and we are currently working with a version that we edited in prefilter.py. Next time we upgrade IPython we would have to re-introduce this modification. For the time being we can work this way.

[ LP comment 5 by: Fernando Perez, on 2010-04-27 18:08:52+00:00 ]

OK.  Since the upcoming 0.11 release is going to be somewhat of a
'tech preview' (many apis will change), when we put it out we're going
to ask for a lot of feedback from anyone making embedded uses of it.
At that point, we can look at a good api for this (and possibly other
filtering-related things).

Do ping us to remind us when the discussion on api requests comes up,
we want feedback from embedded users on all possible needs.

Cheers,

f

@fperez, from reading the discussion, it seems like this is a 'working as intended' thing.  Or do we want to keep it open as a feature request for optional disabling of autocall protection of binary operators?


Resolving patch:
diff --git a/IPython/core/prefilter.py b/IPython/core/prefilter.py
--- a/IPython/core/prefilter.py
+++ b/IPython/core/prefilter.py
@@ -35,7 +35,9 @@
 from IPython.core.splitinput import split_user_input, LineInfo
 from IPython.core import page
 
-from IPython.utils.traitlets import List, Integer, Any, Unicode, CBool, Bool, Instance
+from IPython.utils.traitlets import (
+    List, Integer, Any, Unicode, CBool, Bool, Instance, CRegExp
+)
 from IPython.utils.autoattr import auto_attr
 
 #-----------------------------------------------------------------------------
@@ -659,6 +661,11 @@ class AutocallChecker(PrefilterChecker):
 
     priority = Integer(1000, config=True)
 
+    function_name_regexp = CRegExp(re_fun_name, config=True,
+        help="RegExp to identify potential function names.")
+    exclude_regexp = CRegExp(re_exclude_auto, config=True,
+        help="RegExp to exclude strings with this start from autocalling.")
+
     def check(self, line_info):
         "Check if the initial word/function is callable and autocall is on."
         if not self.shell.autocall:
@@ -669,8 +676,8 @@ def check(self, line_info):
             return None
 
         if callable(oinfo['obj']) \
-               and (not re_exclude_auto.match(line_info.the_rest)) \
-               and re_fun_name.match(line_info.ifun):
+               and (not self.exclude_regexp.match(line_info.the_rest)) \
+               and self.function_name_regexp.match(line_info.ifun):
             return self.prefilter_manager.get_handler_by_name('auto')
         else:
             return None
diff --git a/IPython/utils/traitlets.py b/IPython/utils/traitlets.py
--- a/IPython/utils/traitlets.py
+++ b/IPython/utils/traitlets.py
@@ -1411,3 +1411,17 @@ def validate(self, obj, value):
                     if port >= 0 and port <= 65535:
                         return value
         self.error(obj, value)
+
+class CRegExp(TraitType):
+    """A casting compiled regular expression trait.
+
+    Accepts both strings and compiled regular expressions. The resulting
+    attribute will be a compiled regular expression."""
+
+    info_text = 'a regular expression'
+
+    def validate(self, obj, value):
+        try:
+            return re.compile(value)
+        except:
+            self.error(obj, value)


Test patch:
diff --git a/IPython/core/tests/test_prefilter.py b/IPython/core/tests/test_prefilter.py
--- a/IPython/core/tests/test_prefilter.py
+++ b/IPython/core/tests/test_prefilter.py
@@ -5,6 +5,7 @@
 #-----------------------------------------------------------------------------
 import nose.tools as nt
 
+from IPython.core.prefilter import AutocallChecker
 from IPython.testing import tools as tt, decorators as dec
 from IPython.testing.globalipapp import get_ipython
 
@@ -46,6 +47,21 @@ def test_autocall_binops():
         yield nt.assert_equals(ip.prefilter('f 1'),'f(1)')
         for t in ['f +1', 'f -1']:
             yield nt.assert_equals(ip.prefilter(t), t)
+
+        # Run tests again with a more permissive exclude_regexp, which will
+        # allow transformation of binary operations ('f -1' -> 'f(-1)').
+        pm = ip.prefilter_manager
+        ac = AutocallChecker(shell=pm.shell, prefilter_manager=pm,
+                             config=pm.config)
+        try:
+            ac.priority = 1
+            ac.exclude_regexp = r'^[,&^\|\*/]|^is |^not |^in |^and |^or '
+            pm.sort_checkers()
+
+            yield nt.assert_equals(ip.prefilter('f -1'), 'f(-1)')
+            yield nt.assert_equals(ip.prefilter('f +1'), 'f(+1)')
+        finally:
+            pm.unregister_checker(ac)
     finally:
         ip.magic('autocall 0')
         del ip.user_ns['f']