Repository: ipython/ipython

Issue:
Qtconsole docstring pop-up fails on method containing defaulted enum argument
[We've found](http://trac.mantidproject.org/mantid/ticket/8422) that an error is generated in the (admittedly rare) situation where a C++ enum is exposed to Python via boost python and included in a python function as a default to an argument. This is seen in IPython 1.1 and the current tip of master.

Here's the simplest example I could come up with, though it does still require compiling C++ and linking to boost python. In a C++ file:

``` c++
#include <boost/python.hpp>

enum MyEnum
{
  Red,
  Yellow,
  Blue
};

BOOST_PYTHON_MODULE(enum_test)
{
  using namespace boost::python;

  enum_<MyEnum>("MyEnum")
    .value("Red", Red)
    .value("Yellow", Yellow)
    .value("Blue", Blue);
}
```

This should be compiled to a shared library with something like `gcc -fPIC -I /usr/include/python2.6 -shared -o enum_test.so enum_test.cpp -lboost_python`

Then, in the IPython qtconsole enter:

```
In [1]: import enum_test

In [2]: def MyFunc(color = enum_test.MyEnum.Red):
   ...:     pass
   ...: 

In [3]: MyFunc(
```

On typing the opening parenthesis a stack trace will appear that culminates in:

```
  File "/usr/lib/python2.6/site-packages/IPython/kernel/zmq/session.py", line 83, in <lambda>
    json_unpacker = lambda s: extract_dates(jsonapi.loads(s))
  File "/usr/lib64/python2.6/site-packages/zmq/utils/jsonapi.py", line 81, in loads
    return jsonmod.loads(s, **kwargs)
  File "/usr/lib64/python2.6/site-packages/simplejson/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.6/site-packages/simplejson/decoder.py", line 335, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.6/site-packages/simplejson/decoder.py", line 353, in raw_decode
    raise ValueError("No JSON object could be decoded")
```

The string that's going into the raw_decode function of decoder.py and leads to the exception is:

```
{"base_class":"<type 'function'>","init_definition":null,"type_name":"function","name":"MyFunc","isclass":null,"namespace":"Interactive","isalias":false,"init_docstring":null,"argspec":{"args":["color"],"varkw":null,"defaults":[Red],"varargs":null},"source":null,"length":null,"call_def":null,"call_docstring":null,"file":"/home/enumproblem/<ipython-input-2-b6e10dea3e06>","string_form":"<function MyFunc at 0x16a8de8>","found":true,"class_docstring":null,"definition":"\u001b[0mMyFunc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcolor\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0menum_test\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMyEnum\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mRed\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n","docstring":"<no docstring>","ismagic":false}
```



Discussion:
This is an interesting case, and affects the `json` module [in general](http://bugs.python.org/issue12657), when serializing subclasses of integer that override `__str__` to something other than their integer value.

Pure Python illustration of the issue:

``` python
In [27]: class MyInt(int):
    def __str__(self):
        return 'int'

In [28]: i = MyInt(5)

In [29]: json.dumps(i)
Out[29]: 'int'

In [30]: json.loads(_)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-30-401dfaf3c283> in <module>()
----> 1 json.loads(_)

ValueError: No JSON object could be decoded

In [31]: json.dumps(int(i))
Out[31]: '5'
```

The ongoing work to replace argspec with Python 3-style signatures should actually address this, but in the meantime I think we can easily address the issue by recasting `float` and `int` subclasses to canonical `floats` and `ints` in `json_clean`.


Resolving patch:
diff --git a/IPython/utils/jsonutil.py b/IPython/utils/jsonutil.py
--- a/IPython/utils/jsonutil.py
+++ b/IPython/utils/jsonutil.py
@@ -194,9 +194,8 @@ def json_clean(obj):
     >>> json_clean(True)
     True
     """
-    # types that are 'atomic' and ok in json as-is.  bool doesn't need to be
-    # listed explicitly because bools pass as int instances
-    atomic_ok = (unicode_type, int, type(None))
+    # types that are 'atomic' and ok in json as-is.
+    atomic_ok = (unicode_type, type(None))
 
     # containers that we need to convert into lists
     container_to_list = (tuple, set, types.GeneratorType)
@@ -205,7 +204,14 @@ def json_clean(obj):
         # cast out-of-range floats to their reprs
         if math.isnan(obj) or math.isinf(obj):
             return repr(obj)
-        return obj
+        return float(obj)
+    
+    if isinstance(obj, int):
+        # cast int to int, in case subclasses override __str__ (e.g. boost enum, #4598)
+        if isinstance(obj, bool):
+            # bools are ints, but we don't want to cast them to 0,1
+            return obj
+        return int(obj)
 
     if isinstance(obj, atomic_ok):
         return obj


Test patch:
diff --git a/IPython/utils/tests/test_jsonutil.py b/IPython/utils/tests/test_jsonutil.py
--- a/IPython/utils/tests/test_jsonutil.py
+++ b/IPython/utils/tests/test_jsonutil.py
@@ -26,6 +26,9 @@
 #-----------------------------------------------------------------------------
 # Test functions
 #-----------------------------------------------------------------------------
+class Int(int):
+    def __str__(self):
+        return 'Int(%i)' % self
 
 def test():
     # list of input/expected output.  Use None for the expected output if it
@@ -48,6 +51,7 @@ def test():
              # More exotic objects
              ((x for x in range(3)), [0, 1, 2]),
              (iter([1, 2]), [1, 2]),
+             (Int(5), 5),
              ]
     
     for val, jval in pairs: