Repository: ipython/ipython

Issue:
NaN in object_info_reply JSON causes parse error
When I send `object_info_request` for `pandas.DataFrame.reindex`, I've got the following JSON reply.

Please look the list at `content.defaults`.  It contains `NaN`.  This causes parse error in Emacs JSON parser I am using for my Emacs Notebook Client tkf/emacs-ipython-notebook#38.

``` js
{ "content" : { "argspec" : { "args" : [ "self",
              "index",
              "columns",
              "method",
              "level",
              "fill_value",
              "copy"
            ],
          "defaults" : [ null,
              null,
              null,
              null,
              NaN,
              true
            ],
          "varargs" : null,
          "varkw" : null
        },
      "base_class" : "<type 'instancemethod'>",
      "call_def" : null,
      "call_docstring" : null,
      "class_docstring" : null,
      "definition" : "\u001b[0mpandas\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreindex\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcolumns\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmethod\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfill_value\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mnan\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mTrue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "docstring" : "Conform DataFrame to new index with optional filling logic, placing\nNA/NaN in locations having no value in the previous index. A new object\nis produced unless the new index is equivalent to the current one and\ncopy=False\n\nParameters\n----------\nindex : array-like, optional\n    New labels / index to conform to. Preferably an Index object to\n    avoid duplicating data\ncolumns : array-like, optional\n    Same usage as index argument\nmethod : {'backfill', 'bfill', 'pad', 'ffill', None}, default None\n    Method to use for filling holes in reindexed DataFrame\n    pad / ffill: propagate last valid observation forward to next valid\n    backfill / bfill: use NEXT valid observation to fill gap\ncopy : boolean, default True\n    Return a new object, even if the passed indexes are the same\nlevel : int or name\n    Broadcast across a level, matching Index values on the\n    passed MultiIndex level\nfill_value : scalar, default np.NaN\n    Value to use for missing values. Defaults to NaN, but can be any\n    \"compatible\" value\n\nExamples\n--------\n>>> df.reindex(index=[date1, date2, date3], columns=['A', 'B', 'C'])\n\nReturns\n-------\nreindexed : same type as calling instance",
      "file" : "/home/takafumi/.virtualenvs/mypy/local/lib/python2.7/site-packages/pandas/core/frame.py",
      "found" : true,
      "init_definition" : null,
      "init_docstring" : null,
      "isalias" : false,
      "isclass" : null,
      "ismagic" : false,
      "length" : null,
      "name" : "pandas.DataFrame.reindex",
      "namespace" : "Interactive",
      "source" : null,
      "string_form" : "<unbound method DataFrame.reindex>",
      "type_name" : "instancemethod"
    },
  "header" : { "msg_id" : "6c393862-7412-4a5f-a4b2-2993f5430a6d",
      "msg_type" : "object_info_reply",
      "session" : "ce9c7083-370e-416f-8fe9-111b68410228",
      "username" : "kernel"
    },
  "msg_id" : "6c393862-7412-4a5f-a4b2-2993f5430a6d",
  "msg_type" : "object_info_reply",
  "parent_header" : { "msg_id" : "d83e0578-d143-48b6-a672-f123034ed316",
      "msg_type" : "object_info_request",
      "session" : "55e244de-e83e-496f-805c-883e62c70462",
      "username" : "username"
    }
}
```

You can check that indeed it's not a valid JSON using http://jsonlint.com/:

```
Parse error on line 17:
...ll,                NaN,               
----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
```



Discussion:
Well that's awkward.

The error, of course, comes in writing out the call specification for `pandas.core.frame.DataFrame.reindex` which is:

```
    def reindex(self, index=None, columns=None, method=None, level=None,
                fill_value=np.nan, limit=None, copy=True):
```

Apparently there is no NaN or +/-Inf in the JSON spec.  There some stack overflow questions about this, like [Python: sending NaN in JSON](http://stackoverflow.com/questions/6601812/sending-nan-in-json).

I'm not sure what an appropriate resolution for this issue is yet.

Well, reading the stackoverflow it looks like browsers support it so maybe it is fine.
But I guess sending `repr()` should do the job, as it is only used for UI (not used for something like remote procedure call).

stdlib json has an `allow_nan` flag, so we can set allow_nan=False to catch this (but not fix it).

We already have a json_clean method (IPython.utils.jsonutil), which is where we handle non-jsonable objects, so that's where the fix should be.


Resolving patch:
diff --git a/IPython/utils/jsonutil.py b/IPython/utils/jsonutil.py
--- a/IPython/utils/jsonutil.py
+++ b/IPython/utils/jsonutil.py
@@ -11,6 +11,7 @@
 # Imports
 #-----------------------------------------------------------------------------
 # stdlib
+import math
 import re
 import sys
 import types
@@ -161,11 +162,17 @@ def json_clean(obj):
     """
     # types that are 'atomic' and ok in json as-is.  bool doesn't need to be
     # listed explicitly because bools pass as int instances
-    atomic_ok = (unicode, int, float, types.NoneType)
+    atomic_ok = (unicode, int, types.NoneType)
     
     # containers that we need to convert into lists
     container_to_list = (tuple, set, types.GeneratorType)
     
+    if isinstance(obj, float):
+        # cast out-of-range floats to their reprs
+        if math.isnan(obj) or math.isinf(obj):
+            return repr(obj)
+        return obj
+    
     if isinstance(obj, atomic_ok):
         return obj
     


Test patch: