Repository: ipython/ipython

Issue:
2 test failures in Python 3.4 in parallel group
Not sure where you are on Python 3.4, but I'm getting a couple of test failures. This is on Ubuntu 12.04, and only occurs when I kick off the tests from within an IPython notebook -- even though they're in a totally separate Python process started via `subprocess.call`.

Steps:

Clone Snake Charmer: https://github.com/andrewclegg/snake-charmer.git

Checkout this revision (so as to avoid any confounding factors): 7d5e90e8b8816b0a28b037cf3094cdf4f24bf19f

Start up a VM: `vagrant up`

... wait about an hour ...

Open notebook browser on VM: [http://localhost:8834/tree](http://localhost:8834/tree)

Open "Snake Charmer QA" notebook

Run the first code cell (under "Initialization"), then the second (under "Serial tests")

The first test to complete should be

```
python3.4 `which iptest3` --all -j
```

with exit code 1.

When it completes, check out `log/charmed34/test_output/ipython.out` (inside your `snake-charmer` directory)

I see these two errors:

https://gist.github.com/andrewclegg/823d005491850d7f66cb

One of them is an instance of the error described here -- https://github.com/ipython/ipython/pull/5291 -- the other is in test_elapsed_single and looks maybe unrelated?

The interesting thing is that when I ssh into the VM and run the tests from bash, using the exact same command line I passed to `subprocess.call`, those tests pass.



Discussion:
Another data point:

The scenario I described above, and the test transcript I included, were on a Snake Charmer VM with 16 vCPUs. That is, I did this

```
export CHARMER_CPUS=16
```

before spinning it up. (Also `CHARMER_RAM=8000`.) I didn't think to include this info in the OP as it didn't seem relevant.

I recently retried on a fresh VM with only 12 vCPUs, and **all of the tests passed** -- not as much as a warning anywhere.

From my point of view I'm happy to consider this closed, as my top priority is getting tests passing in Snake Charmer, which this has achieved :-) ... But if you would rather leave it open, since it suggests something a bit wrong in IPython.parallel, no complaints from me.

The first failure is a real bug that has been eluding me for months. The second failure is probably just an issue in the test, not properly handling slow or overworked test bots.


Resolving patch:
diff --git a/IPython/utils/jsonutil.py b/IPython/utils/jsonutil.py
--- a/IPython/utils/jsonutil.py
+++ b/IPython/utils/jsonutil.py
@@ -47,22 +47,18 @@
 def rekey(dikt):
     """Rekey a dict that has been forced to use str keys where there should be
     ints by json."""
-    for k in dikt:
+    for k in list(dikt):
         if isinstance(k, string_types):
-            ik=fk=None
+            nk = None
             try:
-                ik = int(k)
+                nk = int(k)
             except ValueError:
                 try:
-                    fk = float(k)
+                    nk = float(k)
                 except ValueError:
                     continue
-            if ik is not None:
-                nk = ik
-            else:
-                nk = fk
             if nk in dikt:
-                raise KeyError("already have key %r"%nk)
+                raise KeyError("already have key %r" % nk)
             dikt[nk] = dikt.pop(k)
     return dikt
 


Test patch:
diff --git a/IPython/utils/tests/test_jsonutil.py b/IPython/utils/tests/test_jsonutil.py
--- a/IPython/utils/tests/test_jsonutil.py
+++ b/IPython/utils/tests/test_jsonutil.py
@@ -64,6 +64,13 @@ def test():
         json.loads(json.dumps(out))
 
 
+def test_rekey():
+    # This could fail due to modifying the dict keys in-place on Python 3
+    d = { i:i for i in map(str, range(128)) }
+    d = jsonutil.rekey(d)
+    for key in d:
+        nt.assert_is_instance(key, int)
+
 
 def test_encode_images():
     # invalid data, but the header and footer are from real files