pydata/pandas — pull request #6137 by Komnomnomnom (2014-01-28 08:05:02+00:00)
Title: PERF: fix JSON performance regression from 0.12 (GH5765)

This fixes the main JSON performance regression in v0.13 (see #5765). The main bottleneck was the use of intermediate NumPy scalars (from PR #4498). I introduced code to avoid the use of NumPy scalars. Also:

- checks current locale so locale fudging is only done if necessary
- added numpy 1.6 preprocessor switches as it still requires the use of scalars due to its weird datetime handling
- reorgranised some of the if/else checks in `objToJSON.c` based on likelihood.
- added some JSON benchmarks to vbench to try and avoid future regressions.

0.12
```python
In [1]: import pandas as pd, numpy as np

In [2]: df = pd.DataFrame(np.random.rand(100000,10))

In [3]: %timeit df.to_json(orient='split')
10 loops, best of 3: 119 ms per loop

In [4]: pd.__version__, np.__version__
Out[4]: ('0.12.0', '1.8.0')
```

This PR
```python
In [1]: import pandas as pd, numpy as np

In [2]: df = pd.DataFrame(np.random.rand(100000,10))

In [3]: %timeit df.to_json(orient='split')
10 loops, best of 3: 119 ms per loop

In [4]: pd.__version__, np.__version__
Out[4]: ('0.13.0-406-ga18e5e6', '1.8.0')
```

While this solves the main performance issue, using vbench this PR still appears to be a bit slower than v0.12 (~ 2 - 5%) but it's unclear where the slowdown is coming from.

@jreback do you still have a windows box / vm that you could test on?