pydata/pandas — pull request #3734 by y-p (2013-06-01 19:49:33+00:00)
Title:  BLD: test_perf.py, add --base-pickle --target-pickle options to test_perf

Until now test_perf either generated a single commit report for HEAD,
or fell back to vbench to compare two commits. with this change, can now
compare results between saved results of test_perf -H invocations.

Flow:
- Use build_cache (cdev from #3156) to jump to desired commit using build_cache
- checkout current vb_suite from upstream/master
- use `test_perf -H -d -foo.pickle` to save timings to file, for
target and base commits
- Use test_perf options `--base-pickle`, `--target-pickle ` to generate comparison report.

cc @jreback

```
λ ./test_perf.sh --base-pickle 31ecaa9-0.10.1.pickle --target-pickle f9eea30-0.11.0.pickle
This script compares the performance of two commits.
Make sure the python 'vbench' library is installed.

Setting the BUILD_CACHE_DIR env var to a temp directory will
potentially speed up subsequent runs.

base_commit not specified, Assuming base_pickle is named -foo.*
target_commit not specified, Assuming target_pickle is named -foo.*
f9eea30
Verifying specified commits exist in repo...
31ecaa9: RLS: set released to true
f9eea30: RLS: Version 0.11
***
Invoked with :
--ncalls: 3
--repeats: 3

-------------------------------------------------------------------------------
Test name | head[ms] | base[ms] | ratio |
-------------------------------------------------------------------------------
frame_reindex_columns | 0.3260 | 0.2900 | 1.1241 |
-------------------------------------------------------------------------------
Test name | head[ms] | base[ms] | ratio |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [f9eea30] : RLS: Version 0.11
Base [31ecaa9] : RLS: set released to true
```