rubinius/rubinius-benchmark — pull request #4 by y8 (2014-01-16 13:41:33+00:00)
Title: 'benchmark-dsl' framework (WIP)

I need your feedback on this implementation, but be aware that it's still a bit WIP.

* It uses DRb, since it is a part of stdlib, and it's available on all major implementations. It can be easily replaced with unix sockets, but I don't see why it will be better that DRb.
* It lacks report formatters right now, I'll add them later this week
* It can be easily modified to collect more data see (ideas welcome)
* It lacks support of "bm_" cases, since I'm not sure how they should be measured, since "i/s" is not applicable here. Ideas welcome as well

Also, I suggest to separate benchmark cases and benchmarking framework. It will simplify interaction with cases (no need to care about "bin/bm")

Only few array cases ported now, I'll port all remaining "core" cases as soon, as you will be okay with this solution.

You can try it right now: ``bin/bm2 -r rbx, 2.0, 2.1 Array``

Tested with: rbx, mri 1.9.3-2.1 (latest stables), jruby.

TODO:
- [ ] Add formatters (column & json for starters)
- [ ] Decide how "bm_" cases should be measured
- [ ] Implement "bm_" measurements
- [ ] Cover with documentation