evanphx/benchmark-ips — pull request #20 by headius (2014-11-10 17:50:28+00:00)
Title: Formatting improvements.

* Scale iter counts by 10^3s with suffix, for better readability.
* Reduce printed precision to 3 decimal places throughout.
* Fix stdddev percentage width to 4 characters, for alignment.

Here's the outputs before and after. I was always having trouble reading ips counts when they get into the hundreds of thousands or higher, and the stddev alignment thing was just bugging me. The ips scaling change also helps keep alignment consistent.

The precision changes warrant some justification. For the time elapsed, I figured that anything lower than ms is not useful for an "iterations per second" benchmark, and the only times shown are all around 5s range. For i/s, anything scaled 1/1000 is going to be in the neighborhood of benchmark noise. If something can do 50M iterations in 5 seconds, that's 10M per second, and 1000 iterations (the 1/10000 lost precision) takes 0.1ms. The error rate for that benchmark is +/- 1000 iterations, or 0.0002%. Acceptable to have good, readable results.

BEFORE:
```
Calculating -------------------------------------
uncontended MD5 'foo'
29296 i/100ms
uncontended MD5 'foo' * 1000
9558 i/100ms
4x contended MD5 'foo'
599 i/100ms
4x contended MD5 'foo' * 1000
519 i/100ms
10x contended MD5 'foo'
267 i/100ms
10x contended MD5 'foo' * 1000
229 i/100ms
100x contended MD5 'foo'
21 i/100ms
100x contended MD5 'foo' * 1000
19 i/100ms
-------------------------------------------------
uncontended MD5 'foo'
543446.5 (±5.2%) i/s - 2724528 in 5.028890s
uncontended MD5 'foo' * 1000
134059.2 (±2.9%) i/s - 678618 in 5.066376s
4x contended MD5 'foo'
123911.7 (±10.6%) i/s - 609782 in 4.994339s
4x contended MD5 'foo' * 1000
31893.7 (±6.9%) i/s - 158814 in 5.006711s
10x contended MD5 'foo'
47335.5 (±9.4%) i/s - 233625 in 4.994032s
10x contended MD5 'foo' * 1000
12726.5 (±5.5%) i/s - 63433 in 5.001239s
100x contended MD5 'foo'
2463.1 (±8.1%) i/s - 12222 in 4.999318s
100x contended MD5 'foo' * 1000
976.0 (±5.2%) i/s - 4883 in 5.018057s
```

AFTER:
```
Calculating -------------------------------------
uncontended MD5 'foo'
29.733k i/100ms
uncontended MD5 'foo' * 1000
9.572k i/100ms
4x contended MD5 'foo'
618.000 i/100ms
4x contended MD5 'foo' * 1000
531.000 i/100ms
10x contended MD5 'foo'
273.000 i/100ms
10x contended MD5 'foo' * 1000
234.000 i/100ms
100x contended MD5 'foo'
22.000 i/100ms
100x contended MD5 'foo' * 1000
19.000 i/100ms
-------------------------------------------------
uncontended MD5 'foo'
553.452k (± 3.5%) i/s - 2.765M in 5.003s
uncontended MD5 'foo' * 1000
134.654k (± 2.9%) i/s - 679.612k in 5.051s
4x contended MD5 'foo'
124.699k (±10.1%) i/s - 614.292k in 4.994s
4x contended MD5 'foo' * 1000
32.174k (± 6.9%) i/s - 160.362k in 5.012s
10x contended MD5 'foo'
47.166k (± 9.2%) i/s - 233.142k in 4.999s
10x contended MD5 'foo' * 1000
12.643k (± 5.6%) i/s - 63.180k in 5.015s
100x contended MD5 'foo'
2.513k (± 7.2%) i/s - 12.496k in 5.002s
100x contended MD5 'foo' * 1000
978.344 (± 5.1%) i/s - 4.883k in 5.006s
```