Shopify/liquid — pull request #426 by jasonhl (2014-08-21 18:47:14+00:00)
Title: Reduce benchmark volatility by disabling GC

Sparked by https://github.com/Shopify/liquid/pull/415#issuecomment-51926314, I took a look at why the Liquid benchmark had a standard deviation of 15-20%. The answer appears to be GC. I can now get the standard deviation consistently down below 5%, and often in the 2-3% range.

Three back to back runs on my machine:

```
parse: 58.4 (±1.7%) i/s - 588 in 10.068657s
parse: 58.2 (±1.7%) i/s - 585 in 10.060376s
parse: 57.3 (±3.5%) i/s - 575 in 10.047092s

parse & run: 25.3 (±3.9%) i/s - 254 in 10.042037s
parse & run: 24.9 (±4.0%) i/s - 248 in 10.004997s
parse & run: 25.1 (±4.0%) i/s - 252 in 10.060830s
```

Output now looks a bit different due to trying to reduce memory usage and volatility between runs:

```
/usr/lib/shopify-ruby/2.1.2-shopify1/bin/ruby ./performance/benchmark.rb lax

Running parse: benchmark for 10 seconds (with 1 seconds warmup).

Calculating -------------------------------------
parse: 5 i/100ms
-------------------------------------------------
parse: 57.3 (±3.5%) i/s - 575 in 10.047092s

Running parse & run: benchmark for 10 seconds (with 1 seconds warmup).

Calculating -------------------------------------
parse & run: 2 i/100ms
-------------------------------------------------
parse & run: 25.1 (±4.0%) i/s - 252 in 10.060830s
```

@Shopify/liquid