evanphx/benchmark_suite — pull request #7 by headius (2012-01-19 15:43:01+00:00)
Title: Add a half-length pre-warmup, to avoid penalizing later tests.

IPS currently warms up each test immediately before running it. This can potentially skew later results that cause invalidations or deoptimizations along code paths optimized earlier. For example, a later test may cause a site that was previously monomorphic to go polymorphic, causing that test to have degraded perf compared to earlier tests, even though a steady-state system would penalize all equally.

My change inserts a pre-benchmark half-length warmup run of all the benchmarks together without gathering any metrics. This brings us closer to steady-state numbers for the later warmup+benchmark loop.

The logic for the pre-warmup is mostly identical to the in-bench warmup to keep the system impact as similar as possible.