angular/angular.js — pull request #10348 by jeffbcross (2014-12-06 00:01:39+00:00)
Title: WIP: add karma-benchpress support for all benchmarks

With this PR, there is a karma test that will run the compiled largetable-bp benchmark in `build/benchmarks` with 1 sample for each variable. The results are just logged to the console with some basic tabbed formatting. Anyone should be able to make this work by the following steps (requires Chrome Canary):

```shell
$ npm install . (may require removing npm-shrinkwrap.json to get latest benchpress & karma-benchpress dependencies)
$ grunt bp_build
$ karma start karma-benchpress.conf.js
```

Karma should start up, and should automatically run each benchmark/variable combination in a new incognito window (with other flags set for memory reporting), and should log the results in the process console.

TODOS for this PR:
* [ ] Update shrinkwrap (having trouble figuring out which version of node I should use to generate this properly. Will probably end up hand-coding the changed dependencies.)
* [ ] Update remaining benchmarks to new Variables and ready() APIs
* [ ] Create tests to run remaining benchmarks with each variable with 25 iterations & 20 samples
* [ ] Create jenkins job to run benchmarks on each SHA
* [ ] Squash commits

TODOS for later:
* Report standard deviation and coefficient of variation back from karma-benchpress. This needs to be fixed in the karma-benchpress framework.
* Push the results to a logging service, along with environment information and other metadata

Whatever CI server this ends up running on (Jenkins, my workstation), I don't think it should be part of the regular angular.js job given how long it would take to run all benchmarks with all variables n times. Ie if the average sample of all steps takes 750ms (made up number, probably not far off), multiply that by number of iterations (25) by number of benchmark variables (33 at present). This would take about 10 minutes for each revision.