JuliaLang/julia — pull request #3813 by staticfloat (2013-07-24 07:04:16+00:00)
Title: First draft of codespeedization of test/perf

Summary:
=======
* Change webhook notification URLs to julia.mit.edu (Linux) and criid.ee.washington.edu (OSX)
* Add `codespeed` target to test/perf/Makefile, which will run perfsuite and upload result to hardcoded codespeed installation location
* Note that `make codespeed` will attempt to `Pkg.add()` the packages `JSON` and `Curl` if not already installed.
* Exclude `sort` subtests for now, see https://github.com/JuliaLang/julia/commit/453b4efc30bfdabf8a1db33a496ef5972afc9812
* Annotate all @time macro calls with an extra "Description" field, used to populate the codespeed database for the first time
* Again, exclude the `sort` tests for now, as this integration with codespeed will have to wait. For now, empty descriptions
* Until extra build-time information is provided, expect environment variables such as JULIA_FLAVOR, JULIA_BRANCH, and JULIA_COMMIT_DATE on the command line when `make codespeed` is invoked.

This might be my biggest pull request yet, but I seem to have finally gotten things to the point where they're uploading to codespeed regularly. I know it doesn't look like it due to the lack of history, but that's because I recently purged the memory of the codespeed installation.

Comments and critiques are welcome! Once this gets merged, `master` will start to automagically get tracked; it fails right now due to the lack of the `codespeed` target in `test/perf/Makefile`. I'm not sure if we ever want to backport this to `release-0.1`, perhaps this should be a 0.2+ thing. :)

Next on my TODO list:
==================
* Build scaffolding around C code to create a "baseline" in codespeed to compare against
* Enjoy the pretty graphs, and make sure the build process is all toughened up (Only time will tell, now).
* Something I'm surely forgetting, but can't seem to put my finger on right now.....