cloudkick/whiskey — pull request #50 by sam-falvo (2013-04-15 22:41:40+00:00)
Title: Initial changes to explore istanbul integration.  DO NOT MERGE.

DO NOT MERGE.

This is work-in-progress, and things are subject to change. All changes
will be accumulated onto this branch, subject to review. If/when it's
decided that everything looks good, I'll "merge --squash" into the
actual feature branch for final review by the team, and if _that_ is
approved, I'll merge the commit.

We use mkdirp package to create directories containing Istanbul coverage
reports and JSON files for each integration test invoked. Istanbul can then be
invoked manually over these directories to aggregate the results. E.g.,

istanbul report --dir /tmp/coverage
python -m SimpleHTTPServer 12345

or something very much like that, then view the report at
http://localhost:12345.

In order to generate the coverage, we need to invoke Istanbul with the
'cover' subcommand instead of node. For example, instead of running
"node foo ", you'd need to run "istanbul cover
foo -- ". Additionally, we need to supply a number of
istanbul-specific command-line parameters, as distinct from the
parameters each integration test receives normally. The new args array
construction code reflects this change. I'm still not quite sure how
best to implement a "switch" to turn on or off Istanbul at this point,
as I'm not sure if I can overload the existing coverage options for this
purpose. I believe at least one coverage option will become vestigial
after we embrace Istanbul, however, as Istanbul follows
convention-over-configuration.

Problems with the current implementation include:

1. I need to parameterize where to find Istanbul in case it's installed in a
non-standard location.

2. I need to parameterize where the output directories for each of the
integration tests is placed.

3. You still need to run "istanbul report" manually to aggregate the
results. It does support a number of different output formats, so before
I work on this, I need to know what the desired output format actually _is_.

Once I get feedback on these changes, I'll work on finding how to patch things
up with the current set of coverage-related options. My goal is to minimize
changes to Whiskey's current CLI, but am not afraid to add new things (or even
to break them) if necessary. Feedback welcome!