cloudkick/whiskey — pull request #61 by sam-falvo (2013-06-12 01:37:50+00:00)
Title: Propegates exceptions from child to parent process.

This commit addresses https://github.com/cloudkick/whiskey/issues/57 .

PROBLEM

Too many times when working on Whiskey, I might make a seemingly
innocuous change to the program which causes timeouts to occur. The
cause of the timeouts remain a total mystery. The problem is the result
of a cascade of failures:

1) Something I changed causes an exception.
2) stdout/stderr is written to, and then the child Node process dies.
3) Meanwhile, the parent is unaware of this happening, and so sits on
its socket until it times out,
4) After the timeout, the parent process has no idea what killed the
child process, and so the only indication of a problem is that it's
listed in [TIMEOUT] status.

SOLUTION

Capture all exceptions that you can at the top-most level of the child
process, and marshal that data through the IPC channel back to the
parent process.

Then, alter the parent process to die immediately if an exception is
discovered. Fail fast, fail often.

OTHER BENEFITS

Knowing what's causing some of the timeouts will, I *think*, be
beneficial from a Buildbot perspective as well. No gaurantees, but it
*could* be useful in uncovering some of the causes for intermittent
buildbot failures we've seen in the past.

HOW IT WORKS

It works by capturing all exceptions that leak to the main
run_test_file.js script, and forwards them to the _reportException
method on the current TestFile. This stringifies the exception into a
record that is sent, via the usual Whiskey socket, to the parent Whiskey
runner process.

Once there, the lib/util.js parseResultLine method will decode this new
format and, if discovered to be an exception, will encapsulate it in a
triple, which can be uniquely identified as follows:

[true, filename, null] = File end without coverage data
[false, filenameIThink, coverageData] = File end with coverage data.
[null, testName, testResults] = Test invokation results
[string, string, null] = Exception report. (New as of this patch)

This is interpreted by the lib/run.js method
TestRunner._handleConnection(), which encapsulates the provided results
into a new exception to be handled by the parent process immediately.

The exception that is thrown is not an exact duplicate, but rather just
a simple object with a message field, formatted with text suitable for
human-readable display, as that appears to be Whiskey's default action
at this stage. I'm not sure where the code resides that handles the
exception thrown in run.js.