geemus/shindo — pull request #14 by maxlinc (2013-11-05 03:28:08+00:00)
Title: support errors (don't fail fast)

@krames brought up that shindo aborts when it comes across the first error it doesn't know how to handle (an "error occurred outside of a test"). This is that was behavior introduced by #13 instead of ignoring errors.

The current fail-fast behavior may be the safest or most desirable approach at times, but sometimes it's better to continue on with as many tests as possible and produce a full report. This PR tries to make that possible.

I'd prefer it with a fail-fast option to choose between aborting immediately or continuing, but I don't see an existing way to pass user preferences to shindo.

Also note that an error will likely cancel prevent all remaining tests in the same scope from being run. You can see this difference in behavior in the error vs contained-error tests in tests/data/bin_tests.rb