plioi/fixie — pull request #11 by joshuaflanagan (2013-09-19 02:46:51+00:00)
Title: Repeatable shuffled test runs

Running tests in a randomized order is a good way to expose inadvertent dependencies between tests. However, when a test-order related issue is suspected, it is important to be able to repeat that exact test run that had a failure.

When using the `ShuffleCases` instruction in your convention, you will almost always want to know the seed that was used to initialize the randomizer. You will also want to be able to re-run a test-suite with a given seed, when a failure is found.

I figured at the very least, a sample should be provided to demonstrate this approach. However, I think it makes sense to add some built-in support for this additional functionality, since it will likely be used by anyone that cares about the `ShuffleCases` feature. At this point, I'm not as familiar with the codebase to know how communication with the user (which seed was used) should be handled.

> Note: to really demonstrate this sample, you need a `Listener` that shows the name of passing tests, so that you can see the order is repeatable. Changing `ConsoleListener` seemed out of scope for this PR, so I did not include that change.