grosser/parallel_tests — pull request #288 by mhodgson (2014-01-10 19:08:58+00:00)
Title: Add ability to only run one of the generated groups

Not ready to merge yet, but wanted to start a discussion.

This PR adds the ability to tell parallel_tests to group the tests, but only actually run the specified group. This is particularly useful when running on platforms that offer parallel build capabilities, but the tests are run on separate VMs (like Travis-ci).

The functionality is provided by way of a new CLI options called --only-group with expects an integer (group number). Use it like so:

```bash
parallel_tests spec/ ---n 5 --only-group 1 --group-by found --type rspec
```

This would group the tests into 5 groups, but only actually run the first group.

It currently requires the group-by option to be 'found' so that the grouping is deterministic across multiple separate processes. It would probably be nice to add a group-by option for filesize, but right now that is mixed up with the runlog which might not be the same across different processes (if they are on different systems with different logs).

Here's an example of how this can be really useful on Travis-CI:

```yaml
...
matrix:
- "NUM_GROUPS=5 GROUP=1"
- "NUM_GROUPS=5 GROUP=2"
- "NUM_GROUPS=5 GROUP=3"
- "NUM_GROUPS=5 GROUP=4"
- "NUM_GROUPS=5 GROUP=5"

script:
- bundle exec parallel_tests spec/ -n $NUM_GROUPS --only-group $GROUP --type rspec
```

Finally, if `--only-group` is set, parallel_test will always behave as though the test number is 1 and ENV['TEST_ENV_NUMBER'] will be blank. The thought here is that is you're running all of your tests on separate hosts anyway (like in Travis) you don't need to go through the hassle of setting up multiple databases, etc. Of course this may break use cases where people are using ENV['TEST_ENV_NUMBER'] to manipulate their test suite internally. Not sure if people do this or not.

I think there are probably other options that should be incompatible with this but I'm not sure what they all are. Thoughts? Useful?