cylc/cylc — pull request #672 by arjclark (2013-09-04 14:40:58+00:00)
Title: test-battery: more versatile framework

This ports the existing test battery to a more versatile framework which allows tests beyond just running suites to be carried out, laying the groundwork for forthcoming plans for expanding coverage of the test-battery (which I'll be doing in chunks as a follow-up to this).

The existing tests have been ported to the new framework and are now validated prior to running the reference tests. The `validation` set of tasks, whose main purpose was to pass validation rather than being run, are validated only. I've gone for validating all the suites here as a means to maximise the different combinations of things we might want to validate and ensures all our tests are compliant with how cylc expects suites to be constructed. I believe this will prove useful when we hit the ISO8601 time changes.

Also added as part of this is a test for `cylc get-config` (courtesy of @benfitzpatrick) which tests `cylc get-config` and serves as a good demonstration of the things we can now test and types of tests that can be carried out (and how).

Each set of tests for a particular thing has its own `.t` file in which the tests are defined e.g. `/cylc/tests/shutdown/01-task.t` contains tests relating to suite shutdown based on tasks. Reference test suites can be generated as before but will now need a `.t` file with instructions on what to do with them if they are to be included in the test battery.