translate/translate — pull request #110 by dwaynebailey (2014-03-10 11:23:05+00:00)
Title: cli tests

This is a ``bash`` based framework to allow easy testing of command line tools. This is landing as a first review as there are some changes that I would like to make but would like some feedback on what I do have before continuing. Or even a nod that what I have is good enough to land and we can fix the rest as we implement tests.

How the tests work is documented in the ``docs/developers/testing.rst`` in the first commit.

### Some changes on my mind
1. The first change, but it requires quite a bit of work, is to move the test data files in $data/$testname/ directory so that it is easy then to manually run the test in its data directory without the clutter and confusion of filenames. The $data directory is already crowded and we don't have even a fraction of the test scope that will probably be required.

2. I have one issue that has just emerged when I want to check that a result isn't an exact match but contains, startswith or endswith some data. At the moment we just diff the files, empty diff means success, diff output means failure.

3. Introduce some sort of command to check if the TTK command returned a failure and treat that as a pass. I.e. checking EXIT values are correct not just checking the diff value.