elasticsearch/elasticsearch — pull request #4376 by javanna (2013-12-07 21:21:20+00:00)
Title: Added REST test suites runner

The REST layer can now be tested through tests that are shared between all the elasticsearch official clients.
The tests are based on REST specification that can be found on the elasticsearch-rest-api-spec project and consist of YAML files that describe the operations to be executed and the obtained results that need to be tested.

REST tests can be executed through the ElasticsearchRestTests class, which relies on the rest-spec git submodule that contains the rest spec and tests pulled from the elasticsearch-rest-spec-api project.

The following are the options supported by the REST tests runner:

- tests.rest[true|false|host:port]: determines whether the REST tests need to be run and if so whether to rely on an external cluster (providing host and port) or fire a test cluster (default)
- tests.rest.suite: comma separated paths of the test suites to be run (by default loaded from /rest-spec/test). it is possible to run only a subset of the tests providing a sub-folder or even a single yaml file (the default /rest-spec/test prefix is optional when files are loaded from classpath) e.g. -Dtests.rest.suite=index,get,create/10_with_id
- tests.rest.spec: REST spec path (default /rest-spec/api)
- tests.rest.shuffle: boolean value that determines whether the tests are run in a random order (default) or alphabetically
- tests.iters: runs multiple iterations
- tests.seed: seed to base the random behaviours on
- tests.cluster_seed: seed used to create the test cluster (if enabled)