Replace non-documented ThreadPool with multiprocessing.Pool

Replace ThreadPool in rally/benchmark/runners/periodic.py

Change-Id: I585f60ba8f8879a585cde5bc377aea489fe0b5ba
Add benchmark scenarios for ceilometer alarms api

Change-Id: I332fb918a413869e77f9c74c966272d4a7b778a5
blueprint: benchmark-scenarios-for-celiometer
Fix check for tempest tests name

All tests except `tempest.api.*` are invalid.

Change-Id: Ic8cbc64ce43fec982cc7d60c1301a4b1fece7632
Closes-bug: #1316986
Merge "Fix check for tempest tests name"
Rename number validator parameter none_ok

Rename none_ok to nullable, clearer, more concise.

Change-Id: Ic918d22d00e91c60977b3bd9be17bc46c1583ea0
Correction of tempest verification

1) Revert direct usage of subunit

Test repository should be used as launcher for tempest tests, since direct
usage of `subunit` broke execution of `compute` set as well as `full` set.
Tests discovery and execution can't be split at the moment.

2) Cleanup of tempest folder afterfaulty installation

Also, execution of tempest sets cannot be correct without confidence in
correct installation of tempest repository. Cleanup of tempest folder after
faulty installation should be added to fix check of tempest installation.

3) Length of database field for verification results

MutableJSONEncodedDict limited by 64k. This length is not sufficient for
storage results of those verifications, which have a huge number of failures
and tracebacks.

4) Incorrect usage of mock in tests

Class `mock.MagicMock` doesn't have method `assert_called_once`.
Instead of this one, method `assert_called_once_with` should be used.
Based on the fact that this is critical error in testing, this patch
affects not only tempest verification.

Change-Id: I68660e38f2cae3f60a01e99cdbbb57bdb8a9bb04
Closes-Bug: #1313744
Closes-Bug: #1313742
Closes-Bug: #1305991
Merge "Correction of tempest verification"
Skip non yaml/json files when validating samples

This prevents editor temp files from causing test failures.

Change-Id: I7728c0cd79141424df7ec544d73af9a37c1a147f
Closes-Bug: #1314369
Merge "Replace non-documented ThreadPool with multiprocessing.Pool"
add possibility of using public_urls for benchmarking

Add 'use_public_urls' option to the deployment config
This option will be used later for switching access method to the cloud.
Because in some use-cases cloud may be accessible only by public endpoints.

Add 'admin_port' option to the deployment config
This option defines administrative keystone port on the public network.

Closes-Bug: #1306448

Change-Id: Ifb838c5fe9eab5e81353d697aa747bd8adf77572
debug level logs should not be translated

According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.

Change-Id: I558e855175d382cf3ac739f91d66b728b5ac2a34
Add benchmark for tempest. Part 2

Several new scenarios added in tempest benchmark:

* scenario for launching all tests

* scenario for launching all tests from given set(special set_name
validator was added for this scenario)

* scenario for launching all tests from given list of test names
(Also, test coverage for validator `tempest_tests_exists` was increased
for situations like in this scenario)

* scenario for launching all tests which match given regex expression

bp benchmark-scenarios-based-on-tempest

Change-Id: I3e715cb360dec3d5d8683a9001c4f2221b49d1ac
Merge "add possibility of using public_urls for benchmarking"
Add Neutron benchmarks scenario for creating and listing subnets

This patch contains:
    - scenario NeutronNetworks.create_and_list_subnets is added
    - method base.Scenario._generate_random_name is added
      to provide a unified generation of random names.
    - a piece of refactoring for scenario
      NeutronNetworks.create_and_list_networks

blueprint benchmark-scenarios-for-neutron

Change-Id: I4ba67eb3babcf48e1842904c9e39b22a01351a8b
Merge "Add benchmark for tempest. Part 2"
Merge "debug level logs should not be translated"
Invalid result for unexisting task in rally task results

Closes-Bug: #1303286

Change-Id: I4da9bff10c74633c47b3181c7e14cd4568cfbee9
Make cleanup methods lazy

Instantiate oscliens just before calling cleanup method

Change-Id: Icbca58952f575bcbe991cf94e7ef017fb45ea893
Closes-Bug: #1318988
Fix typo, rename libxslt1 to libxslt

* Name of package differs on rhel and debian systems
* On rhel based systems name is libxslt
* On debian based systems name is libxslt1

Change-Id: I49d1c17681642bcaafabdd8535b2fe7ed96e2303
Merge "Add Neutron benchmarks scenario for creating and listing subnets"