Cleanup and make HACKING.rst DRYer

Reference the OpenStack hacking guide in HACKING.rst and remove
duplicate entries.  Add section for tempest specific rules.

Change-Id: Idea236dfa7430fb49892c0d677f6f18175c1249d
Merge "Cleanup and make HACKING.rst DRYer"
Fix tox job for stress tests.

This commit fixes the behavior of the tox stress job. To do this a
new json tests file was added for the tox job. This json file will
contain a list of all the actions that will get run as part of the
tox job. (and by extension the periodic jenkins stress test job) The
tox job is then updated to run each of the tests listed in the json
file for an hour each.

Part of: blueprint stress-tests

Change-Id: I5eb0652799601631e34cc2bbef836624d71bb3e7
Add an option to run_stress.py to run tests serially.

This commit adds a new option to run_stress.py to run the stress
tests serially. If more there is more than one JSON test in the
file passed into run_stress.py it would by default run the tests
in parallel. However, there was no way to specify running the tests
serially. This commit fixes this by adding a -s/--serial option
which will run each test to completion before launching the next
one.

Part of: blueprint stress-tests

Change-Id: I7188929485e29a761327ab24e2bb1213fbd052a9
Rework stress to be more UnitTest like

This patch reworks the stress test framework and
the scripts to be a bit more flexible and to be
like how unit tests are written.  Instead of
writing a module that contains functions, we now
write an object that can implement a setUp and tearDown
method.  By default the object's 'run' method will be
called by the driver.

The setUp method will be called prior to calling run
and tearDown will be called after run is stopped.

Also added debug logging to the cleanup mechanism.
Also added removing snapshots prior to removing
volumes during cleanup

DocImpact

Implements:  blueprint stress-tests

Change-Id: I39b8a1d38f70ddda4867126b58bb7053b45654d6
Merge "Rework stress to be more UnitTest like"
Merge "Fix tox job for stress tests."
Merge "Add an option to run_stress.py to run tests serially."
Switch to using testr as the test runner for everything non-gating.

This commit switches the test runner in tempest to testr from nose for all
jobs that aren't gating. This will allow the usage of parallel testing with
the use of a group_regex in testr. Group_regex will ensure that the classes
get scheduled together and the run times are tracked together. Than the
tools/run_test_classes.py script will filter the test_ids and pass only the
classes to subunit.run to ensure we are only running setupClass once.

This commit also adds a new option to run_tests.sh. -t/--with-testr can be
used to optionally run tempest in parallel with testr. Once running with
testr gets stable enough this will become the default for run_tests.

It also adds a testr-full tox job so we can have a tracking non-voting job
that runs the same tests as the gate but with testr.

Implements: blueprint speed-up-tempest

Change-Id: Iedc9bd92b8f8471c60c614c7d7c05046d7b32743
Fix stress-tox-job.json action file.

Change I39b8a1d38f70ddda4867126b58bb7053b45654d6 changed how the
actions are called, but stress-tox-job.json was merged at the same
time. This commit updates the json definition to use the new paths
so that the tox job will work again.

Part of: blueprint stress-tests

Change-Id: Idadf08e0cc3f01650e72ca078a223781a0f1669a
Merge "Switch to using testr as the test runner for everything non-gating."
Remove duplicate flaky test.

Remove two flaky tests:
- test_create_image_when_server_is_terminating
- test_create_image_when_server_is_rebooting

Fixes: bug #1191590
Change-Id: I7cd1d0a9b3b87ce15cbfd4b28805e43a691aedd3
run_stress.py -h doesn't work without a connection

run_stress.py tries to connect to openstack even though it's just a
--help / -h call. Just moved the import to the corresponding function.

Implements: blueprint stress-tests

Change-Id: Id0f9784a64f3bf53ea91a67452a78d17389d7f51
Merge "run_stress.py -h doesn't work without a connection"
Use subunit colorizer from nova for run_tests.sh

This commit copies the colorizer.py script over from nova and enables
it for testr with run_tests.sh. This enables real time output from
testr about which tests ran and the result with colors.

Change-Id: Idd1b820d103bf4d71f2645e98e86e5fadbd6cb0c
Add unittest like output for testr-full in tox

This commit adds real time output similar to that when running with
nose so that it's possible to watch the status of the job when using
testr with tempest.

Change-Id: I7a65be714627692ad80b3640af0ee617fd4697a9
Document E125 as a won't fix.

As per the discussion in https://review.openstack.org/#/c/36788/ the
tempest team does not want to use E125 until
https://github.com/jcrocholl/pep8/issues/126 is  resolved.

Change-Id: I124e6f14db81d030a470ef3e73a33b15e9dea9ef
Remove unneeded class filter from .testr.conf

This commit removes the unnecessary class filter run_test_classes.py
and reverts the .testr.conf to the standard format. The class filter
is not needed when using test grouping.

Change-Id: I460c48121c26d07eb29f355c07a2059f5e6b24b0
Skip test that is not relevant to Neutron

Neutron does not support DNS domain feature,
thus this test must be skipped in Neutron environment.
Fixes: bug #1202991

Change-Id: I78f4608cce488cd515d72df2178d7a5b45bf2b85
Merge "Remove duplicate flaky test."