per test_method logging

Using the FakeLogger/LoggerFixture fixture in order to capture the
per test method logs.

The per test method log capturing visible in the console on error or
failure.

The normal logging is preserved and only the INFO level log is
captured.

Change-Id: Id42f79bd3bcaba9aba57f4c0b8a373f33b8cd6bd
Fix ssh timeout issue

SSH client is using a socket timeout and an execution timeout. For
command execution, if the executed command did not generate output
within the channel timeout, test dies with TimeoutException. This is
misleading, because it wasn't the command that timed out, it just did
not generate output within channel timeout.

fixes bug 1215794

Change-Id: I90b9cb7c9cebcb78655b6c1a46f9fb09ff7deb5e
refactor - _is_timed_out using instance timeout

Each _is_timed_out client specified self.timeout as timeout parameter.
As _is_timed_out is an instance method, timeout parameter could be
retrieved from the object itself.

Change-Id: I56c6d5a21a36cf052d3bde86c454d7be72b22a7c
Use common create_keypair method for autoscaling

The scenario test for autoscaling was implemented at the same time
with cleanup patch of create_keypair methods, and the cleanup patch
could not cover autoscaling test.
This patch covers autoscaling test.

Change-Id: Ibf4e98a9d4616c1dc6d2f08bde3d04a40b910067
uniforms skip messages

this is an attempt to uniform skip messages and ensure multi-line
skips are correctly accounted by https://review.openstack.org/#/c/42424/
I think a future update to hacking/checks.py is also needed

Change-Id: I8ef1ac22a8b137f397847d217467df5572afde43
Adds tests for Heat

Tests and client methods for:
- template-show
- template-validate
- template-url-validate
- stack-list
- stack-show
- resource-list
- resource-show
- resource-metadata
- event-list
- event-show

Testing Neutron resources:
- network
- subnet
- router_interface

Testing server property:
- subnet_id

Change-Id: I47bb0dd653da51c9ff1d2ffe0b02c102cc0098d5
Stress ssh_floating test

Add stress test for floating IP association
 with multiple configuration options.

Change-Id: I184ce2df1ee58043cf9b26c46637c1d8160910e8
Updating HACKING.rst

* Adding a test case independence rule
* assertTrue/assertFalse msg argument
* Encouraging  matcher usage for assertion

Change-Id: I89612a0dd66513f10e8bf53632f6a3d81ccf4129
Add large-ops option to tox

Add new option because we want to keep separate tox jobs for each
jenkins job.

Part of the regression test for bug 1199433.

Change-Id: I4e59d79d3ce3a9621243a81b7da5d9c3abaf1d6f
Rename heat logical_resource_id to resource_name

This attribute has been renamed in the heat API.

Fixes bug: 1216903

Change-Id: I4b3b65564eed08d0e077302e747f5ea1124ff7d6
Add logging to the python-clients

The python-clients are very different in how they are logging,
the cinder and nova clients needs and additional option.

This change enables the http request/response logging to the
 nova and cinder python client.

The nova and cinder python client wants to create unnecessary logger
 to the console, unless you add a handler earlier.

Change-Id: I5bc72e52d16d2380ee36c8586cd1694ac1b4fa3d
Move the network api tests to smoke

Make the existing neutron tests voting.
However what should be smoke test, does not
have clear definition yet, but this change let these tests to vote.

Change-Id: I02612b3591057c4f59d6571e90acda74de802444
Switch run_tests.sh to run in parallel by default

This commit switches run_test.sh to run in parallel by default. Since
the tox jobs were moved to parallel by default run_tests.sh should
have the same behavior.

Change-Id: Id144725bfd22d72adbad62772c7496b843dfece8
emit warning while running flake8 without virtual env

run_tests.sh -N -p
it will call the flake8 installed on your host to detect PEP8, and
the flake8 plugin "OpenStack hacking" may not installed on your
host, so this command may not detect the OpenStack Style Commandment
supplied by hacking(e.g H202).

run_tests.sh -p
it will call the flake8 from virtual env, flake8 plugin "OpenStack
hacking" installed in virtual env will be triggered.

The result from "run_tests.sh -p" should be trusted, and jenkins uses
virtual env to run flake8 too.

When "-N" is enabled, emit warning to stderr to remind user.

Bug #1215405

Change-Id: Ic0fc8d39a87942e175256453c6c4347f6a5cfbba
Create discovery option for stress tests

Adds a decorator @stresstest which automatically sets the type
attr to "stress". It uses the testtools discover functionality to
automatically discover stress tests out of all tempest test.

It is possible to filter stress test with a given attribute type.
For instance to filter out only smoke tests that have a stress test
attribute.

blueprint: stress-tests
Change-Id: I8acf0b608cb500c2679a36a4a00ca4fa14668fad
Fix test_admin_catalog_list

This commit fixes test_admin_catalog_list() so that it tests more
specific details of the admin catalog. It also removes the test's
dependency on each table in the output having a service label. This
dependency was occasionally causing a failure if the test ran at the
same time as the keystone api tests.

Fixes bug 1217159

Change-Id: I4446abb86a3f1f6514fc9de1d00c06f82943b9d9
Merge "Create discovery option for stress tests"
Merge "Rename heat logical_resource_id to resource_name"
Add common "create_server_snapshot" method

There are some "create server-snapshot" methods in scenario tests,
and they will increase in the future by increasing scenario tests.
This patch adds common "create_server_snapshot" method to parent
class and uses it instead of each implementation.

Change-Id: Ib8a222d74828933c48bc6b0e8350cc072f01af4d
Merge "Switch run_tests.sh to run in parallel by default"