Easier to read console output

The console output was hard to read because it was displayed on one
line as a string as part of a tuple. Split them up so it's readable.

Change-Id: Ieb59909802a587027c4691924e70923c61476591
Clean up tox.ini and make tempest and unit test distinct

This commit creates a separate base env for tempest tests and
tempest's unit tests. However, to keep the same interface the tempest
env is setup by overloading the testenv using substitution from a
common base section.

As part of this internal reorganization in the tox.ini several other
things are cleaned up or removed. Usless or duplicate job definitions
are removed. The unit tests are switched to pretty_tox. Previously
they were run without any output which made it difficult to figure out
exactly what was running. Additionally, this removes the unnecessary
regex filter on the unit test job. The unit tests use a different
discovery path so there is no need to also filter it on the same path
this will allow the use of more specific regexes with tox to filter
tests.

Change-Id: I996d81dec2739bbe907f5fe87a335994c01697b3
Fix finding router for scenario tests

Finding router return tuple resp,body and body only contains router details.
Closes-Bug: 1375859
Change-Id: Ic405d20991561e8773153d829750826a82ed751c
Revert "Skip telemetry_notification_api test due to bug 1336755"

As https://review.openstack.org/#/c/114514/ has been merged, we can
try to check if changing of Ceilometer backend to the MongoDB makes the
situation better with running Ceilometer notification tests. Locally the
timeout issues were not reproduced both on MySQL and MongoDB, so this
change aims to check if the issue was solved by trying experimental runs
of MongoDB devstack Ceilometer testing.

This reverts commit 88004daa7b0fd746a60dd3b17d68ed4427b40e58.

Closes-Bug: #1276157

Change-Id: I41fce7e56a0e9fb111a95a266def786cb0f90c5e
Adds status check for FloatingIP in network scenarios

Verifies FloatingIP changes status correctly on association and dis-association

Scenarios:
test_network_basic_op
test_network_advanced_server_ops

Status is verified after connectivity check and without a waiter in order to
not mask unknown races in Neutron

Changes show() to refresh() to keep consistency

Partially Implements: blueprint fip-op-status (this is a Neutron bp:
https://blueprints.launchpad.net/neutron/+spec/fip-op-status)
Partially Implements: blueprint neutron-advanced-scenarios

Change-Id: Ibd45f3092cebc8f9fd0d16a3a543c2ef72156409
Remove OfficialClient dependency from HACKING.rst

This commit migrates OfficialClientTest to ScenarioTest about
TestVolumeBootPattern in HACKING.rst. We should not have such an example
in the HACKING.rst because we don't use OfficialClientTest class in the
scenario tests anymore.

Partially-implements: bp:tempest-client-scenarios

Change-Id: Id95d011f3d080effc7bfee6232578451f7ba72d9
Add expected response checks of secgroup rules

"add security group rule" API returns additional rule in its response
body but current tests don't check the response.
This patch adds some checks of the response.
In addition, this patch removes status code checks because the codes
are already checked in client side.

Change-Id: Iab4ce492c060b8a2b79a3b3d9a6d30a359f6f312
Related-Bug: #1373832
Orchestration use stack timeout waiting for ping response

The test_server_cfn_init scenario test boots a full Fedora image,
not the cirros image which is presumably used for compute tests
where the default 120s ping timeout may be appropriate.

To ensure we don't encounter any spurious failures due to booting the
much bigger orchestration image (which is Fedora in the gate), use
the stack timeout (default 900s) to override the ping timeout for
this test.

Change-Id: I4eba18333bdaa0ef41122cda85fd3f4d91b2cf5c
Closes-Bug: #1374175
Fix duplicate ref of volume_types_client in vol tests

tempest/api/volume/base.py admin base tests class has duplicate
reference for 'volume_types_client'. Tests use mix of both.

This patch fix this duplicate reference and modify tests accordingly

Closes-Bug: #1379106

Change-Id: I016d364ad792c97d1dce12ee7da225ebe0a690f6
Add Credentials Provider factory

Add a method to select the appropriate credentials provider based
on configuration, and replace all the flow branches based on
allow_tenant_isolation to use the factory.

This also includes handling NotImplemented exceptions on admin creds
for scenarios tests which is a new potential condition by using the
factory.

Partially-implements bp:test-accounts

Change-Id: I280f2b1659c593e14ab9213a161b55bcfc08fa5c
Port  API Tests Enhancements

Add function to update port with security groups
   -Function take list of security groups name
    as parameter
   -Create security group based on the names in
    the parameter
   -Create a port without security group
   -Update the port with the security group
   -Verify that security groups are updated in
     the port  successfully

Add a test to update the port with security group
    -Call the update port with security groups
     function with one name of security group

Add a test to update the port with two security
group
   -Call the update port with security groups
    function with two names of security groups

Change-Id: I490f3c7eb66e74f4185588870f9b2b44b54bd64b
Remove @author tags from copyright statements

We have git to track the authorship of files. It is not necessary to use
the @author tags in the copyright statements.

Change-Id: I67163ffe46540423ca1f5584881d7c2db47fe276
Don't use a specific arg list for stress in tox.ini

Callers should be able to specify the arguments.

Change-Id: I1d60af5e5300ff91ce3777ac46df655442098013
Add a server console-log test with unlimited length

Now server console-log API is important for debugging gate problems
and the API is called with unlimited length for debugging. However,
there is not the API test with unlimited length.
This patch adds the test.

Change-Id: I5a35f086deddfcd54e995cb1156699592d1daea6
Run telemetry in javelin if resources.yaml says so

In previous code telemetry checks were always run (if there
were servers to check). If the environment did not have ceilometer
services this would fail.

check_telemetry now checks that there is a true value on the
'telemetry' key in resources.yaml. No configuration is required for
the telemetry checks, so only a key with a value is required.

Change-Id: I84e0f5938ec808537043dba29a896f991779b04c
Add exceptions.BadRequest to the test case of reserve_fixed_ip

we change the exception of reserve_fixed_ip in fixed_ip api to
HTTPBadRequest, add this exception to tempest, and also we need
to accept HTTPNotFound exception to support juno test cases.

Change-Id: Iafd2fb82ad5d50c2f4b3f0819ac7a6f8aa901264
Make rest_client.wait_for_resource_deletion timeout error more specific

Currently the timeout error message just says 'Failed to delete resource
within the required time' and includes the test class/test case, but
when we're seeing all volume-related tests racing with a delete hang, we
can't fingerprint an elastic-recheck query on the various test
class/test case names, and there aren't errors in the cinder logs.

The best thing we have to fingerprint is the timeout message, but we
need it to be more specific to the type of resource rather than any
resource type that might have timed out during a delete operation.

This is similar to the is_resource_deleted method in the various rest
clients in that the concrete implementations of the base class return
their specific type so we can use that in the timeout message.

Closes-Bug: #1380712

Change-Id: I36b49c59daa95219c4377d1e207ea4a0c8cf8782
Merge "Revert "Skip telemetry_notification_api test due to bug 1336755""
Move status check to client for "messaging" tests

This patch moves the response status code check from
tests to client side.

NOTE- Some API return different success status code on fly so
leaving those status code specific check in tests only.

Partially Implements blueprint: client-checks-success

Change-Id: Ibbab26d6a99bf67eb684cc9c3ce7845c91fa092e
Merge "Add exceptions.BadRequest to the test case of reserve_fixed_ip"