Fix resource_setup order in scenarios

As resource_setup() document says, we should check skip condition before
resource_setup. However some functions in scenarios setup resources
before skip checks. We should avoid to setup unnecessary resources as
possible. This commit fixes the order.

Change-Id: Id0529fc80f2b80f028fd078140fe5fd7459e9309
Don't double log API request

If debug logging is enabled the full API call, request and response is
logged, so don't log the request twice.

Change-Id: I45baa04a2daae8fd31c30183ac10464b7affbc21
Fix execute status of cleanup script files

cleanup.py should be executable, cleanup_service.py is a module.

Change-Id: Ibb8e06a0165d9b2f42c1aeec8bf639227a5480d3
Add list servers to large ops test

Add regression test for bug 1282842, which is hit when running nova list while
waiting for instances to move to ACTIVE.

Change-Id: Ic0714834846ad94b213d97e7756ca607e1f4d710
Related-Bug: #1282842
Make duration in serial test approximate total run time

The duration argument is supposed to mean how long the test will take to
finish. Before this change, if running serially it would take n*duration where
n is the number of test cases specified to run.

Change-Id: I9bd525b17e41fae811e3bf48642b728151e65737
Fix and re-enable test_disassociate_not_associated_floating_ip

Fixes test_disassociate_not_associated_floating_ip so it sets the
ec2_client class variable needed by a base class correctly. There
has been some bit-rot since it was disabled.

Note that this test will not pass until the nova fix
https://review.openstack.org/#/c/124320/ has merged.

Change-Id: Ice39358f4e8123334f24056daede68bead59729b
Related-Bug: 1179816
Added neutron cli test case for vpnaas

vpn_list,ikepolicy,ipsecpolicy,ipsec_site

Change-Id: If61eb1506da464b0e173ef63c4dbccdd8f757616
Test to update port with CIDR value

creating a port with allowed address pair
verifying the port creation with allowed address pair

creating a port, updating a port with CIDR value
verifying the port creation with CIDR value

creating a port, updating a port with Multiple ip and Multiple mac
verifying the port updation with Multiple ip and Multiple mac

Change-Id: I22a041df97f9f630b66c48f92f4c0da12db75b0a
Bump version string to coincide with kilo release

The tempest-2 tag has been pushed to mark the start of support for the
kilo release. This commit bumps the version to 3 to show the current
working tag number. Tempest-3 when it is tagged after the kilo release
will mark support for icehouse, juno, kilo, and the 'L' release based
on the current stable maintenance schedule.

Change-Id: If401356d441ef3dc82a5b8069168c4dd71c8b3d8
Merge "Add list servers to large ops test"
Merge "Make duration in serial test approximate total run time"
Remove AltManager class from client

As Change-Id: I280f2b1659c593e14ab9213a161b55bcfc08fa5c has modified
the Manager instantiation for non isolated tenant, AltManager class
is no more needed.

Tests runs for non isolated by instantiating the Manager with 'alt'
credential.

This patch removes the class AltManager from client.py

Closes-Bug: #1329065

Change-Id: I0dcf4050d8524655b017b4b5308fe3070aa2f840
Fix resource leakage in messaging.test_queues

Test messaging.test_queues.test_delete_queue does pop 1 element from
shared list (self.queues which is being used by other tests as well
as in resource cleanup) and then delete request is done.

In case where delete request fails then it cause other tests to fail
and resource leakage as element has been removed from self.queues
even before actual deletion.

Individual test should not be side-effecting mutable class variables
that are used by other tests. So merging delete tests with create tests.

Closes-Bug: #1377031

Change-Id: If24a903160e716a685d7121e0968881542f072fc
Adds test for list/get volume attachments V2 APIs

This patch adds the test for following Nova V2 APIs-
- List volume attachments
- Get volume attachment
Response Schema has been added for the same.

To make consistent in server_client and easy for tests to compare
the response, this Patch also change the returned response body from
server_client's attach_volume().

Note- XML test cases is not implemented as it is deprecated.
New test cases has been skipped for XML.

Change-Id: Iee055ceaad135bba4a57b010d1e08652e2d13598
Create subnet without gateway and explicit IP ver

Now it's impossible to create subnet without gateway in network
tests. This patch allows you to set gateway explicitly to None.
Backward compatibility is supported: by default it creates
subnet with default gateway as before. Also it adds possibility
to create subnet with specific IP version when you need to create
two subnets in one tenant of different IP version ("dual-stack").
Fixed attributes test for new requirements and added 2 anothers.

Change-Id: I7aca5e07be436f20cba90339785b46182d97fead
Fix AttributeError when test_function_help fails

This commit fixes a missed usage of the CommandFailed exception during
the migration moving it from tempest.cli to tempest.exceptions in change
Ibf5f1cbbb847d32976b54c4484acfc3c0e3b4f48. The outdated module
path could cause an AttributeError to be raised because the exception no
longer existing in tempest.cli.

Change-Id: I502eaef3486d903931dab43069cec1ae7ee570f1
Migrate cli test framework to tempest-lib

This commit switches the cli tests to use tempest-lib. The code which
was migrated to tempest-lib is removed and all the existing test cases
are converted to use the framework provided by the library. Future
improvements to the cli testing framework should be made in
tempest-lib after this commit.

Partially implements bp tempest-library
Change-Id: If2c9b3895a30905bc3d62b8daef783878d78399a
Switch to using subunit-trace from tempest-lib

This commit removes the tempest copy of subunit-trace and uses the
entry-point from tempest-lib instead.

Partially implements bp tempest-library

Change-Id: Ic1aeddc2d0b217c41b6b9914ac3f269b80c4ec8e
Add config option for XML support in Networking

The first step to removing XML in Kilo is making XML testing optional.
Stable test configs will need to be updated to enable this feature to prevent
regressions in stable releases.

Change-Id: I30b9565e01c623814a9c503c4b1c26642afed878
Merge "Don't double log API request"