Removes test_force_delete_server_invalid_state negative test

Commit I06a476701ccb47d8f7fd6e43c394ccb3fac2b3aa allows the force
deleting of running instances as well as those in soft-delete state
so this negative test is no longer appropriate.

Related-Bug: #1351001
Change-Id: Ia6d09fea753196a8bc9f33ac8acaae49f1d306e7
Allow new quota types

On Nova commit Ib281e43eabfbd176454bde7f0622d46fb04fcb79, we are trying
a new quota type for server groups.
However, current Tempest denies unexpected quota types on its own tests,
and we cannot add the new quota without Tempest changes.
This patch changes quota tests for checking mandatory quota existence
and allowing new quota types.

Change-Id: I4409a7240dc0a3fa1e11c64c5f5a56ca656b70a4
Fix errors on import with service tags

When skips were added to service tags the CONF check was being
evaluated at import. This will cause an exception when there is
no config file. This commit fixes the issue by moving the config
eval inside the decorator wrapper so that it is only done at run
time.

Change-Id: I4127e2b837e103a8e0c24b324458bee97a2326ec
Remove test start output in pretty tracer

Sean does not want and its currently broken.

Change-Id: I7c3cf17bfd8d273823774cd162ae011161c36d3d
Don't run lbaas scenario test as smoke test

Load Balancer as a service  is not part of neutron's primary
functionality, it is one of the advanced services that is not fully
baked yet. As part of the effort to only test advanced services against
neutron and not against all projects, drop this test from smoke.

Related patch: Ib315d30ff3c4463203f7ebe00e159a9680cf1616

Change-Id: I43239e9dc3e99e382ea12fb94f940f9029f4e488
Allow FWaaS API test to accept more than one end state

To support distributed routers, it is useful to have this API tempest test
to accept a firewall resource to be either in ACTIVE or CREATED state. After
all, this is not a scenario test, so whether the resoure is actually wired
should be irrelevant.

Closes-bug: #1360351

Change-Id: I2367c2877ca2ebabc08095b33a85659de2b882bd
Fix tox coverage job

To run the tempest unit tests with coverage enabled the coverage
module needs to be installed. This requirement is properly listed in
test-requirements.txt, however the coverage job definition wasn't
installing coverage. This commit fixes this oversight by adding the
test-requirements file as a dependency for the job.

Change-Id: Ic443c6c4a3166e6b198cbe2af0a77ff2f0a38e83
Improve the selectability of Swift tests

Before this commit, Tempest tests of Swift were selectable only for WSGI
middleware on proxy servers. With this commit, other following Swift
functions are also selectable with Tempest's config file.

Following Swift functions are selectable now:
 - Old-style Container Sync: mirroring objects in the container to another
   container
 - Object Versioning: versioning all objects in the container
 - Discoverability: providing details about the Swift installation

Implements blueprint more-selectable-swift-tests

Change-Id: Idd4afb675c70eea6ba4f8ce59c635f32f040c61e
Refactor ways of creating anonymous test user

Some Swift tests require random user creation to test ACL features or
prepare special data by reseller admin's requests, but that occurs
code duplication. Especially, assigning role id code was too lengthy.

This commit aggregates common codes and remove unused clean-up statements.

Change-Id: I4b965fceeec7c9cd1857375053e148d85f5f5bca
Heat SwiftResources : only count containers created by Tempest

Some containers in the demo tenant could be there before the Tempest
run so we should exclude them when we count containers created by
the stack.

Change-Id: I20ae92e8cc2b61197774963d049fd8231dafdb56
Closes-Bug: 1360832
javelin: implement resource destruction

Checking and creating already exist.
This patch aims to bring resources destruction support.

Change-Id: I9d11db79e10f53a52692ef286323bdb652c42bff
Signed-off-by: Emilien Macchi <emilien.macchi@enovance.com>
Use safe_setup instead try block in setUpClass

To avoid resource leakage in case of exception in setUpClass,
some tests have try catch block in setUpClass and call tearDownClass
explicitly.
safe_setup decorator is for same purpose which should be
used instead.

Change-Id: I59372ec96dcec0058d30e503b22540ce09c43d43
Cleanup neutron default security groups on tenant deletion

This fix cleanups tenant default security groups that are
automatically created and must be deleted on tenant deletion

Change-Id: Ifcf5ec0391799110f158d113f1b6db8a55a31da9
Closes-Bug: #1283083
Fix response body format of orchestration_client to dict

'create_stack' function in orchestration_client return raw json strings.
However, in most of cases, raw json strings are not expected because it's
too hard to use. So we should do 'json.loads(body)' for the response body.
And the other apis such as {update,delete,suspend,resume} do not return a
response body in the API spec docs[1]. So we do not need to add
'json.load(body)' to the functions for now.

[1] http://developer.openstack.org/api-ref-orchestration-v1.html

Change-Id: Ic7d37d70aa0373d9bbf3ad302d1a521f5adaafe9
Adds v2 keystone service test to service v3

This submission adds missing v3 tests in test_services.py
when compared with v2 test_services.py. Also added corresponding
support functions to service_client.py.

Change-Id: I73d44af9851cbfb33b490a313b1c2808c291840a
Fix IPv6 default masks for SLAAC

IETF doesn't recommend use of IPv6 networks
with masks not "/64". Recommended subnetting is
3-1-4 - 48 for provider, 16 for subnetting and
64 for network. For example SLAAC will not
work if you use subnets different from /64.
Test with port update will not work until
bug 1364166 is fixed, added attribute to it.

Change-Id: I260c820dbde1f98b569e564c7b5e7e558e431c0d
Allow out of quota failure status code to be 413 or 403

Due to HTTP response code changes for Bug 1298131, tempest skip
some cases to have jenkins pass. As nova change is merged now,
remove these skip tags. Also need to allow both HTTP413 and
HTTP403 because branchless Tempest needs to support both master and
icehouse/stable branches.

Change-Id: Iee392859a9fd1eec767cbf6292290d5b3ad7dcca
Related-Bug: #1298131
Fix network/secgroup scenario tests for ironic

Because the ironic nodes in devstack are all on a shared network
so they can communicate with the tftp server, all tenants must use
this same network in the scenario tests. This fixes up the networking
and security group tests to take this into account.

Code from the setUp() methods has been moved to a new, separately
called method, to speed up execution time when tests are skipped.
Since setUp() is executed even if a test is skipped, we can avoid
the time it takes to create an unneeded server by moving this to
a new method.

Co-Authored-By: Adam Gandelman <adamg@ubuntu.com>
Partial-Bug: #1361904

Change-Id: If711d69a79f19031dc1d8c529f1219680d22d9bb
Fix load balancer scenario if no tenant network

The load balancer scenario is currently dependent on isolated
tenant networking.  This updates it to allow it to pass in environments
where tenant networking is not present (ie, a shared cross-tenant network).
If a network cannot be resolved by tenant, it will be resolved by the
configured private network name as a fallback and its associated subnet
will be used.

Change-Id: Idbac7c10669a0e2db518223cdd762313aa752876
(cherry picked from commit 3e9e99fb7d7c43b5a6340919f4f1f611f387cbbb)
Adds the clients and tests for CINDER QoS V1 & V2 APIs

This patch adds the JSON clients and tests for following
CINDER QoS-specs V1 & V2 APIs :

1. qos-create
2. qos-delete
3. qos-list
4. qos-get
5. qos-key
6. qos-associate
7. qos-get-association
8. qos-disassociate
9. qos-disassociate-all

Partially Implements blueprint: client-checks-success
Partially Implements blueprint: cinder-v2-api-tests

Co-authored-by: Abhijeet.Jain <abhijeet.jain@nectechnologies.in>

Change-Id: I7fc424b57e4e8decf61e991b2988e1cead5dda9b