Fix resource_setup order in scenarios

As resource_setup() document says, we should check skip condition before
resource_setup. However some functions in scenarios setup resources
before skip checks. We should avoid to setup unnecessary resources as
possible. This commit fixes the order.

Change-Id: Id0529fc80f2b80f028fd078140fe5fd7459e9309
Added neutron cli test case for vpnaas

vpn_list,ikepolicy,ipsecpolicy,ipsec_site

Change-Id: If61eb1506da464b0e173ef63c4dbccdd8f757616
Test to update port with CIDR value

creating a port with allowed address pair
verifying the port creation with allowed address pair

creating a port, updating a port with CIDR value
verifying the port creation with CIDR value

creating a port, updating a port with Multiple ip and Multiple mac
verifying the port updation with Multiple ip and Multiple mac

Change-Id: I22a041df97f9f630b66c48f92f4c0da12db75b0a
Fix unit tests to detect import failures

This commit adds onto the existing unit test which simply runs
'testr list-tests' to run discovery on tempest. This is done to ensure
that from a bare tempest repo with the requirements all present we can
actually import and list all the tests. However, there wasn't a check
of the return code. Historically this has been an issue with discovery
failing so adding a check to enforce it moving forward should prevent
it from slipping back in. Part of this is to also fix the test so that
it runs discovery on the correct test suite.

In order to make the unit test pass a config dependency in the
scenario test test_basic_server_ops during the testscenarios call in
load_tests had to be removed. Instead if there is a configuration
error raised while running load tests on this module instead of
failing it will just return the base test class. This is fine
because the test itself will fail on the same configuration
dependencies that load_tests is, however it won't happen silently
during discovery anymore.

Change-Id: Ie74c020de7c9c27adc4ff68dddbae5d4481d1224
Prevent external router IPs from being sent back

In anticipation of change Iba7c606eea48181fc10e9d0d5dc667e6f48f37de
being merged or at least a minimum a fix for bug #1255142, the
router response will have external IP address information available
that is not mutable by a regular tenant. This patch removes that
information from the response before replaying it back for a router
update.

Change-Id: Id1da9246bca2a2eee69f751856ffa7052eb18ef9
Increase default maximum safe_body function

There is this specific test test_absolute_limits_negative which's xml
version is bigger than 2048 characters and was truncated on tempest.log
that is fixed increasing that maximum even though I'm not sure how long
xml tests will live here.
Closes-Bug: #1383996

Change-Id: I771921e3c14333afc791e21395c0edf9feba5ee0
Remove "Disabled Reason" from test_cinder_service_list

The CLI test test_cinder_service_list checks the output of
"cinder service-list" for a set of column titles. One of the titles
it expects to find is "Disabled Reason". However, this title is not
guaranteed to be in the output; cinderclient only appends it if
the value shows up in the response from the server.

Change-Id: I060bad255ff17b132737a8878c26e7d5cf9dce82
Closes-Bug: #1392662
Adds "list external networks" test to regular user

For non-admin user verifies that Public networks are visible but their subnets
are not.

Change-Id: I327bed82a9fe2bbd2d574900dbfd182d27892dc8
Revert "Skip test_list_server_addresses if neutron"

This reverts commit 3c7f98f0b78444e3cfc411a331d40ceb07629c21.

Since blueprint admin-event-callback-api in Icehouse we shouldn't
have this issue so enable the test again.

Change-Id: I21aa94068129b856e6a2965ac749c414b8b0cc9e
Closes-Bug: #1210483
Decouple resources and preconditions in secgroup scenario

Handle secondary tenant checks based on CONF file only instead of using clients

Skip only cross-tenant test instead of whole class if secondary tenant is not
available

Change-Id: I9863ff840b9c5b18fa46826aab3fc107454dcc24
Update docs for regenerating sample conf file

Change-Id: I0999b75f43248b7e3e826845aaabff239aeab013
quota testcases use tenant_name should use tenant_id

Test case related to quota usage (test_show_quota_usage,
test_cinder_quota_defaults and test_cinder_quota_show)
uses tenant name, ideally it should use tenant id as quota
requires UUID of tenant and not tenant name.

Cinder quota-show ideally requires tenant_id to show quota.
As there is bug in cinder (BUG: 1307475) , that if we give
non-existent tenant_id still cinder show the quota and
returns 200 (OK). Hence, these testcases should use
tenant_id and not tenant_name.

Change-Id: Icab25afd482b34290bc74be2ba420f6b016dde18
Closes-Bug: #1321617
Creating individual resources for vpnaas test case

As per design there is only one vpn-service object per router.
Tempest test cases were failing due to creation of multiple
vpn service object for the same single router by raising an
router in use execption.so creating individual resources for
each test cases.

Closes-Bug: 1376104

Change-Id: Id327488c039374160bbfaa9705b869a38aa0583c
Fixes grammatical errors in HACKING.rst

There are a few lines in HACKING.rst that were incorrect
grammatically. This fix changes those.

Change-Id: I8173dae3cf3a3e78a504b4edd4a7c1ff5b7fb9a4
Closes-Bug: #1389603
Fix empty string as id in create flavor request

test_get_flavor_details_for_deleted_flavor specify "" as the
id value for creating the flavor.

Nova V2 API validation is loose and the API accepts this
value. However, V2.1 API will not allow empty string as id.

This patch fix this by passing 'None' as id in this test.

Change-Id: I86419368a4c85aaf039278999d3d8c3c1e224c74
Make delete_container use expected_success

This is the new standard for tests that use the api client.

See also: I07316db1f51d0562919ed2a50f002011944bfdb1

Change-Id: I309084ab0d13fa3bf3a84dcbe4ddee5b24284b66
Use addCleanup when running swift scenario tests

Helper methods that create containers or objects are now responsible
for providing insurance that the container and object are removed.
delete_wrapper is used so that test cases may choose to delete objects
themselves and the resulting failure to cleanup in addCleanup will
not be an error.

Because deletion is a reasonable "basic op" scenario, the deletion
tests in test_swift_basic_ops are updated so they explicitly
validate deletion while also using addCleanup as a safety measure in
the event the test has unexpected failures.

Change-Id: I70b7172e9485cbd1eb881db3b2add3860da0935a
Stop resource leakage for negative tests

get_client_manager should only create a new isolated_creds class if
there isn't already a isolated_creads instance bound to the test.

Change-Id: I44fd05d3c1dc713ec7bd176af689b008fbb5d7eb
Fixes-bug: #1371706
Merge "Stop resource leakage for negative tests"
Support for IPv6 tests for API

Added IPv6 everywhere where it's relevant in network API tests.
Partially implements: blueprint ipv6-api-testing-parity

Change-Id: If2baaa3f2433b88ea4a3ed6dbee61ad6ee9ad3b0