Fix resource_setup order in scenarios

As resource_setup() document says, we should check skip condition before
resource_setup. However some functions in scenarios setup resources
before skip checks. We should avoid to setup unnecessary resources as
possible. This commit fixes the order.

Change-Id: Id0529fc80f2b80f028fd078140fe5fd7459e9309
Added neutron cli test case for vpnaas

vpn_list,ikepolicy,ipsecpolicy,ipsec_site

Change-Id: If61eb1506da464b0e173ef63c4dbccdd8f757616
Test to update port with CIDR value

creating a port with allowed address pair
verifying the port creation with allowed address pair

creating a port, updating a port with CIDR value
verifying the port creation with CIDR value

creating a port, updating a port with Multiple ip and Multiple mac
verifying the port updation with Multiple ip and Multiple mac

Change-Id: I22a041df97f9f630b66c48f92f4c0da12db75b0a
Adds test for list/get volume attachments V2 APIs

This patch adds the test for following Nova V2 APIs-
- List volume attachments
- Get volume attachment
Response Schema has been added for the same.

To make consistent in server_client and easy for tests to compare
the response, this Patch also change the returned response body from
server_client's attach_volume().

Note- XML test cases is not implemented as it is deprecated.
New test cases has been skipped for XML.

Change-Id: Iee055ceaad135bba4a57b010d1e08652e2d13598
Use "ip" instead of "ifconfig" to get MAC address

"ifconfig" get different formats of output under different systems,
like CentOS and Fedora, but "ip" can provide uniform format, so we
use "ip" instead of "ifconfig" to get MAC address.

Change-Id: If7559ea96c96ed2f33ddb00737ee0f53f7de4990
Closes-Bug: 1381416
Fix unit tests to detect import failures

This commit adds onto the existing unit test which simply runs
'testr list-tests' to run discovery on tempest. This is done to ensure
that from a bare tempest repo with the requirements all present we can
actually import and list all the tests. However, there wasn't a check
of the return code. Historically this has been an issue with discovery
failing so adding a check to enforce it moving forward should prevent
it from slipping back in. Part of this is to also fix the test so that
it runs discovery on the correct test suite.

In order to make the unit test pass a config dependency in the
scenario test test_basic_server_ops during the testscenarios call in
load_tests had to be removed. Instead if there is a configuration
error raised while running load tests on this module instead of
failing it will just return the base test class. This is fine
because the test itself will fail on the same configuration
dependencies that load_tests is, however it won't happen silently
during discovery anymore.

Change-Id: Ie74c020de7c9c27adc4ff68dddbae5d4481d1224
Prevent external router IPs from being sent back

In anticipation of change Iba7c606eea48181fc10e9d0d5dc667e6f48f37de
being merged or at least a minimum a fix for bug #1255142, the
router response will have external IP address information available
that is not mutable by a regular tenant. This patch removes that
information from the response before replaying it back for a router
update.

Change-Id: Id1da9246bca2a2eee69f751856ffa7052eb18ef9
Fix service validation schema

In case when nova has 'servicegroup_driver' not 'db'
field 'updated_at' in services response is null.

Change-Id: I03610827b87719b76c043b85f1f9002869eb4a5e
Closes-Bug: 1387983
Don't setup identity v3 client if v3 is disabled

Some tests report failures about the identity v3 url not being
specified, even if api_v3 is set to false in tempest.conf.

Change-Id: Ifb5c6c8bd154373d33c90236ff37fcc5b8759405
Closes-Bug: #1389742
Remove "Disabled Reason" from test_cinder_service_list

The CLI test test_cinder_service_list checks the output of
"cinder service-list" for a set of column titles. One of the titles
it expects to find is "Disabled Reason". However, this title is not
guaranteed to be in the output; cinderclient only appends it if
the value shows up in the response from the server.

Change-Id: I060bad255ff17b132737a8878c26e7d5cf9dce82
Closes-Bug: #1392662
Skip some tests when cinder is not enabled

test_delete_server_while_in_attached_volume:
    Skip when cinder is not enabled.
ServerRescueTestJSON:
    No test access the volume resource_setup creates, so stop creating one.
ServerRescueNegativeTestJSON:
    Move volume creation into tests that use it and skip them when cinder
    is not enabled.

Change-Id: I8587c3e5ff3dbf5f23d09a5fd49b2c55c2ef7657
Adds "list external networks" test to regular user

For non-admin user verifies that Public networks are visible but their subnets
are not.

Change-Id: I327bed82a9fe2bbd2d574900dbfd182d27892dc8
Fixes volume client in volume tests

volumes_client is coming from cls.os and volume_client is coming
from cls.os_adm. Here it creates a naming disaster for the volume
client. Adding admin_volume_client at the place of volume_client
fixes the client for admin user.

Change-Id: I0c55d3f7fbf4a096c012ffee3697f14869fe9962
Decouple resources and preconditions in secgroup scenario

Handle secondary tenant checks based on CONF file only instead of using clients

Skip only cross-tenant test instead of whole class if secondary tenant is not
available

Change-Id: I9863ff840b9c5b18fa46826aab3fc107454dcc24
Update docs for regenerating sample conf file

Change-Id: I0999b75f43248b7e3e826845aaabff239aeab013
Fix tox stress invocation to not quote arguments

Change-Id: I6fc1e64c3e52a02126235d5cc42831ee2d25f997
javelin: add network and secgroup resources

This patch aims to bring Neutron support in tempest/javelin by:
- create networks
- create subnets in the networks
- create routers
- connect routers to networks
- Create security groups and rules
- Assign a security group after server creation
- Check security groups survive after upgrade

Partial-bug #1330178
Change-Id: I2f54aec71e452361f9741b4ccb5d5bb6f358abf9
Co-Authored-By: Jakub Libosvar <libosvar@redhat.com>
javelin2: destroy functions above network resources

Implements functions for removal of following resources:
 - networks
 - subnets
 - routers
 - security groups

Change-Id: I447deaa9f320116fc22bb0222eedee10af2f4c59
Partial-Bug: 1330178
Merge "javelin: add network and secgroup resources"
Merge "Adds test for list/get volume attachments V2 APIs"