Fix resource_setup order in scenarios

As resource_setup() document says, we should check skip condition before
resource_setup. However some functions in scenarios setup resources
before skip checks. We should avoid to setup unnecessary resources as
possible. This commit fixes the order.

Change-Id: Id0529fc80f2b80f028fd078140fe5fd7459e9309
Added neutron cli test case for vpnaas

vpn_list,ikepolicy,ipsecpolicy,ipsec_site

Change-Id: If61eb1506da464b0e173ef63c4dbccdd8f757616
Test to update port with CIDR value

creating a port with allowed address pair
verifying the port creation with allowed address pair

creating a port, updating a port with CIDR value
verifying the port creation with CIDR value

creating a port, updating a port with Multiple ip and Multiple mac
verifying the port updation with Multiple ip and Multiple mac

Change-Id: I22a041df97f9f630b66c48f92f4c0da12db75b0a
Security Group ICMP validation for icmp-code

This patch would ensure that Security Group rule creation would
fail when icmp-code alone is provided.

Partially implements: bp ipv6-api-testing-parity
Change-Id: Ib630a72b0979c9738c80cbb0df862bab10b30e5c
Adds test for list/get volume attachments V2 APIs

This patch adds the test for following Nova V2 APIs-
- List volume attachments
- Get volume attachment
Response Schema has been added for the same.

To make consistent in server_client and easy for tests to compare
the response, this Patch also change the returned response body from
server_client's attach_volume().

Note- XML test cases is not implemented as it is deprecated.
New test cases has been skipped for XML.

Change-Id: Iee055ceaad135bba4a57b010d1e08652e2d13598
Create subnet without gateway and explicit IP ver

Now it's impossible to create subnet without gateway in network
tests. This patch allows you to set gateway explicitly to None.
Backward compatibility is supported: by default it creates
subnet with default gateway as before. Also it adds possibility
to create subnet with specific IP version when you need to create
two subnets in one tenant of different IP version ("dual-stack").
Fixed attributes test for new requirements and added 2 anothers.

Change-Id: I7aca5e07be436f20cba90339785b46182d97fead
Use "ip" instead of "ifconfig" to get MAC address

"ifconfig" get different formats of output under different systems,
like CentOS and Fedora, but "ip" can provide uniform format, so we
use "ip" instead of "ifconfig" to get MAC address.

Change-Id: If7559ea96c96ed2f33ddb00737ee0f53f7de4990
Closes-Bug: 1381416
Fix unit tests to detect import failures

This commit adds onto the existing unit test which simply runs
'testr list-tests' to run discovery on tempest. This is done to ensure
that from a bare tempest repo with the requirements all present we can
actually import and list all the tests. However, there wasn't a check
of the return code. Historically this has been an issue with discovery
failing so adding a check to enforce it moving forward should prevent
it from slipping back in. Part of this is to also fix the test so that
it runs discovery on the correct test suite.

In order to make the unit test pass a config dependency in the
scenario test test_basic_server_ops during the testscenarios call in
load_tests had to be removed. Instead if there is a configuration
error raised while running load tests on this module instead of
failing it will just return the base test class. This is fine
because the test itself will fail on the same configuration
dependencies that load_tests is, however it won't happen silently
during discovery anymore.

Change-Id: Ie74c020de7c9c27adc4ff68dddbae5d4481d1224
Prevent external router IPs from being sent back

In anticipation of change Iba7c606eea48181fc10e9d0d5dc667e6f48f37de
being merged or at least a minimum a fix for bug #1255142, the
router response will have external IP address information available
that is not mutable by a regular tenant. This patch removes that
information from the response before replaying it back for a router
update.

Change-Id: Id1da9246bca2a2eee69f751856ffa7052eb18ef9
Increase the 'all' timeout because slow tests are included

The default for tempest in .testr.conf is 500s. This was too short for
the heat slow tests so the time for that was increased to 1200. This
change is to a periodic job so if we need to it can be increased more
in the future.

Change-Id: If27ddef9865c54b997130c707382d2014d82e0a6
Fix service validation schema

In case when nova has 'servicegroup_driver' not 'db'
field 'updated_at' in services response is null.

Change-Id: I03610827b87719b76c043b85f1f9002869eb4a5e
Closes-Bug: 1387983
Fixed a comment on dirty logs list

We've recently changed default behaviour to forbid any dirty logs for
services outside the list instead of explicitly enabling log checks for
specific services. The comment was not updated to reflect the change
though.

Change-Id: I5d5ba64d156a9b5318cfc2740b2a69d75de3c127
Don't setup identity v3 client if v3 is disabled

Some tests report failures about the identity v3 url not being
specified, even if api_v3 is set to false in tempest.conf.

Change-Id: Ifb5c6c8bd154373d33c90236ff37fcc5b8759405
Closes-Bug: #1389742
Increase default maximum safe_body function

There is this specific test test_absolute_limits_negative which's xml
version is bigger than 2048 characters and was truncated on tempest.log
that is fixed increasing that maximum even though I'm not sure how long
xml tests will live here.
Closes-Bug: #1383996

Change-Id: I771921e3c14333afc791e21395c0edf9feba5ee0
Remove "Disabled Reason" from test_cinder_service_list

The CLI test test_cinder_service_list checks the output of
"cinder service-list" for a set of column titles. One of the titles
it expects to find is "Disabled Reason". However, this title is not
guaranteed to be in the output; cinderclient only appends it if
the value shows up in the response from the server.

Change-Id: I060bad255ff17b132737a8878c26e7d5cf9dce82
Closes-Bug: #1392662
Skip some tests when cinder is not enabled

test_delete_server_while_in_attached_volume:
    Skip when cinder is not enabled.
ServerRescueTestJSON:
    No test access the volume resource_setup creates, so stop creating one.
ServerRescueNegativeTestJSON:
    Move volume creation into tests that use it and skip them when cinder
    is not enabled.

Change-Id: I8587c3e5ff3dbf5f23d09a5fd49b2c55c2ef7657
Add wait for async secgroup add to minimum basic scenario

Security group add actions are asynchronous so we can't assume that
the security will be available immediately to ssh into the server.
This commit fixes the issue by adding a wait for the secgroup to show
up in the server before we move on.

Fixes-bug: 1390355
Change-Id: Idd561d77ce2e4b83421bb7bfb6327a6c940b62ed
Remove ephemeral size assertion in baremetal scenario

The test currently fails if the the baremetal flavor has no
ephemeral disk set.  This removes an unnecessary assertion and
unblocks tests against environments that do not deploy ephemeral
disks.

Change-Id: I7f13d4d7b40ea8879eaa30a19c117aef284586ff
Merge "Increase the 'all' timeout because slow tests are included"
Merge "Remove ephemeral size assertion in baremetal scenario"