Fix arguments for method expected_success

In expected_success method we expect to get status code of
API response, but we use incorrect argument 'resp' instead
of 'resp.status' in several files.

Change-Id: I28f9e4862ddd42322757f5b206ddfb6ddabcfe0f
Closes-Bug: #1354389
Fwaas API Test Enhancement

Add Test to insert and remove firewall rule to firewall policy

-Create a firewall rule
-Create a firewall policy
-Insert a firewall rule to policy
-Validate the insertion of rule to policy
-Remove the firewall rule from policy
-Validate the removal of firewall rule from policy

Change-Id: I3c9711294e1e6323bdad4ff783de1ed19e133c1b
Add test list_servers_filtered_by_name_regex

The nova servers API supports full regex filtering on names. Add a test
for basic regex support.

Change-Id: Iddd5908d7daa5d5b20cb29dd0f403fcdd3ecefcf
Add tempest post-run cleanup script

Implemented all features defined in blueprint.

Change-Id: If3c9b82f85095eeb22bbf32841be534024213567
Implements: blueprint post-run-cleanup
Fix use of code name in services decorator

This commit fixes an incorrect usage of ceilometer in the services
decorator, telemetry is the correct service tag name. This also adds
a unit test to catch the incorrect usage of a service tag.

Change-Id: I5c231ec3ba064013f8d5fc2f49dbdf28e5c637c1
Fix MismatchError for LB scenario test

It has been observed that at a given iteration only one of the two HTTP
services responds to requests made to the VIP. This is somewhat rare and
it may be caused by the fact that the number of requests being sent is
not high enough for the ROUNDROBIN algorithm to equally distribute requests
across all the members in the pool.

Increasing the number of retries makes the test runtime only slightly
longer, but it should give us more confidence that, if we keep observing
the same type of failure, this can be ruled out.

At the same time, this patch also relaxes the check so that it just verifies
that member pools are load balanced at least once. Verifying that the number
of requests are spread evenly across is always going to be prone to transient
condition and it is beyond the scope of a scenario test that validates whether
Neutron LBaaS is wiring things correctly.

Closes-bug: #1358857

Change-Id: Ibc0aeb092a4643eb151cef8d7d942d9e3d1ed71c
Fixes image create utf-8 multibyte test

Fixes test_create_image_specify_multibyte_character_image_name
so that it actually sends a UTF-8 multibyte character rather
than unicode. Also changes it to a 3 byte multibyte character
rather than a 4 byte multibyte character because glance currently
doesn't handle 4 byte utf-8 and will 500.

Change-Id: I064b563024a090899a4adcbbd462a18c5450830c
Related-Bug: 1006725
Add an internal test method for rebooting a server

test_reboot_server_hard and test_reboot_server_soft contain almost
the same code and they are redundant.
This patch adds an internal test method and makes them use the method
for cleanup.

Change-Id: I98fc519b5bb2de8cc47007ba9138247f4c0f3369
Reduce complexity during import phase

In the current state of the negative testing framework the import phase
consists in many complex operation during import phase. This fix reduces
the complexity during that phase to improve the performance,
debugability and robustness.

In this fix we only determine the test scenarios in the import phase. The
payloads are created while test execution. This also improves the
flexibility since the test executor has full access to the test variables.

Change-Id: I8deed33d37c5bf07c26118d6d66ada105d259c89
Partially-implements: bp autogen-negative-tests
Fix schema definition admin_flavor_create

The api of flavor_create has a missing property level. Property "id"
is noneable and need to be excluded from negative testing.

Change-Id: I411eb8c29d0c00d6d64358ede26992ee2fe380c9
Closes-bug: #1375142
javelin: fix user destruction

Use the right Python method to destroy an user:
- get the tenant_id
- get the user_id (with tenant_id)
- delete the user using user_id

Change-Id: I2121fedf563cc4d41f8ad8dfcaf55d767d87d69f
Add the start message for debugging net info

When a network scenario test fails due to networking timeout, we can
get both server console log and network information for debugging.
Current network information doesn't contain the start message and
it seems a little confusion where starts network info.
This patch adds the message.

Change-Id: I8490c08b7073302d299f7b63dd9d6380d8a8d27e
Related-Bug: #1323658
Drop OfficialClientManager and references to it

OfficialClientManager is unused now that scenario tests use
clients.Manager instead. Dropping the class, the hacking
rule that forbade its use in API tests and related unit tests.

Change-Id: Ib359b9ab0a903436d9d9c265e8c4d82bdd80a496
Drop client_type for auth module

Client type was designed to allow the auth module to consume
the official keystone client for tests based on the official clients.
Scenario tests have been migrated to tempest client, and CLI tests
are being moved out of tempest, and they don't need an external
auth provider anyways.
There's no need for client_type anymore, removing it.

Change-Id: I6eebd80ea2259b72e8013e65e9a83615f86a3f84
Partially-implemnts: bp/tempest-client-scenarios
Removes deprecated net_common module

Module net_common was replaced by module net_resources as part of migration of
scenario tests to tempest clients.
This patch fixes a few rogue references and removes this module completely.

Change-Id: I5426e030895d219ac456b3cceb361675c839b876
Partially-implements: bp/tempest-client-scenarios
Merge "javelin: fix user destruction"
Remove "nova-manage flavor list" CLI test

The "nova-manage flavor" sub-commands were deprecated in Juno and Nova
change I1798c8a0697009b3dcebeb818580c8323880e395 removes them.

This removes the test_flavor_list test and changes the test_debug_flag
and test_verbose_flag tests to use 'nova-manage service list' instead of
'nova-manage flavor list'.

Change-Id: I4e2399f2fbab2c20299dd62c104b9220c1d36fb2
Revert "Add "reboot_instance" EC2 API test"

This reverts commit 120739800aac59875f2eef003f51bd2da1c1a5a4.

This test never worked well it hit bug #1352341.

Change-Id: Ia254c47119f8936c9f5b3ab60a8a2d75c10e32f2
Related-Bug: #1352341
skip cfn_init test, it is failing too much

And there is no current progress on the bug.

Change-Id: I2604805d06a87973b8c30e56457fcd3648a666f4
Related-Bug: #1374175
Merge "Add test list_servers_filtered_by_name_regex"