Fix arguments for method expected_success

In expected_success method we expect to get status code of
API response, but we use incorrect argument 'resp' instead
of 'resp.status' in several files.

Change-Id: I28f9e4862ddd42322757f5b206ddfb6ddabcfe0f
Closes-Bug: #1354389
Do not run fwaas API tests in smoke

Now that we are running full tempest against neutron, and because fwaas
is not part of core functionality (and not fully baked). Remove these
test from smoke as part of the effort to only test advanced services
against neutron and not against all projects.

Related patch: Ib315d30ff3c4463203f7ebe00e159a9680cf1616

Change-Id: I2e0ce9641becabbeed5f6ad82ad478d84a37f36f
Work toward Python 3.4 support and testing

Change-Id: If36e1a5e51cf63e499acb374d4aeace608c13512
Make output from check_logs less verbose

This commit changes the output of check_logs.py to be a bit more
concise and clear. Instead of printing out every instance of errors
found in the log files it will only print out which individual files
had errors. If a file is marked as not allowing ERRORS or TRACES it
also prints that along with the filename. This should hopefully avoid
confusion around the operation of this script running in the gate.

Change-Id: Ib2a9a9736470cd3b445fb51a95a1a6c6696bb2cc
Fix unstable assertion in test_cinder_endpoints

In simple_read_only.test_cinder.test_cinder_endpoints, the test case
assumes that the first table of 'cinder endpoints' always ['nova',
'Value']. However, it is sometimes wrong.
So in this commit, the test verifies that they have 2 or more headers
and has a 'Value' header.

Change-Id: Idb3f2ebc6b7e05178621ac3701b7c3d02e1d7591
Closes-Bug: #1363018
Closes-Bug: #1362703
Fwaas API Test Enhancement

Add Test to insert and remove firewall rule to firewall policy

-Create a firewall rule
-Create a firewall policy
-Insert a firewall rule to policy
-Validate the insertion of rule to policy
-Remove the firewall rule from policy
-Validate the removal of firewall rule from policy

Change-Id: I3c9711294e1e6323bdad4ff783de1ed19e133c1b
Make SwiftScenarioTest methods public

An earlier refactoring (commit
0d49411dbb7d6818bb6ca82ba5b769eeffd8f305) extracted the test manager
to a new class. In order to limit the surface area of the change
existing methods name were left unchanged.

This change updates those method names to make them explicitly
public so future test developers recognize their availability.

Change-Id: I3f3492dbe122797d9683e84373830e820cd8c038
Add a test for deleting multiple objects by POST method

From swift Icehouse version, bulk deletion supports POST method as well as
DELETE.

Extra fixes:
 - Bulk is an optional function with WSGI middleware, therefore each test
   is annotated with @test.requires_ext
 - Refactor test_account_bulk.py by separating common codes to one method
 - Simplify delete_account method of the Swift's account service client

Partially implements blueprint add-icehouse-swift-tests

Change-Id: I3b63bc730f993cdb626c5a77c18ae917d98e911b
Backward compatibility to credentials in conf

Before switching to the YAML based configuration as the default for
credentials, we need to introduce backward compatibility to allow
credentials to be configured in tempest.conf.

Partially implements bp:test-accounts

Change-Id: I29c66c87d02f442b1d5b0b2d7c82ff2a140c111f
Add test list_servers_filtered_by_name_regex

The nova servers API supports full regex filtering on names. Add a test
for basic regex support.

Change-Id: Iddd5908d7daa5d5b20cb29dd0f403fcdd3ecefcf
Unskip test_create_and_get_delete_bucket()

test_create_and_get_delete_bucket() was skipped because of bug 1076965
which is now in a state which should allow us to unskip it.

Change-Id: I56bc2bfb5a3019cf4a2979b40c7d7cbe3c8cf71a
Related-Bug: #1076965
Unskip test_update_port_with_second_ip()

test_update_port_with_second_ip() was skipped because of bug 1364166
which is now in a state which should allow us to unskip it.

Change-Id: I263bd4ef45ea02b18c6d00b6e27f0b12442be9ee
Related-Bug: #1364166
Migrate test_server_cfn_init to tempest clients

Migrate the test and change OrchestrationScenarioTest used by the test.
This is a part of the migrating all scenario tests to using tempest own
clients.

Change-Id: Iafc1b021fe8a9a5daa0e2d7eef0542b7929c415a
Partially-implements: bp:tempest-client-scenarios
Add response schema for V2 sec grp default rule

This patch adds response schema for nova V2
'os-security-group-default-rules' APIs.

Response body of create/show 'os-security-group-default-rules' -
{
    "security_group_default_rule": {
        "from_port": 80,
        "id": 1,
        "ip_protocol": "TCP",
        "ip_range":{
            "cidr": "10.10.10.0/24"
        },
        "to_port": 80
    }
}

Response body of list 'os-security-group-default-rules' -
{
    "security_group_default_rules": [
        {
            "from_port": 80,
            "id": 1,
            "ip_protocol": "TCP",
            "ip_range": {
                "cidr": "10.10.10.0/24"
            },
            "to_port": 80
        }
    ]
}

Delete API only returns 204 status code.

Change-Id: Id24857ee95445570d45aac3e1a25e607d8df78d1
Add subnet tests for extra attributes

* Tests added for ipv4/6
test_create_delete_subnet_with_allocation_pools
test_create_delete_subnet_with_gw_and_allocation_pools
test_create_delete_subnet_with_host_routes_and_dns_nameservers
test_create_delete_subnet_with_dhcp_enabled
test_update_subnet_gw_dns_host_routes_dhcp
test_create_delete_subnet_all_attributes

* Adds helper to create/verify/delete subnet
* Adds helper to delete network

Implements: bp subnet-extra-attributes

Change-Id: I7c1b00f4fad797eb5dd904f1123b5bcd5e5af649
Fix scenarios not passing down specific network

Nova v2 api expects nics in specific networks to be passed
down in an array of "networks". nics is used in novaclient
but converted to networks in
python-novaclient/tree/novaclient/v1_1/servers.py#n520

Apply the conversion to the parameters to make the check
pass in case where there would be multiple networks
matching the requirements. Also remove a case where scenario
manager forced in the wrong network into the tests.

Closes-Bug: #1368834
Change-Id: Ic0b87377c5878e240a0fbcc6549023c828dd97ab
Removes bogus negative create image compute

Removes test_create_image_specify_multibyte_character_image_name
as it is not a valid test. It currently does not send an
invalid utf-8 sequence, but a valid unicode one. Also we can't actually
send an invalid utf-8 sequence via JSON as it is not possible to
encode an invalid UTF-8 sequence into a series of unicode characters
which is what JSON uses as strings. So its not possible for it
to fail this way. And we don't do XML testing for this type of test.

Change-Id: I39b5c017c8a173f8dd56a14c1bc7dc6952ac787d
Partially-Closes: 1006725
Migrate TestStampPattern to tempest client

Migrate TestStampPattern to tempest client. The test is currently
disabled in the gate, so this will be tested by adding a temp
patch on top removing the skip.

Partially-implements: bp:tempest-client-scenarios

Change-Id: I42524372ae64121f2e3b58f37b960f8d1bbc8b6a
Framework for resource safe class level fixtures

Modify BaseTestCase, add resource_setup and resource_cleanup
methods, invoked by setUpClass and tearDownClass respectively.

Partially-implements bp:resource-cleanup

Change-Id: I5f025b4f76d6d85fbefbada9475daf784425037a
Migrate computev2 API tests to resource_* fixtures

Partially-implements bp:resource-cleanup

Change-Id: I59eb2d9789368c812282ce12ae7e571be720badb