Adds test for Sec group default rule Nova V2 API

This patch adds the test for below 'os-security-group-default-rules'
Nova V2 APIs-
- Create Security Group default rule
- Delete Security Group default rule
- List Security Group default rule
- GET Security Group default rule

This patch also adds the new json client.

Related-Bug: 1311500

Change-Id: Icf52a64b5719587cd02b906f3a0f8d17886241aa
Read template's expected value orchest test Part-2

In tests under /tempest/api/orchestration/, resources gets
created based on parameters defined in template (yaml, json).
But while verification of output, expected values are hard coded
in tests.
Expected values should be read from template only.
This patch covers the following files -
    test_environment.py
    test_nova_keypair_resources.py
    test_swift_resources.py

Closes-Bug: #1327919

Change-Id: Ie241b7f299487cd0db384778225823e96fecf440
Enable some volumes v2 tests by sharing codes

1. create volumes v2 tests using the existent test class.
2. create a new test class for v1, which inherits v2.
3. add variable 'special_fields' to deal with the different
   field's name and reps code between v1 and v2.

This patch includes the following volumes tests:
- test_volume_metadata.py
- test_volume_transfers.py
- test_volumes_actions.py
- test_volumes_get.py
- test_volumes_negative.py

Partially implements: blueprint cinder-v2-api-tests

Change-Id: I30b8f38d1f8132ea28266a32d2c9cb912a70e2c3
Add "capabilities:" to multiple_backends extra-spec key

cinder-scheduler uses all the "capabilities:" to scope the
cinder-volume. With no "capabilities:" prefix set in extra_spec,
it will be no effect from cinder-scheduler to schedule the
cinder-volume, which is not the purpose of the testcase.
To address the review comment, This change will not remove but
adds 2 cases for the "capabilities:" prefix to volume type key:
extra-spec "capabilities:volume_backend_name".

Change-Id: Ie870091f2a5657b3ab6a0171de2cffe6945a0f7c
Closes-Bug: #1295050
Call clear_isolated_creds in scenario tests

After commit b7144eb08175d010e1300e14f4f75d04d9c63c98, tempest.scenario.manager
doesn't clear isolated credentials anymore. This patch adds a tearDownClass
method to class OfficialClientTest so isolated credentials can be cleared.

Change-Id: Ic03e0d5026e09bff6d1d20daf2ee72bfdb106145
Closes-Bug: #1343027
Fixes incorrect assertion check

assertNotIn/assertIn does not work if the second
variable is of type List of dictionaries.
Suppose if it of only dictionary type, it will try
to match only 'key' but not 'value'.
Hence, the assertion check done here is not correct.

This patch fixes an incorrect assertion check for
"BasicOperationsImagesTest.test_delete_image" test

Change-Id: I7058d209cc92a2e381308e59e07d180e84bde182
Closes-Bug: #1346757
Add client response checking for data processing service

* Checks for response status codes to client side were added
* Checks for response status codes from all tests were removed

Partially implements: blueprint client-checks-success

Change-Id: Ib2a2632460aa39f5c7cfc66dbc601c5fa1a1baa5
Check rebuild server action API attributes

This patch adds the JSON schema for Nova V2 & V3 server action
'rebuild' API response and validate the response with added
JSON schema to block the backward incompatibility change in the future.

The response body of server rebuild V2 API is given below:
{
    "server": {
        "accessIPv4": "1.2.3.4",
        "accessIPv6": "fe80::100",
        "addresses": {
            "private": [
                {
                    "addr": "192.168.0.3",
                    "version": 4
                }
            ]
        },
        "adminPass": "seekr3t",
        "created": "2012-09-12T17:20:36Z",
        "flavor": {
            "id": "1",
            "links": [
                {
                    "href": "http://openstack.example.com/
                            openstack/flavors/1",
                    "rel": "bookmark"
                }
            ]
        },
        "hostId": "1e3da81662360b7e5ea6d8123031f67168b6992f20bb84df69",
        "id": "075e40fe-9f03-4652-ba8e-5f8e2547899a",
        "image": {
            "id": "70a599e0-31e7-49b7-b260-868f441e862b",
            "links": [
                {
                    "href": "http://openstack.example.com/openstack/
                            images/70a599e0-31e7-49b7-b260-868f441e862b",
                    "rel": "bookmark"
                }
            ]
        },
        "links": [
            {
                "href": "http://openstack.example.com/v2/openstack/
                        servers/075e40fe-9f03-4652-ba8e-5f8e2547899a",
                "rel": "self"
            },
            {
                "href": "http://openstack.example.com/openstack/
                        servers/075e40fe-9f03-4652-ba8e-5f8e2547899a",
                "rel": "bookmark"
            }
        ],
        "metadata": {
            "meta var": "meta val"
        },
        "name": "foobar",
        "progress": 0,
        "status": "ACTIVE",
        "tenant_id": "openstack",
        "updated": "2012-09-12T17:20:37Z",
        "user_id": "fake"
    }
}

The response body of server rebuild V3 API is given below:
{
    "server": {
        "addresses": {
            "private": [
                {
                    "addr": "192.168.0.3",
                    "mac_addr": "aa:bb:cc:dd:ee:ff",
                    "type": "fixed",
                    "version": 4
                }
            ]
        },
        "admin_password": "seekr3t",
        "created": "2013-11-14T06:29:00Z",
        "flavor": {
            "id": "1",
            "links": [
                {
                    "href": "http://openstack.example.com/flavors/1",
                    "rel": "bookmark"
                }
            ]
        },
        "host_id": "28d8d56f0e3a77e2089172b68032e017045e20aa5dfc6cb66",
        "id": "a0a80a94-3d81-4a10-822a-daa0cf9e870b",
        "image": {
            "id": "70a599e0-31e7-49b7-b260-868f441e862b",
            "links": [
                {
                    "href": "http://glance.openstack.example.com/
                    images/70a599e0-31e7-49b7-b260-868f441e862b",
                    "rel": "bookmark"
                }
            ]
        },
        "links": [
            {
                "href": "http://openstack.example.com/v3/servers/
                a0a80a94-3d81-4a10-822a-daa0cf9e870b",
                "rel": "self"
            },
            {
                "href": "http://openstack.example.com/servers/
                a0a80a94-3d81-4a10-822a-daa0cf9e870b",
                "rel": "bookmark"
            }
        ],
        "metadata": {
            "meta_var": "meta_val"
        },
        "name": "foobar",
        "progress": 0,
        "status": "ACTIVE",
        "tenant_id": "openstack",
        "updated": "2013-11-14T06:29:02Z",
        "user_id": "fake"
    }
}

Partially implements blueprint nova-api-attribute-test

Change-Id: Ia75f7862bf8eac40f7a9b125fb7e101830da514b
Use python abc in auth class

Instead of using NotImplementedError the abc class has the advantage
that it fails faster if a class doesn't implement the needed interface.
Adapt unit test and test class.

Change-Id: Ide914a4bb577a28a3d202cbf088c404323a29b21
Partial-Bug: #1346797
Fix exception when api_extensions is set to empty

test.is_extension_enabled() is not checking if
config_dict[service] list is empty

Added a check if config_dict[service] is empty the
function will return False which means no extensions
are enabled

Change-Id: I2ee64f205c393637b5fc65897f1489292781c0be
Closes-Bug: 1342516
Adds "user-password-update" to v3/test_users

This submission is to verify user-password-update functionality
and also added required support functions in both XML/JSON
interfaces.

Change-Id: Ifcac99095fb4e07ccaa2b6f87ac836c3959ef37a
orchestration api test_stacks wait for deletion

Wait for delete to complete after triggering it, so we avoid
a conflict exception when the test _clear_stacks cleanup tries
to delete it again.

Change-Id: I64065ad8444d814e2cd7f162b869a2f8eedb1fad
Closes-Bug: #1344989
Check rescue_server action attributes of Nova APIs

This patch adds the JSON schema for Nova V2 & V3 rescue server
action APIs response and validate the response with added JSON schema
to block the backward incompatibility change in the future.

For V3 API, "admin_password" in response body is depend on config
variable "enable_instance_password". So schema and request
is adjusted accordingly.
NOTE - V2 rescue API's response does not on above config variable.

The response body of rescue server V2 APIs is below:
{
    "adminPass": "%(password)s"
}

The response body of rescue server V3 APIs is below:
If CONF.enable_instance_password is true-
{
    "admin_password": "%(password)s"
}
Else
    {}

Partially implements blueprint nova-api-attribute-test

Change-Id: I85f18eaa58bfef1871100816f97487f9fa9bc0ee
Add glance notifications tests for ceilometer

Added ceilometer test for measurements:
image.update, image.upload, image.delete, image.download, image.serve

measurements: image, image.size are not added to tests because they 
have status 'both', these measurements are pollster and notification
ones at the same time.

Partially implements: blueprint add-basic-ceilometer-tests

Change-Id: Id049b6cb8ab3092c50f35894cf5a6bda7ff04617
Avoid errors in log when neutron tests are skipped

The BaseNetworkTest class setUpClass() method will initialize
various class attributes which the tearDownClass() method
expects to exist. Unfortunately when Neutron is not
available, the entire of the setUpClass() method is skipped,
but tearDownClass() still runs. This causes the logs to get
polluted with errors about missing attributes during
tearDownClass(). eg

ERROR tempest.test [-] tearDownClass failed: type object
 'DHCPAgentSchedulersTestJSON' has no attribute 'ipsecpolicies'
TRACE tempest.test     for ipsecpolicy in cls.ipsecpolicies:

Make the tearDownClass() code be conditional on neutron
being enabled too.

Change-Id: I042ca5c47e5c1e8d1042754fd7f36036a40b2f03
Run javelin2 check at the end of create

As part of the creation process we should make sure things are working.

Change-Id: I4941414d88a8d43c4022a52be925204a9eab6a78
Fix javelin2's skip logic in checks

The create logic always requires the top level section in the yaml file,
but it can be empty.

Change-Id: Ia06bba8877be1cf4e66dad21eaeb2a323bf9f9b5
Add links to the field guide index

This commit takes the subsection summary in the field guid index
and adds internal links to the field guide page. Previously when
viewing the field guide docs it wasn't clear how to navigate between
all the different test class pages, this is remedied by adding a link.

Change-Id: Ica93a92c57e5d5d805a8b25d7c7e17509695152b
Add unit test section to the field guide index

The unit tests are another class of tests in tempest, but they were
excluded from the field guide index. This commit adds a section at the
bottom of the index for the unit test field guide.

Change-Id: I0f081c7a94bca46322381f8a5d28d7cadf334bbc
Move CommandFailed exception to tempest.exceptions

When the CommandFailed exception was added it was only being used by
the CLI tests. However, it is generally useful for anything that is
using subprocess to make external calls. This patch moves it to
tempest.exceptions to make using it simpler for non-cli tests to use
the exception.

Change-Id: Ibf5f1cbbb847d32976b54c4484acfc3c0e3b4f48