Return Customer's Quota Usage through Admin API

DocImpact: This patch adds an extension 'UsedLimitsForAdmin'. If this
extention is enabled then it extends the used limits API behavior, such
that admin can fetch the details of any customer's quota usage by passing
the customer's tenant id in query parameters.The API signature for the
same is 'v2/{tenant_id}/limits?tenant_id={customer_tenant_id}'

Change-Id: I89b8b5083e46b899458407426c89a3865e960faa
Implements: blueprint customer-quota-through-admin-api
add xml api sample tests to os-tenant-network

Change-Id: Ib79defe49c82a175cb8605aa3a65ccd2c48d68f7
Add missing conversion specifier to ServiceGroupUnavailable

The message in the ServiceGroupUnavailable exception was missing the 's'
conversion specifier.

Fix bug 1183733

Change-Id: I54869431511a72c49ccf61f5e7b1a30607959423
Removing misleading error message

This function (_get_instances_on_driver) was reporting instances are on the
hypervisor and not in the database with no consideration for the filters being
used in the database query. This resulted in error log messages indicating running
instances were not in the database when in fact the query was filtering for deleted
instances.

To avoid misleading error logs of this nature I've removed the logging from the
function. Currently the locations which call this function provide more accurate
logging on the state of the instances that are returned based on the filters they
have called _get_instances_on_driver with.

bug 1183276

Change-Id: I4ee52fe417f73a90cf157bc531b483be93e34275
libvirt: improve the specification of network disks

Allow zero or more hosts, and make the name optional, since only a
host is needed to use nbd. This allows cinder to fully specify the
location of a sheepdog or rbd volume without depending on extra
configuration files, and lays the groundwork for supporting nbd.

Rename the internal attribute source_host to source_name to reflect
its usage in the libvirt xml, and add source_hosts and source_ports
attributes to store the new information.

Fixes: bug 1077817
blueprint better-libvirt-network-volume-support
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Change-Id: I8ac431751692e52ba0786768cea996388962922d
Fix msg version type sent to cells RPC API

Regression fix from 6b16c8731c44e4a6c80b803f3e8afdd88386d577, cannot
build instances through cells.

The RPC dispatcher dislikes floats for version numbers.

bug 1190371

Change-Id: I18994ce8a524f88910cfe37c9b302296ce30102f
Fix _drop_unique_constraint_in_sqlite() function.

We got incorrect unique constrains in function
_drop_unique_constraint_in_sqlite().
New function `_get_unique_constraints_in_sqlite()` was added to fix it.

Change-Id: Iebb4c0e6c3e6a069ba5cde2fdb2a2ff1200b826f
Fixes: bug 1184526
Fix nova-compute fails to start if quantum is down

When nova-compute starts it needs to check the state of instances that
were previously running and needs to query quantum. Previously, if
it queried quantum and quantum raised an error or was unreachable
nova-compute would terminate. This patch adds a retry loop to prevent
this from happening.

Fixes bug 1186357

Change-Id: I0d865e464f7bf60a0b6dc4ca8a42436d16e29790
Add notes about how doc generation works.

Change-Id: I562c743798aeae5e49bd2f96944c3cacd776a53d
Add yolanda to the mailmap.

Noticed a duplicate entry in the AUTHORS file, added to .mailmap.

Change-Id: I649df9d7273347bec72076dc16a5c2217d1bf1c1
Normalize path for finding api_samples dir

When nova/tests/integrated and doc/api_samples are installed
in different subtrees and are only combined via symlinks,
running test_api_sample fails due to traversing into the
wrong tree. Use path normalization to avoid this problem.

Fixes: LP Bug #1186775

Change-Id: Ibe9e2442dc9bf3ba7bcb429cbac124dcaeea2cef
Silence exceptions from qpid connection.close() (from oslo)

This is commit 15d8d698b7 in oslo-incubator.

Fixes bug 1185465

Don't log exceptions that happen during connection.close().  They're
not surprising (a closed socket can't be read or written), don't
require sysadmin attention, and provide red herrings in the logs
to distract from finding real root causes of problems.

Change-Id: Ia0e25fbc11094b5ad5d6d2cf7d0e786acb3df1f6
Refactor libvirt driver exception handling

Centralize exception handling for id based instance lookups in new
method _lookup_by_id().

Change-Id: If8d215b85fabf235afcf78a9203378fbb26e68db
Fix a race where a soft deleted instance might be removed by mistake

When a soft deleted instance is restored there is a tiny risk that it
might be deleted if the reclaim periodic task runs at the same time.
This happens because the restore operation sets deleted_at to None
which makes the reclaim job think it is ok to delete the instance.

This can be resolved by restricting reclaims to only those instances
whose task_state is None.

Resolves bug 1186243.

Change-Id: Ia138e2d504b7482c46900583f019a32c70513245
correctly set iface-id in vmware driver

bug 1183452

Commit 70c659059b8fbef811ce79700aecb01c60242ebd updated the vmware
driver to use the new vif model, but was incorrect in how it tried to
grab the iface-id and set it in the vmx file spec.  This patch fixes
that issue and adds a check for this in the existing unit tests.

Change-Id: I756539871d1844a828f9a0a295fc9fa3e59610f7
Add x-compute-request-id header when no response body

The x-compute-request-id header was only being added for requests which
generated a response body, like an index or show.  This adds it to
requests which generate no response body, like create/update/delete, or
actions on servers like resize.

Bug 1183712

Change-Id: Ia0e9d75e4b84280caab060fae58036ed818e00c3
Allocate networks in the background

Setting up networks and allocating IP addresses has the potential to
take an undesirable amount of time, blocking the build of a new
instance.  We can parallelize some work by querying for this information
*while* a new instance is being provisioned in the virt driver.

This adds a wrapper around the NetworkInfo model which allows one to
query for network information in a greenthread.  When a code path is hit
that actually needs the network information, it waits for the
greenthread to finish, if it hasn't already.

Network allocation will only be async for virt drivers that accept this
NetworkInfo model.  Other drivers that expect the legacy format end up
waiting for the operation to complete before driver.spawn is called.

Implements blueprint async-network-alloc

Change-Id: I389f939b9a36bc20b3b66e98d006d694622bd963
Regenerate missing resized backing files

Fixes bug 1185588

The only way to reliably determine whether we are using a legacy
(pre-Grizzly) backing structure is to look at the backing file
reference for the instance/disk itself. Thus, if we determine the
backing file contains an underscore, we use the value following it
as our legacy_backing_size, which is then used to create the
necessary backing file.

Change-Id: I145f71068c972a604e5361ae79d6315286136786
Converts 'image' to json primitive on compute.rpcapi.prep_resize

The 'image' should be converted to primitive since it contains
complex structures. Today it works because it relies on
scheduler.rpcapi.prep_resize which does the conversion.

This is part of other changes that will move resize/cold migration
to conductor. scheduler.rpcapi.prep_resize will be moved and then
compute.rpcapi.prep_resize has to properly convert parameters.

Partially implements bp cold-migrations-to-conductor

Change-Id: Id89896bfb379ab9eb8bf13c3513b9290d88408fd
Enhance unit test code coverage for availability zone

In current unit test code for availability zone, the update and
delete operation for AZ is missing. So this review submits two
test methods to test AZ update and delete.

Change-Id: Id98756351f0ecdf5405b9dfea2602a9aee0c5522