Fix HTTP response for PortInUse during boot (v3 API)

If an in use port is passed as a network parameter in the request, Nova API
should raise an HTTPConflict (409) error. This was fixed in commit
48b31c9a1f5d2a45d881e70c6223411c5abbcb72 for the V2 API but not the V3 API.

Fixes bug 1212098

Change-Id: I94429f2c04cf81cb88a8739f65a49aa7ca0d49bd
Add RBD supporting to libvirt for creating local volume

The block storage in OpenStack is divided into Nova and Cinder, Nova is
responsible for local volume such as root, swap and ephemeral partition. Cinder
is responsible for normal volume(global volume?). This mode will make difficult
for planning storage allocation. In order to improve the availability of
storage capacity and reduce the complexity of storage allocation, implementing
the unified storage for local volume in Nova and volumes in Cinder is
necessary.

Now Nova supports lvm backend for creating local volume. Ceph as a Cinder
backend we want to implement it too in Nova.

This patch add RBD backend to libvirt image backend, and users can use specify
rbd as the source of root, ephemeral and swap partition.

Implement: blueprint bring-rbd-support-libvirt-images-type
Change-Id: Id97de5c9af4ddc5c58493a5cef21b52386afac3e
Revert task state when terminate_instance fails

Calls to terminate_instance leave the instance task state as-is when
they raise an exception - this was introduced in Ie5701e5c as a
work-around for a bug with quota accounting, but introduced bugs with
power-state syncing. Ib72de1a4 removed the need for the original
work-around, and this change reverts Ie5701e5c.

Ie5701e5c worked around a bug in quota accounting by using the instance's
task state as a signal to communicate between the nova.compute.manager
and nova.compute.api modules: if the manager's terminate_instance method
failed, it would leave the instance task state as DELETING. If the user
retried the delete, the api would check the task state: if the
instance's task state was already DELETING, it would not decrement the
user's quota a second time.

Unfortunately, leaving an instance with an active task had the
side-effect of preventing the VM power-state from synchronizing, wedging
the instance (bug 1177584 and bug 1212420).

Ib72de1a4 changed the quota accounting mechanism, moving the quota
handling code out of the api module. The api code for the work-around to
check the DELETING task state was removed. However, the code for the
work-around in the manager was never cleaned up.

This change cleans up the nova.compute.manager work-around code, making
terminate_instance revert the task state to None upon failure.
Power-state syncing should work again, and quota handling is unaffected
by the change.

Fixes: bug 1177584, bug 1212420
Change-Id: I42c68d16dca56703d6b192bb452883e4d5ca80df
Removes fixed ips extension from V3 API

The fixed ip extension was one of the first ported to the V3 API
but a later decision around not supporting nova network in the V3 API
means that this extension is not required for V3. The fixed ip extension
only supports nova network, not neutron and the same functionality
can be accessed directly through neutron.

Also changes some extension tests to use a different extension
for white/black list testing

Partially implements bp nova-v3-api

Change-Id: Ie589170e863ee489661e789c5134ed8349ced279
Add volume snapshot APIs to driver interface

Add volume_snapshot_create and volume_snapshot_delete to the virt driver
interface.  This is needed to be able to do hypervisor assisted snapshot
operations.

Part of blueprint qemu-assisted-snapshots

Change-Id: I933e000ee56430365f9e380196714f43a85f147f
Add db.block_device_mapping_get_by_id

Add a new db API method for getting a block device mapping for a given
volume ID.  This is needed when Cinder calls over to Nova asking nova to
complete a snapshot operation on its behalf.

Related to blueprint qemu-assisted-snapshots

Change-Id: If3f011e2db14235f4137c9d9cb610f0c4cd7954a
Remove capabilities.enabled from test_host_filters

Update the unit tests to reflect the fact that capabilities will not be
returning enabled any more.  Leave capabilities.enabled in the json
filter tests, as even though capabilities.enabled isn't there the test
still checks the json filters functionality.

Part of bp no-compute-fanout-to-scheduler

Change-Id: Ia8d1dd1ac1ecf609586528f82d4c11650bb328c5
Add context information to download plugins

This patch sends the context information into the download plugins
via  kwargs.  This will allow download modules like swift to use
the keystone auth information.

Change-Id: I105cc7d7dd76e4eadfeefdc7ca65f9eab38098b7
blueprint: image-multiple-location
Introduce Invalid* EC2 error codes

ec2_code attribute with valid EC2 error code was added to several
Nova exceptions to allow simple error code mapping in EC2 API.

Implements: blueprint ec2-error-codes

Change-Id: Ica5cdf4a9662b4b569ded1f1b1931b9a3cdaa9f3
Use implicit nullable=True in sqlalchemy model

In sqlalchemy nullable defaults to True, so we don't need to explicitly
set the value to the default.  By removing all nullable=True everywhere
it makes it easier to spot the nullable=False and also should lead to
less confusion when adding a new column (do I need to add nullable or
not, only some other columns do it)

Change-Id: I22fe2d0caca77005553a6a6f0c7cc2d3e8ea5512
fix bad usage of exc_info=True

To use parameter exc_info in logging, here
(https://review.openstack.org/#/c/38077/) also uses some bad usage of
that. This patch will fix:
* use trackback.stacktrace() instead of exc_info
* use "Error: %s" % exc_info[1] (removing .message to follow PEP352)

Change-Id: Ia661ea9ce760f51359193904b77f7323434ed463
Pass objects for revert and confirm resizes

The revert_resize and confirm_resize compute API methods were previously
converted to accept and use objects.  This actually passes them over RPC
instead of their primitive forms.

Implements blueprint unified-object-model

Change-Id: I6dbd5fc0ddcaa37013e885b71aabede92eedf120
Sets policy settings so V3 API extensions are discoverable

Adds the policy settings so the V3 API extensions
are by default discoverable through /v3/extensions. Some of
these policy settings were missed during the porting of
the extensions as the ability to make extensions not discoverable
was done part way through.

Partially implements blueprint nova-v3-api

Change-Id: If2efa15a437d97be8bc026fb04c46ec47eef056f
Add mock to test-requirements

There are several changes which depend on using mock rather than mox for
unit tests:

https://review.openstack.org/#/c/42474/
https://review.openstack.org/#/c/43161/

And the general trend seems to be to move from using mock over mox for
new tests (see blueprint mox-to-mock-conversion).

And given mock is listed in the global-requirements.txt file in the
openstack/requirements project, we should require mock for nova so we
can start using it.

Change-Id: Iaa0f233c28de2f92a237ee6d089858ee10bc6d2f
Merge "Use implicit nullable=True in sqlalchemy model"
Merge "Pass objects for revert and confirm resizes"
Make compute_api use objects for lock/unlock

Converts compute_api.lock and unlock to require objects and modifies the
callers to pass them.

Related to blueprint compute-api-objects

Change-Id: Ice8361328cdfeb92207c778beb07b55a9d4eda8d
Convert reset_network to use instance object

Converts all calls to reset_network to use objects.  Also adds the
cells rpcapi methods to match compute rpcapi and ditches the special
cells code in compute/cells_api.py

Related to blueprint compute-api-objects

Change-Id: If4da142864cbe993a7cc852358e3b27bd8c02629
Make inject_network_info use objects

Makes the inject_network_info path use instance object.

Related to blueprint compute-api-objects

Change-Id: I8d6b90cac0c1f638b623def319f676e95b416e09
Make backup and snapshot use objects

This makes the backup and snapshot paths use objects.  To make things
cleaner for syncing compute rpcapi and cells rpcapi, the
single snapshot_instance rpc method that was used for both snapshot and
backup has been split into backup_instance and snapshot_instance.  The
appropriate cells versions have been added.

The backup and snapshot tests have also been converted into unit tests and
moved into test_compute_api.py.

Related to blueprint compute-api-objects

Change-Id: Ibdaee07290cf8d61c106702aa7068dd308bc9aac