compute rpcapi 2.29 is not backwards compatible

patch 6fcf4133b49cfefa77151937dec4db097a85c349 which added compute
rpcapi version 2.29 does not provide any backwards compatibility.

Fix this issue with 'self.can_send_version'

Fix bug 1217068

Change-Id: Idc49040b712058aab8d3ed769bdf18bdc0a4f067
Remove a couple of unused stubs

Found a couple of unused stub functions in API fakes when grepping for
something else related to volumes.  Remove these.

Change-Id: I02412f586465d8f628af0a16af969035d779ff1e
Make compute_api use Service and ComputeNode objects

This converts compute_api (and associated nova-api and test code)
to use Service and ComputeNode objects for their work.

Note the removal of the nova.db import from compute/api.py.
There is still a chunk of code that uses self.db, but it's
progress.

Related to blueprint compute-api-objects

Change-Id: I7d0b4c0d02559ec16b35dd38278cdf738e0fda6c
On snapshot errors delete the image

The snapshot, backup and live-backup method create images in glance that
are then uploaded to. Should an issue occur during the image upload, we
should remove that glance image.

Fixes bug 1200804
Change-Id: Ib9b84c6721932c5184cf283b47f74ca57bfb83ce
Add scheduler support for PCI passthrough

PCI tracker updates the PCI stats information which provide an abstract
resource view to the scheduler

Openstack clients request PCI devices using PCI aliases, which are
converted to PCI requests. The PCI passthrough filter checks the PCI stats
for nodes that can satisfy the request of the instance, and gives the
schedule result.

bp:pci-passthrough-base

Change-Id: I0ede4caefab8b22126d0c8b9293f023aa4780e36
Signed-off-by: Yongli He <yongli.he@intel.com>
Signed-off-by: Yunhong Jiang <yunhong.jiang@intel.com>
Introduce AuthFailure EC2 error code

AuthFailure is a valid EC2 error code, use it where appropriate.

Add 'AuthFailure' ec2_code attribute to NotAuthorized exception.

Return AuthFailure during Keystone authentication errors and add unit
tests to cover this. When Keystone returns HTTP error code, use
the same code for EC2 error response.

Implements: blueprint ec2-error-codes

Change-Id: I17306d9a5364e095e787dbf0dbb5b88a899372e3
Introduce IncorrectState EC2 error code

'IncorrectState' ec2_code attribute was added to ImageNotActive and
VolumeUnattached exceptions.

ImageNotActive is now used instead of useless EC2APIError.

Although IncorrectState EC2 error code is used for volumes only in
Amazon EC2, it's used for incorrect image state in nova as well
because there is no error code for this (EC2 images are stateless)
and inventing new error code is a bad idea.

Implements: blueprint ec2-error-codes

Change-Id: I3566bbc4b2eb13482c93933932dd619f90cbc234
Introduce SecurityGroupLimitExceeded EC2 error code

SecurityGroupLimitExceeded is a valid EC2 error code. Add it as an ec2_code
attribute to the exception of same name and use it where appropriate
instead of useless EC2APIError.

Implements: blueprint ec2-error-codes

Change-Id: I1558b9f69c912dab9d829aacd5c5aae1c37f12b9
Introduce UnsupportedOperation EC2 error code

'UnsupportedOperation' ec2_code attribute was added to following
exceptions that are OpenStack specific and don't have mapping to EC2
error codes because EC2 simply doesn't encounter these problems at all.

 * CannotDisassociateAutoAssignedFloatingIP
 * FloatingIpAssociated
 * FloatingIpNotFound
 * NoFloatingIpInterface
 * NoMoreFixedIps
 * InvalidVolume

Catch these direcly in Executor instead of translating them to useless
EC2APIError. This way, user will also get meaningful error messages.

Implements: blueprint ec2-error-codes

Change-Id: I62ea97658cee85e5285f55cba600a08d01fe7cdd
Introduce InternalError EC2 error code

New InternalError exception with 'InternalError' ec2_code attribute was
introduced for the case of timeout when waiting for instance to stop
because there is no fitting existing exception.

InternalError is a valid EC2 error code. Unlike the other implemented
error codes, it's a server error with 500 status code. For that reason,
it's not listed as an expected error in the Executor and thus it's
treated as an unexpected error - it's logged as ERROR and returned
with 500 status code. That is a wanted behaviur because this shouldn't
ever happen and if it does, it should be visible.

Implements: blueprint ec2-error-codes

Change-Id: I11d25abfe46f5b3acdecb4fa2d31f4cec054cba1
Remove EC2 postfix from InvalidInstanceIDMalformedEC2

With new EC2 error code handling based on ec2_code attribute,
there is no justification for the EC2 postfix.

Implements: blueprint ec2-error-codes

Change-Id: I97ede9a3bcba3a6ac48de50fb6f451f8357a4848
Guest-assisted-snaps libvirt implementation

This implements volume_snapshot_create and volume_snapshot_delete
methods which create snapshots of QCOW2-based Cinder volumes.

The update_snapshot_status API from cinderclient is used to
inform Cinder of the result of these operations.

This feature requires a minimum libvirt version of 1.1.1 due
to a bug in libvirt BlockJobInfo handling in earlier versions.

Libvirt implementation for blueprint qemu-assisted-snapshots

Change-Id: I76eb2e4da027a13525314bd58264f482374d270d
Adds 'instance_type' param to build_request_spec

This change adds a new optional parameter to
scheduler.utils.build_request_spec. It also fixes Bug 1219761
by adding the new 'flavor' for cold migration instead of the
instance current one.

Change-Id: I28ac5f636e22e4d02ad5495b7b50c8226678c413
Check ephemeral and swap size in the API

Validate that ephemeral and swap disks passed in trough the API are
within size limits for the given instance type. The validation is done
in the API layer.

Closes-bug #1218977

Change-Id: I96c6e651e4b221313c39dfc41e79d536585fb955
xenapi: add support for auto_disk_config=disabled

Previously, only True and False were supported. This change allows an
administrator to specify which images should reject any user requests
for DiskConfig=AUTO, i.e. reject the request to automatically resize
that instance's root disk.

To specify that an image's disk can't be resized, add the following
property to the image:
auto_disk_config=disabled

In addition, when uploading a snapshot we need to ensure that if
auto_disk_config is "disabled", that property is inherited by any
snapshot. Currently, the disk_config from the database, which is always
False in the case of "disabled", would be persisted instead. When
auto_disk_config is not "disabled", then the current True or False value
for that server should continue to be persisted in any snapshot, and not
the value stored in the source image for that server.

DocImpact
Implements blueprint auto-disk-config-disabled
Change-Id: Ifd41886b9bc7dff01cdf741a833946bed1bdddc3
Improves Hyper-V vmutils module for subclassing

Blueprint: hyper-v-wmi-v2

This refactoring is needed to enable subclassing of the
vmutils.VMUtils class to support the Hyper-V WMI V2
namespace.

Change-Id: Id17b066a05d347ad6b34f826f265208cc4bb22a4
Add os-block-device-mapping to v3 API

Create a new v3 extension called os-block-device-mapping to handle
the block devices specified on server creation. This extension was
os-block-device-mapping-v2-boot in the old API.

Only the new block device mapping syntax has been ported although the
old one can still be used with v2 API.

blueprint: improve-block-device-handling

Change-Id: I898ae56539c6101680983d1eca4bce6d92bd5ce6
Adds support for Hyper-V WMI V2 namespace

Blueprint: hyper-v-wmi-v2

The Hyper-V APIs are mainly based on WMI. The original 2008 Hyper-V
release introduced the "root\virtualization" namespace which got
superseded in Hyper-V Server / Windows Server 2012 by the
"root\virtualization\v2" namespace (referred as V2 in the sources).

Features introduced in 2012, including as an example live migration
and VHDX support are available via the V2 namespace only.

Furthermore, the original namespace has been dropped in the upcoming
Hyper-V 2012 R2 (currently available in preview), which means that
the Grizzly code will not be compatible with it as is.

The Hyper-V driver is structured with a clear decoupling between OS
interaction classes (so called *utils modules and classes) and the
driver's business logic (e.g.: *ops modules and classes).

This allows us to provide an implementation of the V2 API without
impacting the rest of the driver's code, based on the previously
included utilsfactory module to instantiate the proper version of
the *utils classes: the original "V1" ones for versions of the OS
predating 2012 and the newer "V2" ones starting from Hyper-V 2012
(Windows kernel version 6.2).

Hyper-V previous V1 APIs can be forced by setting the following option
in the hyperv section to true. The default is false.

force_hyperv_utils_v1=false

Unit tests have been added for the relevant factory functions.

DocImpact
Change-Id: I5246bdd33ee8e1b607bbf00b748225e933aa417a
Merge "Adds 'instance_type' param to build_request_spec"
Scheduler rpcapi 2.9 is not backwards compatible

Patch 552693e4ad51291c8bfd28cd1939ed3609f6eeac which added scheduler
rpcapi version 2.9 does not provide any backwards compatibility.

This patch adds the check using 'self.can_send_version', and falls
back to the latest Grizzly compatible version if it the version is
capped.

Also takes the opportunity to make the docstring more consistent in the
scheduler rpcapi SchedulerAPI class.

Closes-bug: #1220112

Change-Id: I5a1e08c1066cae00d47def45f7b87846f679cbbe