Fix security group race condition while creating rule

Previously, it was possible for someone to create a security group rule
where the rule references another security group. To do this nova would
first create the security group rule and then look up the referenced group
(group_id's) name in order to return that via the api. During this time
it's possible for someone to delete this security group rule and the
referenced group before the call returned resulting in a 404 error being
raised. This patch addresses this issue by looking up the group name
first and then creating the security group rule in order to avoid this
from occuring.

Note: this patch also adds a test to cover the case where an invalid group
id was passed though this does not really add any real additional coverage
to the change largely because the code is currently using global stubs. That
said, the previous coverage should hopefully be sufficient

Change-Id: If58ffa5629ba5166f260379ac47922974de31be0
Related-bug: 1262566
Reduce unit test times for glance

The class TestGlanceApiServers did not inherit
test.NoDBTestCase.

Change-Id: Ieafe300d793685f5b146999383695733eba3da19
VMware: cleanup the constructors of the compute drivers

The 'read_only' parameter was never used by the compute drivers.
The 'scheme' parameter should be passed to the base constructor.

Change-Id: I3abd40a50896664ee274f057ded8ec2460679c82
Add debug log for core_filter

Sometimes operator need information why the host doesn't pass
the check of scheduler, this patch adds information for core
filter if virtual cpu is not enough on the host.

Change-Id: Ie05d9184435a4fc4cff1af26812c01a3cc226fff
Partial-Bug: #1301830
object-ify availability_zones

This change converts availability_zones to use the Aggregate and
Service objects instead of direct database access.

Related to blueprint compute-manager-objects-juno

Change-Id: I2ec7373f063ed728dde2af3b7c8259f4391885bc
object-ify API v2 availability_zone extension

This change converts the availability_zone extension to use the
Service object instead of direct database access.

Related to blueprint compute-manager-objects-juno

Change-Id: If5baafd8ec85be1ea1cd6580ae10ef3d51366c9c
Restore instance flavor info when driver finish_migration fails

when instance resize, it will call finish_migration at last to create
new instance and destroy old instance. If driver layer has problem in
create new instance, the instance will be set to 'ERROR' state.

we are able to use reset-state --active to reset the instance and use it
but the instance information is set to new flavor
and not reverted to old one

Change-Id: I5961260f50e7893c7bf03e329e06edfd4f295640
Closes-Bug: #1296519
Fix error status code for multinic

When passing bad body in a request, most APIs return BadRequest response.
However, multinic API doesn't do it.
This patch fixes the error status code and adds a unit test related to
this change.

This patch's idea most came from
following patch: https://review.openstack.org/#/c/107266/

Change-Id: If1ef2077a0e9823dfc04e4ef5a6dfb02daf1a8dc
Restore ability to delete aggregate metadata

Commit I2c778f2237ba5bd2aa8335a0eae80f3aad3e9157 tightened the
input validation for setting the metadata for an
aggregate. However it also accidentally excluded sending
null as the metadata value which has previously allowed the
deletion of the metadata key rather than an update. This
changeset relaxes the input validation to allow deletion of
aggregate metadata again

Change-Id: I54b91c2d421243e7e7521f2352598f73a258b88a
Closes-Bug: 1334846
Add decorator expected_errors to V3 servers core

All v3 api should use expected_errors to prevent unexpected exception
raised. This patch add decorator expected_errors for v3 servers core.

Change-Id: Ia1fa09d6cf68a65c85ac5fd2aff9b9202d76e7ca
Closes-Bug: #1343778
Add decorator expected_errors for v3 attach_interfaces

All v3 api should use expected_errors to prevent unexpected exception
raised. This patch add decorator expected_errors for v3 attach_interfaces.

This patch also remove update action. Let the wsgi return HTTPNotFound for
it.

Change-Id: Icefe9f9adfa7ad01bc3ab11dcb7c132702390d04
Fix error status code for agents

When passing bad body in a request, most APIs return BadRequest response.
However, agents API doesn't do it.
This patch fixes the error status code and adds a unit test related to
this change.

This patch's idea most came from
Ken'ichi Ohmichi's patch https://review.openstack.org/#/c/107266/

Change-Id: I9602183ade4d3c0105e2498aebb7e33eb72f86f0
Remove unnecessary error log in cell API

There are unnnecessary error logs in cell API. It tried to log
error for invalid input which is not necessary because we already
raises exception for the bad request.
This patch removes them in both v2/v3 API.

Change-Id: Idd239903aa830f5d8fe50336dfa63a43b617584f
Keep resizing&resized instances when compute init

During compute manager startup init_host is called. One of the
functions there is to delete instance data that doesn't belong
to this host i.e. _destroy_evacuated_instances.
But this function only checks if the local instance belongs to
the host or not. It doesn't check the task_state or vm_state.

In Resize function, user may want to revert or confirm the resize
operations so the instance on source and dest compute node should
be kept. so for RESIZE_MIGRATING, RESIZE_MIGRATED task states and
RESIZED vm state instances, they should be kept in compute node
when the compute restart. This patch adds check for the task
state and vm state before delete the instances.

Closes-Bug: #1330503

Change-Id: I723fa4a8823019391ea83aa189096531032adab1
Inject expected results for IBM Power when testing bus devices

Same as in https://review.openstack.org/#/c/93621 this patch
inject expected values when the test is running on IBM Power
platform, since the ide bus isn't supported on IBM Power,
and libvirt translate all ide calls directly to scsi
when it's running in power platform

Change-Id: Ibebc96e71a8ba21ae8634e9e22de1cd28f3a7990
Catch CannotResizeDisk exception when resize to zero disk

When old flavor's root_gb is not equal 0 and new flavor's root_gb is 0,
the resize() in nova.compute.api will raise CannotResizeDisk.

Move up the new_instance_type check before using.

Closes-Bug: #1340159

Change-Id: I4abf93530cf919af50a88d6049019fb745547257
libvirt: add support for per-vCPU pinning in guest XML

Expand LibvirtConfigGuestCPUTune to allow setting of a
cpu mask per-vCPU

Blueprint: virt-driver-cpu-pinning
Change-Id: I547be6c7da4e3ebc9759f8310cb6282ac4b2f50c
libvirt: add support for memory backing parameters

Add a libvirt config class for dealing with the XML elements
under the <memoryBacking> section of the guest schema.

Blueprint: virt-driver-large-pages
Change-Id: Icd89508671a8951bed480b901b2d12d44ebb7cab
libvirt: add support for memory tuning in config

Add a libvirt config class for dealing with the XML elements
under the <memtune> element of the guest schema

Blueprint: virt-driver-large-pages
Change-Id: I141fcf4fd95f61f7fc3a5d0209dc2b1319ede941
Add ibmveth model as a supported network driver for KVM

ibmveth (spapr-vlan) is supported by KVM/Qemu on ppc64. This change
allows users to change hw_vif_model='spapr-vlan' property to use
ibmveth driver.

Change-Id: Ibec6abad458c5cb15df57afa05f7f0742e4feb14