Fix security group race condition while creating rule

Previously, it was possible for someone to create a security group rule
where the rule references another security group. To do this nova would
first create the security group rule and then look up the referenced group
(group_id's) name in order to return that via the api. During this time
it's possible for someone to delete this security group rule and the
referenced group before the call returned resulting in a 404 error being
raised. This patch addresses this issue by looking up the group name
first and then creating the security group rule in order to avoid this
from occuring.

Note: this patch also adds a test to cover the case where an invalid group
id was passed though this does not really add any real additional coverage
to the change largely because the code is currently using global stubs. That
said, the previous coverage should hopefully be sufficient

Change-Id: If58ffa5629ba5166f260379ac47922974de31be0
Related-bug: 1262566
Reduce unit test times for glance

The class TestGlanceApiServers did not inherit
test.NoDBTestCase.

Change-Id: Ieafe300d793685f5b146999383695733eba3da19
VMware: cleanup the constructors of the compute drivers

The 'read_only' parameter was never used by the compute drivers.
The 'scheme' parameter should be passed to the base constructor.

Change-Id: I3abd40a50896664ee274f057ded8ec2460679c82
Add debug log for core_filter

Sometimes operator need information why the host doesn't pass
the check of scheduler, this patch adds information for core
filter if virtual cpu is not enough on the host.

Change-Id: Ie05d9184435a4fc4cff1af26812c01a3cc226fff
Partial-Bug: #1301830
Add decorator expected_errors for v3 attach_interfaces

All v3 api should use expected_errors to prevent unexpected exception
raised. This patch add decorator expected_errors for v3 attach_interfaces.

This patch also remove update action. Let the wsgi return HTTPNotFound for
it.

Change-Id: Icefe9f9adfa7ad01bc3ab11dcb7c132702390d04
Inject expected results for IBM Power when testing bus devices

Same as in https://review.openstack.org/#/c/93621 this patch
inject expected values when the test is running on IBM Power
platform, since the ide bus isn't supported on IBM Power,
and libvirt translate all ide calls directly to scsi
when it's running in power platform

Change-Id: Ibebc96e71a8ba21ae8634e9e22de1cd28f3a7990
libvirt: add support for memory tuning in config

Add a libvirt config class for dealing with the XML elements
under the <memtune> element of the guest schema

Blueprint: virt-driver-large-pages
Change-Id: I141fcf4fd95f61f7fc3a5d0209dc2b1319ede941
Add ibmveth model as a supported network driver for KVM

ibmveth (spapr-vlan) is supported by KVM/Qemu on ppc64. This change
allows users to change hw_vif_model='spapr-vlan' property to use
ibmveth driver.

Change-Id: Ibec6abad458c5cb15df57afa05f7f0742e4feb14
VMware: improve log message for attachment of CDROM

Commit 3c59d998c5e6b80549053b01f919ac318cd31b13 config drive
support. The log message makes use of the instance_name which is
not unique. The may be confusing when debugging.

TrivialFix

Change-Id: Iff7f4f938f4980f0c0bf2dc3017ed741453a69f5
Scheduler: throw exception if no configured affinity filter

If the scheduler hint indicates that the scheduling should perform
either anti-affinity or affinity scheduling and the relevant filter
is not configured then a NoValidHost exception will be thrown.

This is valuable if an existing OpenStack installation is running
and these filters are not defined after an upgrade.

Change-Id: I79bb44ad7481b3ff924687a8d6afdd6c715c0b59
Closes-bug: #1302238
libvirt: add tests for _live_snapshot and _swap_volume methods

The _live_snapshot and _swap_volume methods do a non-trivial
sequence of libvirt API calls which should be validated in the
unit tests

Closes-bug: #1346327
Change-Id: I14bfa08f0dbb4eaa7c4f069f70c257ba7605989b
Stop depending on sitepackages libvirt-python

We can install libvirt-python bindings from pypi now because the libvirt
devs are supporting pypi packages and the version of libvirt we are
testing nova master against is new enough to link against that pypi
package.

Making this switch will better isolate nova test environments as all
dependencies can be installed without any site packages leaking into
virtualenvs. It will also make the dependency on libvirt-python a bit
more explicit.

Change-Id: I2890702869f05a02ad5e2ecb419db06433231b36
Only use dhcp if enable_dhcp is set on the network

Also removes an unused call to init_host if there is no network
to setup. The signature of this call was wrong so clearly it was
not being used.

Partially-implements blueprint better-support-for-multiple-networks

Change-Id: I33c86b1500f9a69081b4c6ecbe79151f10f69cea
Use library instead of CLI to cleanup RBD volumes

'rbd list' CLI returns error code when there are no rbd volumes, which
causes problems during live migration of VMs with RBD backed ephemeral
volumes. It's safer to use the library that only raises an exception in
case of a real problem.

The only case where rbd CLI is still justified is import, which is
needed to correctly import sparse image files.

All code related to cleanup of RBD volumes is moved to rbd.py, this
fixes a yo-yo problem with single-use methods scattered across 3
different files, and minimizes impact of this fix on imports in
imagebackend and utils.

Closes-Bug: #1346092
Change-Id: I92cd6b16fbd93b377fe47b15d22efbbf68d02513
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
VMware: Resize operation fails to change disk size

The finish_migration step will resize the cpu and memory to match
the new flavor chosen, but not the disk size. Additional code was
added to extend the disk size using ExtendVirtualDisk_Task.

DocImpact

Change-Id: Iff10443f603d329d01a74a620079f80518b75a50
Closes-bug: #1291741
Deduplicate module listings in devref

devref already contains a list of all modules, so no need to explicitly
include files that just contain lists of modules under the section 'Module
Reference'. Most of the deleted files are very outdated and refer to
non-existent modules anyway.

Leave services for now as it contains some content
that should be preserved. A later patch will clean those files up.

Change-Id: I5dce1b523f3648667fcde49ee51206ae9ccfdec8
Remove api reference section in devref

This previously just linked to api/autoindex which was removed in
I6c93f9aae1994961fdca690742398a0d540d541f (in 2011).

Change-Id: Iba86a5b84743fe6d65b797a2e9237d037c2bf1a8
libvirt: remove 3 unused vif.py methods

In the following commit:

  commit 842b2abfe76dede55b3b61ebaad5a90c356c5ace
  Author: Daniel P. Berrange <berrange@redhat.com>
  Date:   Mon Nov 25 14:00:25 2013 +0000

    Increase min required libvirt to 0.9.11

support for setting up OVS outside of libvirt was dropped,
but 3 of the helper methods were mistakenly not deleted.

Related-bug: #1254727
Change-Id: I53de8703a64d1214ec81b9698e376737a92cb53a
Merge "Use library instead of CLI to cleanup RBD volumes"
Merge "Stop depending on sitepackages libvirt-python"