libvirt: Make sure volumes are well detected during block migration

Current implementation of live migration in libvirt incorrectly includes
block devices on shared storage (e.g., NFS) when computing destination
storage requirements. Since these volumes are already on shared storage
they do not need to be migrated. As a result, migration fails if the
amount of free space on the shared drive is less than the size of the
volume to be migrated. The problem is addressed by adding a
block_device_info parameter to check_can_live_migrate_source() to allow
volumes to be filtered correctly when computing migration space
requirements.

This only fixes the issue on libvirt: it is unclear whether other
implementations suffer from the same issue.

Thanks to Florent Flament for spotting and fixing an issue while trying out
this patch.

Co-Authored-By: Florent Flament <florent.flament@cloudwatt.com>
Change-Id: Iac7d2cd2a70800fd89864463ca45c030c47411b0
Closes-Bug: #1356552
Fix image metadata returned for volumes

When creating a volume from a glance image, cinder stores the original
image metadata in volume_glance_metadata. This is a key/value store,
and all the values are strings.

When Nova boots an instance from a volume, it passes the image
metadata returned by cinder, which is all strings. If a driver
expects these values to be ints, as they are when booting from an
image, it will get a type error.

This change also pulls size from the volume directly rather than
taking the value from the stored image metadata. This is because the
volume will have been created in 1Gb increments, and is unlikely to be
the same size as the original image. It may also have been
subsequently extended.

Change-Id: I7928f6be1ca99f1502941b9df2b443f2ca63a37b
Closes-Bug: #1367540
Fix wrong exception return in fixed_ips v2 extention

Currently, fixed_ips api raises HTTPNotFound if passing an invalid
ip address in url, I think this is not correct. It's better to return
HTTPBadRequest to end user when the api excepts a FixedIpInvalid
exception.

This patch fixes it and also the test cases.

Change-Id: I7b18b76745f9092456d511ad023a9ed3a24cca84
Add 'zvm' to the list of known hypervisor types

zvm is a hypervisor that used for IBM s390 arch and has
many years history on virtualization.
you can refer to http://www.vm.ibm.com/ for more info on it.

According to danpb's comments on
https://review.openstack.org/#/c/122200/:
FYI the list of hvtype constants is intended to define a global set of
all theoretically supported hypervisors. ie it is fine to add more
constants for hypervisors, even if Nova doesn't have a driver in-tree for it.

even if zvm is not in the nova tree now, we are trying to enhance the
code and other related stuffs such as CI test to make it acceptable to
community later. Also, zvm driver code is not in stackforge now since
we are trying to fit other criterias in order to make community accept it and
put into nova tree.

This patch simply adds zvm to the list of known hypervisor types which are
validated during scheduling given commit a4fd236.

Change-Id: I1f5b5bd2a8f082d2e945560ae82593dfd70f2a22
VMware: get_all_cluster_refs_by_name default to {}

Currently get_all_cluster_refs_by_name returns None when no cluster
is found.
driver.py is using the keys() method without checking whether the
object returned is None. This can potentially end up throwing an
AttributeError.

This patch fixes the issue by returning an empty dictionary when no
cluster is found.

Change-Id: I3293c916d79aaa4de8913e8ab9c609c9c0bb023e
Closes-Bug: #1372672
Rename cli variable in ironic driver

Code in ironic driver used "icli" as variable name to reference to
ironic client instance. It has been renamed to "ironicclient" to make
the code clearer.

Change-Id: Ic9e314e882f5e5e305a5ea84f7c2f102f91140dd
Closes-Bug: #1365228
VMware: mark virtual machines as 'belonging' to OpenStack

This patch registers an extension in vCenter that indicates that the
VM's that are run from OpenStack will belong to OpenStack.

If an administrator tries to update the VM via vCenter then they will
get the following message:

"Solution OpenStack manages the selected virtual machine. You should
not modify the virtual machine directly. Use the management console of
the solution to make changes. Do you want to proceed?"

The extension will be called 'org.openstack.compute'. This will make use
of a type called 'instance'.

Closes-bug: #1366506

Change-Id: I1046576c448704841ae8e1800b8390e947b0d457
Switch to nova's jsonutils in oslo.serialization

Very simple import change in each file, just touches a
whole lot of files. jsonutils has graduated into a stand alone
library and has been removed from the oslo-incubator repository.
We should be using the library for all projects.

Change-Id: I09d780a47d377871f357654603ee4b0fa0169d66
Merge "Switch to nova's jsonutils in oslo.serialization"
Merge "Rename cli variable in ironic driver"
Merge "VMware: mark virtual machines as 'belonging' to OpenStack"
Remove unused code in pci_manager.get_instance_pci_devs()

Change-Id: Iec4aa59099a04b684358b44a06290584cc564507
Closes-Bug: 1361797
Remove unused modules copied from oslo-incubator

No longer needed since we moved to oslo.utils and
oslo.serialization and updated to latest oslo-incubator.
As part of the cleanup removed the hack for gettextutils
since the oslo-incubator code no longer uses it

Change-Id: Ib020cd4c3464f1c994a6d77f4b894f41b99e6d02
Merge "Fix wrong exception return in fixed_ips v2 extention"
Merge "Add 'zvm' to the list of known hypervisor types"
Merge "Remove unused code in pci_manager.get_instance_pci_devs()"
Merge "VMware: get_all_cluster_refs_by_name default to {}"
Merge "Fix image metadata returned for volumes"
Merge "Remove unused modules copied from oslo-incubator"
Merge "libvirt: Make sure volumes are well detected during block migration"