VMware Compute Driver Glance improvement

blueprint vmware-compute-driver

Glance change to fix image download to ESX
The current glance code downloads files in a blocking/synchronous
manner, the fix improves it to work with ESX glance file reader
in a non-blocking manner.
This allows ESX glance file reader to read items from the returned
queue.

Change-Id: I05ab7ac00878aeeea3bbcd7546b05398f2630dd3
populate dnsmasq lease db with valid leases

At start dnsmasq retrieves its lease db by calling the nova-dhcpbridge
script with the init action.

Previously we were returning all allocated IPs in the network and
calculating the lease exirpation time based upon instances.updated_at
or instances.created_at.

Now we are only returning entries for IPs that are allocated and leased.
Additionally we set the lease expiration time to be i
now + CONF.dhcp_lease_time

Change-Id: Ie3b4fafd38094dc01ceea592ab5229e581193512
Fixes: bug #1103260
Refactoring/cleanup of compute and db apis

Refactor db call usage and remove some unused code.

Change-Id: I49466f7ab54dd6dbf0fc78bab3d2df44392ba3ee
fix nova-baremetal-manage version printing

version.version_string_with_vcs is no longer present

Change-Id: Iaecf793b82c030f7095de7e42222e28869d1cf33
Removes retry of set_admin_password

* An RPC timeout may occur if an agent is missing and set_admin_password is
invoked. This causes 500 errors in the OpenStack API.
* Implemented a 501 error in API if the password set fails.
* Modified xenapi agent to use NotImplementedError instead of Exception in
  set_admin_password.
* Updated test code around set_admin_password to accept different exceptions.
* Fixes bug 1061045

Change-Id: If7fab56c20f12e0490f4774e00004ed1d94242b9
Use install_venv_common.py from oslo.

This syncs install_venv_common.py from oslo and reworks the
tools/install_venv.py script to use the new library.

Change-Id: I020eae1ec9e3a6aae0ecc7825c0ef723ab247f39
Pass parameter 'filter' back to model layer

This allows the API to pass back additional arguments to filter search
results by.  This will be useful for resource tagging or other filters.

DocImpact: Adds 'Filter' parameter to DescribeInstance (see
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html)

Fixes: bug #922835

Change-Id: I58cc12df650829ed7eb685184a22186d8a5406f4
Signed-off-by: Stephen Gran <stephen.gran@guardian.co.uk>
Add api samples to fping extension

Partially implements bp nova-api-samples
Fix bug 1091747

Change-Id: Ic0060b8ea2a01a304c337a31dc276cbf395c2e5d
Remove deprecation notice in LibvirtBridgeDriver

After further discussions with Quantum team, it was agreed
not to deprecate the existing libvirt VIF drivers until the
Havana release, to allow more time for Quantum plugins to be
updated.

Blueprint: libvirt-vif-driver
Change-Id: I0226c4117630de489543d35662a8ee6b6c7729f9
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Add support for compressing qcow2 snapshots

Fixes bug 1109923

Adds a new configuration option:
    libvirt_snapshot_compression=False (Default)

When set to True and the snapshot output format is qcow2, we pass
the -c option into qemu-img, which enables compression. This patch
also refactors the extract_image function to make construction of
the whole qemu-img command more dynamic.

Change-Id: I7a7ebcf41a91a8a9cb14be79e9bb79c22acb136e
Flags: DocImpact
Code cleanup for rebuild block device mapping

* Removes extra call to get block-device-mappings when detaching during
  recreate.

* Untangle network handling from block-device handling in rebuild

* DRYs up evacuate hosts tests

* Temporary mutation supports dicts as well as objs

* Make instance recreate exceptions more specific to aid testing

Change-Id: I242f9f6a7d329f27b06b7e81b9a418e01682c82e
xenapi: Ax unecessary `block_device_info` params

The xenapi driver uses the 'osvol' other_config to determine which
volumes to detach, making `block_device_info` unecessary for internal
functions.

`destroy` still needs to accept the param since it's part of the
virt-driver API.

Change-Id: I1bbd69f41f9a1e18d4759d0ef3c6de027d3c9981
Use _prep_block_device in rebuild.

This patch makes rebuild slightly more similar to build by calling the
same function to prep block devices.

Change-Id: I3040b49578ad0351be7819c29a449799f50c82a9
DRYing up volume_in_mapping code.

Libvirt, VMWare, and Xen drivers all reimplement this method. This
refactors the code into the `block_device` module so all drivers can
share it.

Future Work: Refactor xen so that strip_prefix doesn't need to be passed
to `volume_in_mapping`.

Change-Id: I026b655f2eb7ea9cebe84058749c05ce93bcb7d4
Fix rebuild with volumes attached

This patch assumes that the correct behavior for instance rebuild is to
maintain attached volumes across a rebuild operation.

Two important changes are:

1) Detaching all volumes during a rebuild so that they won't be 'in-use'
   when prep_block_devices is called to reattach them.

2) xenapi: Allowing additional volumes, not just root volumes, to be
   attached before boot. To handle this, we cycle through all
   block-device-mappings, not just the root-device, create the VDI, and
   later, create the VBD.

Small changes include:

* Using `connection_data` instead of `dev_params` (to match other parts of
  the code base)

* Renaming `get_vdis_for_boot_from_vol` to `get_vdi_uuid_for_volume` to
  reflect its more general and simpler semantics.

Fixes bug 1071547

Change-Id: Ie54a16be4bae2a718ed7d506f32777d0847b9089
Extend extension_authorizer to enable cleaner code.

I have seen and used two different methods for authorizing multiple
methods in a class, and both use code duplication that won't be
necessary with this patch.

The first method involves creating multiple authorizers, which can get
out of hand quickly.  i.e.:

authorize_attach_index = extensions.extension_authorizer('compute',
                                                'volume_attachments:index')
authorize_attach_show = extensions.extension_authorizer('compute',
                                                'volume_attachments:show')

The second method involves creating an intermediate method in the
extension like:

def authorize(context, action_name):
    action = 'os-networksv2:%s' % action_name
    extensions.extension_authorizer('compute', action)(context)

but this ends up getting duplicated in each file.  This patch allows
extensions to use the authorizers they already have but gives them finer
control by passing something like "action='index'" to the authorizer.

Change-Id: I6371b2c631acc9345ee6ca4672afa1f49781f22b
Condense multiple authorizers into a single one.

Reduces code duplication by creating a single authorizer and passing in
the action that should be checked against policy.

Change-Id: I33cfee8695ddfbba39038fbc608c6c74d258745f
Make scheduler modules pass conductor to add_instance_fault

The add_instance_fault_from_exc() method was recently changed to
take a conductor to avoid direct database access. The scheduler was
not updated for this, and thus was not passing it in a couple of
cases.

This makes those calls pass a conductor LocalAPI, resulting in direct
database access (which is desired from the scheduler). The tests that
one might have thought would catch this didn't because they mock out
the method itself. This fixes those and adds two tests that exercise
the add_instance_fault path all the way down to the DB API, which
would have caught it in the first place.

Fixes bug 1110808

Change-Id: If1c2988487d408a39fdf4080541f58f6bdac216c
Merge "Use install_venv_common.py from oslo."
Fixes "is not" usage

Fixes bug #1110973

Change-Id: I68451ce32f3036e43a2bf1f59a02b392a9e40d6d