libvirt: use tap for non-blockdevice images on Xen

This patch reverts 35c4962c0b97bae5b8751d316d5822fe22c1ab6a: 'use
file instead of tap for non-blockdevice images on Xen', which breaks
qcow2 disk working on Xen. If the method pick_disk_driver_name()
returns 'file' for qcow2 format, we would get the following disk XML
snippet:

    <disk type="file" device="disk">
      <driver name="file" type="qcow2" cache="none"/>
      <source file="/data/instances/instance-00000081/disk"/>
      <target bus="xen" dev="hda"/>
    </disk>

which produces configuration in the xen tools like:

    "file:/data/instances/instance-00000081/disk,hda,w"

Guest would not boot successfully, as Xen doesn't think of disk format
as qcow2. Instead, 'tap' works with both raw and qcow2 disk.

For LP#1084618, I think it is related with libvirt verion. In my
deployment(libvirt-1.0.1), 'tap' dirver could produce correct
configuration for both raw and qcow2 disk format.

Change-Id: I636b2fed366474e2ae8c3b52403e3085724b24f0
Add rootwrap filters for password injection with localfs

Allow to 'sudo cat' to read passwd and shadow.

bug 1098077

Change-Id: Ic734bd33223df879b5e1f144bb4c85702eb88dfa
Fix rendering of FixedIpNotFoundForNetworkHost

Fixes bug 1104000

With https://review.openstack.org/18319 a small bug was introduced. The
exception FixedIpNotFoundForNetworkHost is expecting network_id
parameter, and the change modified that parameter to network_uuid. This
patch fixes this problem at the raise point.

Change-Id: Idc0fa7fcba9ad37b73183f8811376c6c8adada68
VMware Compute Driver Glance improvement

blueprint vmware-compute-driver

Glance change to fix image download to ESX
The current glance code downloads files in a blocking/synchronous
manner, the fix improves it to work with ESX glance file reader
in a non-blocking manner.
This allows ESX glance file reader to read items from the returned
queue.

Change-Id: I05ab7ac00878aeeea3bbcd7546b05398f2630dd3
Add debug log when call out to glance

Calling out to another service may take a while, so lets log this

Change-Id: I223105ee578ae82230cbc3936c096e148b07e8f1
populate dnsmasq lease db with valid leases

At start dnsmasq retrieves its lease db by calling the nova-dhcpbridge
script with the init action.

Previously we were returning all allocated IPs in the network and
calculating the lease exirpation time based upon instances.updated_at
or instances.created_at.

Now we are only returning entries for IPs that are allocated and leased.
Additionally we set the lease expiration time to be i
now + CONF.dhcp_lease_time

Change-Id: Ie3b4fafd38094dc01ceea592ab5229e581193512
Fixes: bug #1103260
Relax API restrictions around the use of reboot

Fixes bug 1101082 bug 982108

Reboots are the only sledge hammer users have available to resolve
issues with their instances without administrative intervention.
This patch modifies the API policy to allow reboot calls to be made
in many more power and task states.

Change-Id: Ia8702448f6b7b863da40e4d498f2e2ee0a12882e
Don't warn up front about libvirt loading issues in NWFilterFirewall

Use the same technique as in libvirt/driver.py to delay import of
libvirt module

Change-Id: I500ddde4651f0f747c54533a1a308eee5a87f717
Refactoring/cleanup of compute and db apis

Refactor db call usage and remove some unused code.

Change-Id: I49466f7ab54dd6dbf0fc78bab3d2df44392ba3ee
fix nova-baremetal-manage version printing

version.version_string_with_vcs is no longer present

Change-Id: Iaecf793b82c030f7095de7e42222e28869d1cf33
Removes retry of set_admin_password

* An RPC timeout may occur if an agent is missing and set_admin_password is
invoked. This causes 500 errors in the OpenStack API.
* Implemented a 501 error in API if the password set fails.
* Modified xenapi agent to use NotImplementedError instead of Exception in
  set_admin_password.
* Updated test code around set_admin_password to accept different exceptions.
* Fixes bug 1061045

Change-Id: If7fab56c20f12e0490f4774e00004ed1d94242b9
Use install_venv_common.py from oslo.

This syncs install_venv_common.py from oslo and reworks the
tools/install_venv.py script to use the new library.

Change-Id: I020eae1ec9e3a6aae0ecc7825c0ef723ab247f39
Fix missing wrap_db_error for Session.execute() method

We should add wrap_db_error for Session.execute() method, because
Query.update() method will call Session.execute() and it is not inside
Session.flush(), so exceptions from Query.update() wouldn't be wrapped.

Add test for Session.flush() wrapper
Add test for Session.execute() wrapper

Fixes bug 1107890

Change-Id: I96894e502f9f279999b61a1e66469e34a07a022d
Fix boot with image not active

When boot with image which is not active, nova API response
with http status code 500. We should catch the exception and
raise HTTPBadRequest with more precise information.

Fix LP# 1108645

Change-Id: Ibd78f4b96bfdb925b03d7809d4e28afcc7e1ddcf
Pass parameter 'filter' back to model layer

This allows the API to pass back additional arguments to filter search
results by.  This will be useful for resource tagging or other filters.

DocImpact: Adds 'Filter' parameter to DescribeInstance (see
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html)

Fixes: bug #922835

Change-Id: I58cc12df650829ed7eb685184a22186d8a5406f4
Signed-off-by: Stephen Gran <stephen.gran@guardian.co.uk>
Refactor libvirt volume driver classes to reduce duplication

Most of the libvirt volume driver class repeat the same setup
of the vconfig.LibvirtConfigGuestDisk() instance. Create a
more suitable base class to hold this common setup and simplify
all the other subclasses accordingly

Change-Id: I9a5a70ccf88a1e55cdb120c8c6d29e6847774691
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Add option to control where bridges forward

This adds a new config option to control where bridges can forward.
The default behavior is unchanged, but if you specify:

forward_bridge_interface=foo

Then only traffic on the same bridge or going to and from the foo
interface will be allowed. This is useful in vlan mode when a
deployer doesn't want traffic to be routed between vlans. Note
that forward_bridge_interface can be specified multiple times and
should be specified once for each interface that supports floating
ips. i.e.:

forward_bridge_interface=eth0.10
forward_bridge_interface=eth0.20

DocImpact

Change-Id: I82d6b288b60cd6deae64690b55546730b1c2da55
Add api samples to fping extension

Partially implements bp nova-api-samples
Fix bug 1091747

Change-Id: Ic0060b8ea2a01a304c337a31dc276cbf395c2e5d
Remove deprecation notice in LibvirtBridgeDriver

After further discussions with Quantum team, it was agreed
not to deprecate the existing libvirt VIF drivers until the
Havana release, to allow more time for Quantum plugins to be
updated.

Blueprint: libvirt-vif-driver
Change-Id: I0226c4117630de489543d35662a8ee6b6c7729f9
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Merge QuantumLinuxBridgeVIFDriver into LibvirtGenericVIFDriver

The only difference between the Quantum bridge driver and the
new generic VIF driver bridge support is the name of the bridge.
This is now provided via the VIF mapping parameter, so there is
no need for the dedicated Quantum VIF driver for bridging. This
This has the advantage that libvirt firewall filter parameters
are correctly configured.

The QuantumLinuxBridgeVIFDriver class functionality is merged into
LibvirtGenericVIFDriver. For backwards compatibility with
the Folsom release, the existing QuantumLinuxBridgeVIFDriver class
is made to inherit from LibvirtGenericVIFDriver and directly
call the bridge specific setup APIs. This eases migration to
the new VIF impl during the Grizzly deployment lifecycle, with
an expectation that the QuantumLinuxBridgeVIFDriver stub will be
deleted after the Havana release.

Blueprint: libvirt-vif-driver
Change-Id: I19664f20e5137f495513999214f66279292bda58
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>