libvirt driver: set os_type to support xen hvm/pv

Generate libvirt config for xen hvm and pv guests based on instance
vm_mode. The vm_mode can be set via the vm_mode image property.

Use hvmloader for vm_mode.HVM guests. libvirt_type xen still defaults to
vm_mode.XEN (pv), to not change any behavior.

Change-Id: I5f92bedf6588ab2674d784772a5ecdb895fabe40
Fix HACKING violation in nova/api/openstack/volume/types.py

Change-Id: Ibd28e8de19eb6dc696ab67d2f58b4a14eb9f034f
xenapi: Use instance uuid when calling DB API

The preferred form of calling the DB API is to use the instance UUID
instead of the older instance id. Update that and use the dict form
to be more compatible with blueprint no-db-messaging.

Change-Id: Ib1a06b5385ec627f3daad5bafb3fa1be956a64e2
Add persistent volumes for tgtd.

Currently if you restart the server running nova-volume
or restart tgt, you will loose your iscsi targets that
have been created. This is not good.

In order for iscsi targets to be persistent across
reboots or restarts, one has to have the target's configuration
information in /etc/tgt/targets.conf or /etc/tgt/conf.d.
So when tgtd is restarted then the iscsi targets will be there
as expected.

This patch will add a configuration file to $state_path/volumes
when the volume is created. The configuration file is identified by
the volume uuid. It creates a logicalunit when the volume is created
as well. The iscsi target and configuration file
will be removed once the volume has been removed as well.

In order to use this, you have to include the following in
your /etc/tgt/targets.conf

include $state_path/volumes/*

For upgrades, it will just re-create the volumes
already in the volumes table.

Fixes LP: #1011159

Change-Id: I38fc096ab881ccb52cb688ae46d9d36b0a7b3a45
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Return location header on volume creation

Partially addresses LP 1026600

Normal RESTful idiom would dictate that the "Location" header is set
to reference the newly created resource.

We defer lining up the status code returned with generally accepted
RESTful convention (currently "200 OK", should be "201 Created" or
"202 Accepted") until the next API major version bump.

There is already substantial code duplication between the volumes API
and the os-volumes API extension - this will be addressed in a later
patch.

Change-Id: I166846a4cfea8adc8c156b8ae0e0d288681ac08c
Correct host count in instance_usage_audit_log extension.

This fixes bug 1030106. Basically we were not counting disabled hosts
in the total host count. Also I have refactored the get_audit_task_logs
method, it's really part of the extension (only the extension uses it),
so I've moved it out of nova.compute.utils into the extension itself.

(the tests that exercise it already lived in the extension's tests anyway.)

Change-Id: Iaf42d887b824ba0cbf6ab0ed143a4c01b37b0cb1
Merge "Fix HACKING violation in nova/api/openstack/volume/types.py"
Merge "xenapi: Use instance uuid when calling DB API"
Allow _destroy_vdis if a mapping has no VDI

In _destroy_vdis() in the xenapi vmops.py, a list of VDIs to not
destroy is collected.  If one of the block device mappings does
not have a corresponding VDI, however, _destroy_vdis() fails
(KeyError('vdi_uuid') is raised).  This interferes with
instance deletion.  The problem is corrected by skipping mappings
for which the connection_info data does not contain 'vdi_uuid'.

Fixes bug 1030143.

Change-Id: I5125d557ab799fcf572f665523cd07a893ad3b90
Explicitly send primitive instances via rpc.

This fixes a few bugs in recent no-db-messaging commits.  There were a
few places that I accidentally still used "instance" where I should have
used "instance_p", which is the instance converted to primitive types.

I noticed the problem looking at smokestack logs.  I saw a Qpid
traceback that indicated the problem.  As an interesting point, systems
using Kombu would not have seen this problem.  Kombu uses anyjson.  Our
jsonutils module sets anyjson to go through jsonutils, which will
automatically to jsonutils.to_primitive() if needed to serialize an
object.  Qpid doesn't do json serialization, so it will blow up if it
hits a non-primitive type.  Yay for testing multiple different
configurations to find problems.

Part of blueprint no-db-messaging.

Change-Id: I3487195110572b7ae9586cfcc618ab7ab28d5017
Merge "Return location header on volume creation"
Merge "libvirt driver: set os_type to support xen hvm/pv"
Merge "Explicitly send primitive instances via rpc."
Fix state logic for auto-confirm resizes

In the compute manager, the _poll_unconfirmed_resizes() periodic
task mishandled state checks, and would call confirm_resize() in
invalid states.  This patch fixes the logic so that confirm_resize()
is only called if the instance is in vm_state 'resized' and
task_state None.  The unit test is also fixed for the proper logic.

Fixes bug 1030175.

Change-Id: I66028b7042d88fb0f1f41df07f42b4b02cf60247
Merge "Add persistent volumes for tgtd."
Merge "Fix state logic for auto-confirm resizes"
Support external gateways in VLAN mode.

Add network label when starting dnsmasq. Fixes bug 1012443.

To allow for per-network dnsmasq customization we can use the network
label from the database as the dnsmasq network tag. A user can then
specify a custom dnsmasq config file using the 'dnsmasq_config_file'
configuration flag.

An example dnsmasq config file showing how to set a custom external
gateway or ntp server for particular networks based on their tag/label
follows:

dhcp-option=tag:'private',option:router,10.0.0.254
dhcp-option=tag:'private',option:ntp-server,10.0.0.253
dhcp-option=tag:'private',option:dns-server,8.8.8.8

dhcp-option=tag:'blue',option:router,10.10.0.254
dhcp-option=tag:'blue',option:ntp-server,10.10.0.253
dhcp-option=tag:'blue',option:dns-server,8.8.4.4

Change-Id: I56cb175a968e33b803dfcb965f90f3a5ca4983ca
Check for selinux before setting up selinux.

Check to see if /etc/selinux exists before trying
to setup ssh keys for selinux. If it doesnt exist
it will result in a traceback when setting up the image
since the directory doesnt exist on Ubuntu.

Fixes LP: #1030609

Change-Id: I8af1e6232bc5a84cbb878c2663ab30eb4139484c
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Merge "Support external gateways in VLAN mode."
Send a full instance in rebuild_instance.

Change the rebuild_instance method of the compute
rpc API to take a full instance over rpc instead of just
the instance UUID.  This cuts down on database access needed
by nova-compute.

Part of blueprint no-db-messaging.

Change-Id: I21647a5a5cbf71a2c498f6fee2e4adbe30b8f2ea