Fix wrong regex in cleanup_file_locks.

The sentinel filename actually has form hostname-threadid.pid,
not hostname.threadid-pid.
Launchpad bug 1018586.
Change-Id: I09c01e0e63ee704b1485c196dc0b396ee03b2e5c
tests: remove misleading docstrings on libvirt tests

These docstrings take precedence over the method name,
to identify the test in the results.  So just remove
these misleading docstrings as they don't give extra
info over the method name itself.

Change-Id: I4c415f8c4be588f5de751ab8e4d21e85a18f6b75
refactor all uses of the `qemu-img info` command

* nova/virt/images.py: Define a single function that
calls `qemu-img info` in a robust manner, by avoiding
locale issues etc.
* nova/virt/libvirt/utils.py: Define secondary functions
to parse items returned from images.qemu_img_info().
* nova/virt/libvirt/driver.py: Use the libvirt.utils functions.
* nova/virt/disk/api.py: Likewise.
* nova/tests/test_libvirt.py: Remove no longer needed stub
for utils.execute('qemu-img info'). Adjust to a more restrictive
`qemu-img info` format, and more robust calling signature.
* nova/tests/test_imagebackend.py: Adjust to moved method.

Change-Id: I739dd246410c215f401ebd3a92b8207f46e8fb9a
Xen: Validate VHD footer timestamps.

This is a sanity check to ensure the footer timestamps on a VHD are
reasonable (e.g. not in the future). This condition can occur if the
local time for the source and destination machines in a migration are
not in agreement, requiring an adjustment to /etc/localtime and/or NTP
reconfiguration.

Without this check there is a risk of importing a corrupt VHD into the
SR causing the entire SR to become corrupted.

Change-Id: I17228e50d6f54632f3bfc32a682e511f876517ec
update compute_fill_first_cost_fn docstring

The default value for weight of the compute_fill_first_cost_fn is
actually -1.0, which means it does exactly the opposite of what
the function docstring says it does. Update the docstring to
warn people of this fact.

Issue reported via bug #1023827 which was closed as won't fix
but the docs should be clearer. Updated based on feedback.

Change-Id: I56acde254c932a8e66f941aaeb7304665500610f
Return 400 in get_console_output for bad length.

Fixes bug 1027069. Prior to this fix if the length in the
request body was set to a non-integer value a error code of 500
was returned. The documentation also shows an integer in the
example request body.

The fix tests if the optional length is provided and
if it is either an "int" or "long". If not, a 400 error
is returned.

Additionally, cleaned up some error message to be more
descriptive.

Change-Id: I27a66fb1111e74665969f98e1101a9c37c665abc
Xen: Add race-condition troubleshooting script.

This stress-test runs builds and migrations concurrently making it
easier to uncovers certain types of race-conditions in the virt-layer
code.

Change-Id: I89c382b85aa6d0eb3dc957803f3ea34e3a36e9d4
Allow floating IP pools to be deleted

Fixes bug #1022237

If you delete all the floating IPs associated with a floating IP, the
pool still shows up in os-floating-ip-pools extension.

Fix this by using model_query() in the DB layer so that we ignore
deleted floating IPs.

Change-Id: I30d2e716e931400ca4b798b51bb26a73a7747808
libvirt driver: set driver name consistently

Add pick_disk_driver_name to utils, to be able to set driver_name
consistently across volume and imagebackend. This is mostly required for
xen.

Xen requires specifying both the driver name and type attributes when
defining disks.  E.g. the following disk XML snippet

  <disk type='file' device='disk'>
    <driver type='qcow2'/>
    <source file='/tmp/disk0.qcow2'/>
    <target dev='xvda'/>
  </disk>

produces incorrect configuration in the xen tools

  (uname file:/tmp/disk0.qcow2)

Including the driver name fixes this

  <disk type='file' device='disk'>
    <driver name='tap' type='qcow2'/>
    <source file='/tmp/disk0.qcow2'/>
    <target dev='xvda'/>
  </disk>

produces

  (uname tap:qcow2:/tmp/disk0.qcow2)

Change-Id: Id984e0c431b6fc743a39e6e6f3dc3eb289fbb43a
Define cross-driver standardized vm_mode values

Currently the XenAPI driver allows for a 'vm_mode' parameter to
be set against an instance to determine whether it is launched
a paravirt or fullvirt domain. This allows the values 'pv', 'hv'
and 'hvm'.

The libvirt driver also needs to be extended to allow a 'vm_mode'
parameter, and to facilitate deployment of heterogeneous Nova
compute farms, it is desirable to have standardization of the
VM mode values across all drivers.

To address this, the nova.compute.vm_mode module is introduced
which defines a set of constants. The constants provide four
possible vm modes:

 - vm_mode.XEN = 'xen' - for Xen 3.0 paravirt ABI
 - vm_mode.HVM = 'hvm' - for native ABI
 - vm_mode.UML = 'uml' - for User Mode Linux paravirt ABI
 - vm_mode.EXE = 'exe' - for container virt executable ABI

The existing 'pv' value from XenAPI is deprecated, because it
is ambiguous - both Xen and UML are paravirt, and other
paravirt hypervisor ABIs also exist

The existing 'hv' value is also deprecated since it duplicates
the other existing 'hvm' value.

The 'vm_mode.get_from_instance' method will extract the
vm_mode value for an instance. It normalizes the value to
all lower case, translates 'hv' and 'pv' into the new
values for back compat, and validates that the value
matches one of the defined constants

The XenAPI and libvirt drivers are then updated to use the
nova.compute.vm_mode constants instead of bare strings.
The test_xenapi.py test case is updated to set 'vm_mode'
to exercise the new codepaths. A new test_vmmode.py
case is also written to fully exercise the logic

NB, previously the libvirt driver would set 'guest.os_type'
to 'linux' to request Xen paravirt. This is a legacy value,
with libvirt preferring the string 'xen'. So although the
new code sets a different value for Xen paravirt os_type,
the functional result is unchanged.

DocImpact
blueprint hypervisor-code-consolidation
Change-Id: I23efc5dc1528b0d8472d752a8a30f55c85310b21
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Don't log debug auth token when using cinder.

Updates a log messages in the Nova cinderclient function so
it doesn't log the auth token from the request context.

Also properly localizes the log message.

Fixes LP Bug #1029049.

Change-Id: I89120f0e9b7a33c00b9b4b629d61d44d0e9b518f
Fix docstring for SecurityGroupHandlerBase

Change-Id: I645a03cd1a80dc34a7b5a2e91072604e5156d51d
Add missing tempfile import to libvirt driver.

import tempfile in the libvirt driver.  This is needed in
_create_shared_storage_test_file().  I saw this come up on the mailing
list:

    http://lists.openstack.org/pipermail/openstack-dev/2012-July/000192.html

Change-Id: Ib327772d61be0202b257049d8aca3d04819a48f5
Merge "Define cross-driver standardized vm_mode values"
Merge "Add missing tempfile import to libvirt driver."
Inject instance metadata into xenstore

When using Xenserver, inject instance metadata into the xenstore, for
the use of the instance.  Implements blueprint xenstore-metadata.

Change-Id: I88a59f1b783eaaaef6ba5efd8bd448aece2f869c
Add support for snapshots and volume types to netapp driver.

This is a continuation of blueprint netapp-volume-driver.

Change-Id: Ic79a98f3949c815f532e52a9b9d45ad7d8aa99ec
Handle NULL deleted_at in migration 112.

A NULL value in `instance.deleted_at` will cause the migration to break.
In the absence of instance deleted_at timestamp, we should be
conservative and skip cleanup.

Fixes bug 1029129

Change-Id: Ieb978281bc55727fb7db02ddc8c3e0fd6c8814f0
Merge "Allow floating IP pools to be deleted"
Merge "Handle NULL deleted_at in migration 112."