libvirt fix resize/migrates with swap or ephemeral

Fixes bug 1191608

The code was not properly guarding against backing files not
existing with QCOW2 images. As such, get_disk_backing_file() would
return None, then the subsequent call to basename() on None would
cause an exception.

This bug was introduced with the fix for bug 1185588 and commit
6e768363.

Change-Id: I4b1237ddf01b9398bfdc141180260136b36cae5a
Preserve network order when using ConfigDrive

Pass network_info to be used by InstanceMetadata instead of fetching
it by API and losing originally requested network order. This is
similar to what has been done for "content" where the original data
is lost after the initial call.

This issue only affects the ConfigDrive code path as the original
network_info is used when using file injection.

HyperV and XenApi are still probably affected by the bug but
unaffected by this fix. The fix is the same for these drivers. Simply
pass a network_info and it will be used instead of fetching the
network info by API.

Change-Id: Ie673b725cb47bf491009db99f6cb1258d46b0a69
Fixes: bug #1156844
Sync models.Console* with migrations.

Blueprint: db-sync-models-with-migrations

Change-Id: I2cceab6484b6d69b3f1aa1492e49d76a765b758d
Make sure periodic cleanup of instances continues on error

If an instance cleanup fails with an exception, no more instances
will be cleaned up. If the failure is permanent, this will block
other instances from being cleaned up for as long as it remains.

Resolves bug 1192712.

Change-Id: Ibdd69817ee0e6d0963637f38a8be3949e6e053ed
Make sure periodic instance reclaims continues on error

If an instance reclaim operation fails with an exception, no more
instances will be reclaimed. If the failure is permanent, this will
block other instances from being reclaimed for as long as it remains.

Resolves bug 1195029.

Change-Id: I0379d16a1f74344ef44c843ce19a505e31e79bfa
Hypervisor uptime fails if service is disabled

Getting the uptime of a hypervisor fails because the db query
used to get the service filters out disabled hosts.

Instead we replace that query with one that always gets the
service if it exists, and a check that the sevice is up to
avoid an rpc call to a non running service.

Fixes bug 1195052

Change-Id: I97bc8a5002bd0f333207403d5b1878649612b762
Port consoles extension API into v3 part1

This changeset only copies the v2 files into the appropriate v3
directories unchanged. The copy as-is will not be loaded by either
the v2 or v3 extension loaders. The second changeset will then
make the changes required for it to work as a v3 extension.

This is being tried in order to make reviewing of extension
porting easier as gerrit will display only what is actually
changed for v3 rather than entirely new files.

Partially implements: bp v3-api-extension-versioning

Change-Id: I60fd601e28ed69c2597cfc8742b2e17e706e2a1f
Port consoles extension API into v3 part2

This patch contains the changes required to adapt the consoles
extension and the corresponding unittest to the v3 framework.

Partially implements bp v3-api-extension-versioning

Change-Id: I1f5aa7e1a7f9e6d959bb16ea6c4c3d2cd17958b9
Ports image_size extension to v3 API

This merges the functionality from the v2 image_size functionality into
the images extension (which is no longer core in v3)

Partially implements blueprint nova-v3-api

Change-Id: I577907e671d18712893afcb5477fbd2aa6031956
Port quota classes extension to v3 API Part 2

Ports the quota classes extensions and the corresponding
unittests to the v3 framework.

Adds quota class id to data returned by update which makes
it consistent with the behaviour of show.

Partially implements blueprint nova-v3-api

Change-Id: I7ca95e332933872ff20d719bf07dd10364510cbe
test_xmlutil.py covers more code in xmlutil.py

95% is covered now.

Change-Id: Icdcc4d423a0fc5ebd2433a3f072108f8d7ecb033
Fixes: bug #828617
Reassign MAC address for vm  when resize_revert

In the case of a resize or migrate on the same host, we need to assign
a temp mac address to the source LPAR so we don't have a conflict when
another LPAR is booted with the same mac address as the original LPAR.
We need reassign the original mac address, replacing the temp mac on
the old instance during revert resize. The new instance has been
destroyed before this method, so remove the precondition to make sure
original mac address be set.

Fixes bug #1196404

Change-Id: I709292a7725f0a21f04f1c3c79f2fafc4437e402
Check system_metadata type in _populate_instance_for_create

The _populate_instance_for_create method is passed a dictionary
of base instance values during create but it wasn't checking to
see if system_metadata was a dict or not which can lead to a
TypeError when booting an instance via the cells scheduler.

Fixes bug 1195549

Change-Id: I60b853eda48726b3bb752b7e3542b0fca74bc926
Xenapi shutdown should return True if vm is shutdown

clean and hard shutdown vm methods return a False if the vm
is currently shutdown. It should return a True, so that its
return value can be used to determine if the vm is shutdown
or not.

Fixes: bug #1195587
Change-Id: I758c43d45b15c3ef231e9eee3ecac620a44db824
Select disk driver for libvirt+Xen according to the Xen version

There are two versions of the blktap [1] driver available under Xen.
blktap1 has been replaced by blktap2 that does not support "tap"
devices but "tap2", therefore we should use "tap2" devices by default.

Fixes bug 1188517
Fixes bug 1084618
DocImpact

Change-Id: I8b46885b0909de7cb51fd2d0f6ba2e9bcbf33cc6
Fix compatibility with older confirm_resize() calls

Commit ff4194bec79067bf77ab25caec1e40ad3709ea07 bumped the RPC
version and format for compute_confirm_resize() without providing
compatibility with older installations. This patch converts it
to a instance-dict style of primitive before sending it, if we're
pinned to an older version.

Change-Id: Ic631b70c005bfd0efe6965ad024890cc2caf947c
Add notifiers to both attach and detach volumes

Implements: notifiers for attach and detach volumes
Fixes: bug #1191072
Change-Id: I84bc4d5eea6bfc9fe73ca09a5bec0e1e4d14ae02
Adding action based authorization for keypairs.

Added support for action-based policy to keypairs extensions.
Added the same for v3.
Fixes LP 1194739
Change-Id: Ie2a6f7b98845e0789f25c6fd0b97e119d5302046
Port images metadata functionality to v3 API Part 1

This changeset only copies the v2 implementation file into the
appropriate v3 directory unchanged. The copy as-is will not be
loaded by either the v2 or v3 extension loaders. The second
changeset will then make the changes required for it to work as a
v3 extension.

This is being done in order to make reviewing of extension porting
easier as gerrit will display only what is actually changed for v3
rather than entirely new files

Note that this functionality will be moved from core to an
extension.  Consensus appears to be that we continue to need this
functionality for v3 because some deployers disable the public
image endpoint due security concerns and whether this will be
fixed in glance for Havana. Deployers can disable this extension
if they do not want it.

Partially implements blueprint nova-v3-api

Change-Id: I900ff0f93bb790f4f7bbac66bb2437906cb905f5
Remove notify_on_any_change option

This option isn't really needed, since the case it defends is already
covered by notify_on_state_change. Let's simplify the configuration a
bit by removing it.

DocImpact

Change-Id: I35071abdea2d2380d79c07e9134e095bda65a13d
Signed-off-by: Julien Danjou <julien@danjou.info>