Refactor UnexpectedTaskStateError for handling of deleting instances

To avoid having to repeatedly unfold UnexpectedTaskStateErrors and
do seperate handling of the 'deleting' task state from other states,
everywhere this exception is handled, introduce a separate exception
UnexpectedDeletingTaskStateError that derives from the above.

This way, most handling of InstanceNotFound to handle instances gone
during some action can be made to also include this new exception.

Change-Id: Ia833c2b2361f32c7e33c84ec14d3af6231119a99
Related-Bug: #1223843
libvirt: pass instance to a log() call in the standard way

If we pass the instance in the correct way, then we get nicely
formatted logs which are easier for admins to use.

Change-Id: I27ab372a3388e2aa02491da07c616807bcbb14ef
Conditionalise automatic enabling of disabled host

The change Ib8d67838ceb73c5b1cdc9498c17b335e9e5bb6f3 introduced
automatic behaviour to the nova-compute hosts, which will now
automatically disable themselves if they lose contact with libvirtd,
and re-endable themselves on a successful reconnection.

However, this tramples over any operational service disablement
that may be in place (on an otherwise working host) for any number
of good reasons.

As suggested in bug #1250049, this is a quick fix which attaches
a prefix to any host disable_reason string, and only re-enables
a host if that string is still present.

Fixes-bug: #1250049
Change-Id: Ia7544a357d1385db5fa92e500cb691f49436cfc9
Move some libvirt specific flags into a group

This is backwards compatible via deprecated groups. This change
moves the flags defined in nova/virt/libvirt/driver.py into a
"libvirt" configuration group.

There are other flags which need to be moved, but this change
is quite large, so I will handle those in follow up changes.

Now that the flags in nova/virt/libvirt/driver.py are in a libvirt
group, we can drop the libvirt_ from in front of many of the flag
names as well.

Progresses blueprint libvirt-opts-group.

DocImpact: the following flags are moved into a "libvirt" flag
group: rescue_image_id, rescue_kernel_id, rescue_ramdisk_id,
libvirt_type, libvirt_uri, libvirt_inject_password,
libvirt_inject_key, libvirt_inject_partition, use_usb_tablet,
live_migration_uri, live_migration_flag, block_migration_flag,
live_migration_bandwidth, snapshot_image_format,
libvirt_vif_driver, libvirt_volume_drivers

The following flags have had the libvirt_ dropped from the start of
their flag name now that they are in a flag group:
    libvirt_type (now virt_type)
    libvirt_uri (now conn_uri)
    libvirt_inject_password
    libvirt_inject_key
    libvirt_inject_partition
    libvirt_vif_driver
    libvirt_volume_drivers
    libvirt_disk_prefix
    libvirt_wait_soft_reboot_seconds
    libvirt_non_blocking
    libvirt_cpu_mode
    libvirt_cpu_model
    libvirt_snapshots_directory

The name for the virt_type flag was influenced by bug 1195361,
which shows confusion as to what the flag is setting (the
virtualization type libvirt uses, not the driver libvirt uses
to achieve that).

Finally, the flag libvirt.nonblocking has been renamed to
libvirt.api_thread_pool, which is more descriptive.

Change-Id: I11e79eb5fd79c459374b9bf04707a4e6d79e993f
Move base_dir_name option to somewhere more central

The VMWare hypervisor driver is using the base_dir_name flag, but
it was defined in the libvirt driver. Given I am moving flags for
libvirt into a libvirt group, this flag needs to move somewhere
more central. It has also been renamed to the more descriptive
image_cache_subdirectory_name.

Progresses blueprint libvirt-opts-group.

DocImpact: the base_dir_name flag has been renamed
image_cache_subdirectory_name, although the old name will continue
to work for the forseeable future.

Change-Id: If016456b9f571ff0ec88621bd17e463faeef95ba
Move flags in libvirt's imagecache to the libvirt group

This change continues moving libvirt specific flags into a libvirt
specific configuration group.

Progresses blueprint libvirt-opts-group.

DocImpact: the following flags move into the libvirt group:
image_info_filename_pattern, remove_unused_base_images,
remove_unused_kernels, remove_unused_resized_minimum_age_seconds,
remove_unused_original_minimum_age_seconds, checksum_base_images,
checksum_interval_seconds.

Change-Id: Iea681c7db7efc9da0525ef8e01b4900ca71f10cf
xenapi: kernel and ramdisk missing after live-migrate

After a live migration is performed with xenserver, the kernel
and ramdisk images are not moved to destination.  This means:
* need to manually add them to the destination host
* need to manually clean them on source host

Adding the images is fairly straightforward by calling
create_kernel_and_ramdisk at post_live_migration_at_destination.
Removing images requires us to cache image ids as they are
taken from VM record which is deleted by the time the post
operation comes along.  We use the migrate_data to cache the
image data for removal.

The unit tests are adjusted to handle new calls for existing
tests.

Fixes bug 1224561

Change-Id: I741398db9ddb6764408878a11cf4c80544bbc901
Fix cells instance_create extra kwarg

When the API cell receives a message to update an instance, it'll try to
create it if it doesn't already exist in order to sync up.  However,
it's calling db.instance_create with an extra kwarg that doesn't exist
and failing.  This patch removes the extra kwarg.

The bug was introduced in this change id:

Ife5be9922db0742c8ee4f970517396be86597cce

Change-Id: I69bcb3dda2cbb566b0d69c5eac005703654ea811
Closes-bug: 1247919
Add missing argument max_size in libvirt driver

max_size argument was added in commit 3cdfe894ab58f7b91bf7fb690fc5bc724e44066f,
but missing  in _create_ephemeral and _create_swap method, which cause
exception raised when caching ephemeral and swap disk files of the instance.

Add test for _create_ephemeral used with max_size parameter
and test _create_swap method

Change-Id: I46032ad32f317c923692b8b3d40cedd0dce06cad
Closes-Bug: 1251152
Closes-Bug: 1250032
Merge "Move some libvirt specific flags into a group"
Merge "Move base_dir_name option to somewhere more central"
Merge "Move flags in libvirt's imagecache to the libvirt group"
Merge "libvirt: pass instance to a log() call in the standard way"
Merge "xenapi: kernel and ramdisk missing after live-migrate"
Merge "Add missing argument max_size in libvirt driver"
Merge "Conditionalise automatic enabling of disabled host"
Merge "Fix cells instance_create extra kwarg"
Merge "Refactor UnexpectedTaskStateError for handling of deleting instances"