Policy check for forced_host should be before the instance is created

Move the policy enforcment for force_host and force_node into the
_handle_az() method so that any violation of policy is detected before
the DB entry for the instance is created.

Fixes bug: 1220220

Change-Id: Iaf6df4f8df6cca71fcb860244ccf758d7908b8f1
VMware: Fix volume detach failure

The VMwareVCDriver method "detach_volume" is missing the "encryption"
argument and thus does not conform to the driver API. Adding the missing
argument solves TypeError being thrown when detaching a volume.

Change-Id: Ib785c2b2cf4c25deed272f8e3f0f47abe6139d1c
Closes-Bug: #1226826
Add tests for pre_live_migration

Add tests to ensure retrying plug_vifs in pre_live_migration works.

Closes-bug: #1228749
Change-Id: I80d4cdc27836511c55b2555556edd7a1964c7158
Removes pre_live_migration need for Fixed IPs

This commit removes the validation that requires Fixed IP Addresses
for pre_live_migration.  The original comment regarding the
pre_live_migration requiring Fixed IP Addresses alluded to uncertainty
about it not being required.  Workloads should not require a Fixed IP
Address for migration, typically just the storage is required to be
similar.

The commit also updates the associated test case to ensure that no
exception is thrown in the pre_live_migration if a fixed IP Address
is not specified.

Change-Id: I2c4d4dae2cb0f00c67f0f731b5080196564b37da
Closes-Bug: #1226226
Correct lock path for storage-registry-lock

This fixes the lock path for storage-registry-lock to prevent a
potential data loss due to a race-condition scenario.  Since the
storage-registry-lock is used to modify a json blob that is
authoritative as to which nodes are using the storage so that
we don't remove any "in-use" backing files on the shared storage
segment (if in use).

The potential data-loss could occur in limited cases of the race
that the lock should prevent.  This can only occur if the image
cache cleanup is triggerd when a node has been eliminated from
the register_storage_use JSON incorrectly by this race condition.

Most of the time the race simplyresults in a corrupted json blob.

Fixes-bug: 1194696
Change-Id: Ib1a5785646e75dc90bdd2a7e69a697adc4fbb298
Fix console db can't load attribute pool

Lists consoles failed when populate console_type from console
db's pool attribute. The reason was db api console_get_all_by_instance
didn't join with pool. So add param 'columns_to_join' for db api, and
join pool attribute when lists consoles.

Change-Id: I18b44860270a401c55cf02b4334b973fbf357b03
Closes-bug: #1223205
Don't wrap Glance exceptions in NovaExceptions

Exception objects from glanceclient are not serializable
in Nova and shouldn't be directly wrapped inside NovaExceptions.
The glanceclient portions of Nova that translate from Glance
to Nova should be creating new corresponding exceptions
independent of the Glance counterparts.

Change-Id: Ifbe992796f8c5738ae350617d41a9c312fa8ab94
Windows instances require the timezone to be "localtime"

This was relatively simple to fix, assuming that the image used
to boot the windows instance has the os_type set to "windows".
The use of os_type in this manner is consistent with the
existing use in the Xen hypervisor driver.

DocImpact: if you're booting windows instances, then you need to
set the os_type image property in glance to "windows". Otherwise
your instances will have their clock timezone set to UTC, which
has unexpected side effects in windows.

Change-Id: Id6759a290ed25c9add97ac61bb7d165b3fb908b2
Closes-Bug: 1231254
Get image metadata in to_xml for generating xml

The to_xml method of libvirt driver will generate xml file of instance
during hard reboot/spawn/rescue/migrate and other operations, and the
image_meta is used for generating correct xml of instance.
But the accurate image_meta may not be passed into the get_guest_config,
so some configs related to the image_meta will be lost, we need to get the
image metadata if it is not passed to the to_xml method.

Fixes bug 1206809
Change-Id: I6942837905334a652cda64d7dd41d3b575846d93
xenapi: set vcpu cap to ensure weight is applied

There is a bug in XenServer 6.1 and 6.2 that means the vcpu weight
we set is ignored if the cap is not also set.

Setting the cap to its default of 0, meaning no cap, to ensure the
weight value is applied.

Fixes bug 1213960
Change-Id: I367a9afe188734cf5e95c01fc014b9044003d6e3
xenapi: enforce filters after live-migration

Currently and network filters, including security groups, are
lost after a server has been live-migrated.

This partially fixes the issue by ensuring that security groups are
re-applied to the VM once it reached the destination, and been started.

This leaves a small amount of time during the live-migrate where the VM
is not protected. There is a further bug raised to close the rest of
this whole, but this helps keep the VM protected for the majority of the
time.

Fixes bug 1202266

Change-Id: I84fdb6e2a8ee38d75f243aadbe79945af1d6849d
Check instance exists or not when evacuate

When evacuate instance to another host, similar to instance create,
nova compute must make sure the instance not yet exist on the host.

Currently there is no check for this, the fix was add the checking
of instance exist or not when evacuate.

Fix bug 1216501

Change-Id: I3ccbd5db5174904e17ffa8f9236ca4e1a09f4dbe
Remove deprecated task states

Task states STARTING and STOPPING were replaced with POWERING_ON and
POWERING_OFF in Grizzly and can now be removed.

Change-Id: I0a29f9aad5689c6401a0f76561cbe46ae2d8e0fb
Fix stopping instance in sync_power_states

When an instance is marked as stopped in the DB but is actually running,
the sync_power_states periodic task will try to stop it.  It does this
using the compute API.  The compute API checks the vm state before
proceeding.  The STOPPED state is treated as invalid.  This patch allows
the caller to bypass the state check and makes use of that in this
periodic task.

Change-Id: I447b6dbd1da72ed33067659708e1755bf8c933d0
Closes-bug: #1226965
Fix unicode string values missing in previous patch

Commit 61bedc9aa04b0bac21e5c4bd17d6d2c09e2d9e01 lost some code
around the str_or_none() helper in objects/utils.py during a
rebase which caused almost all of our values to not actually
be coerced to unicode. This adds that.

Also, the instance.uuid value can't (and shouldn't) be unicode,
so add a cstring() helper and convert it to use that so that
the above change doesn't try to make uuid be unicode. Also,
make str_value() enforce non-None-ness like cstring() to avoid
bugs where we get 'None' unexpectedly, and add a test for that
whole function, which was missing before.

Change-Id: I941545d38aee10f140610b268fbd9c744f6545f9
VMware: Fix SwitchNotFound error when network exists

In FlatNetworking mode, when a virtual network exists on the vCenter
host we still check for the presence of a physical adapter specified
in vlan_interface. This check is only needed if we are going to
create a new network on the vswitch associated with the interface.

This fix removes the check for the physical adapter and the vswitch
when network already exists.

Fixes: bug #1226238
Change-Id: I1eecf2c339534d8694ec6a8bdfaff5217ba2f8ef
get_all_flavors uses id as key to be unique

Change I1b0c93b3e12714a5c0798aa87c9e3fe9bc14519a fixed returning
deleted flavors by changing the get_all_flavors dictionary key
away from 'name' to 'flavorid'. However flavorid is also not unique.

This patch fixes get_all_flavors to work properly using the 'id' as
the dictionary key. This allows flavors to be created with flavorid's
that were previously deleted and then have them all returned when
get_all_flavor(inactive=True) is used.

Fixes bug 1215295 properly

Change-Id: I357a804fe9f012143c996d66b9719c4f46f60e86
Make sure periodic task sync_power_states continues on error

If an instance sync_power action fails with an exception, no more
instances will be synced. If the failure is permanent, this will block
other instances from having their power states synced for as long as
this keeps happening.

Make sure to catch and log such exceptions and then continue with the
next instance.

Change-Id: Ie3cf5d7f590cb680e712d76b7a32c630e82f362d
Closes-Bug: #1228160
Add notification for pause/unpause instance

When pause/unpause instance, there are no notifications, this
will cause someone cannot know when pause/unpause operation
would be finished.

All other instance operations do have notifications. We should
also add notification support for those two operations.

Change-Id: I5b8cc33a5992f5f028a3220217dcf7a0288d150d
Closes-Bug: #1043861
xenapi: reduce impact of errors during SR.scan

There are some race conditions in SR.scan leading to errors similar to:
* The SR scan failed [opterr=['MAP_DUPLICATE_KEY', 'VDI', 'sm_config',
  '$UUID', 'vhd-blocks']]
* The SR scan failed [opterr=['UUID_INVALID', 'VDI', '$UUID']]

This seems to be happening because an SR.scan is happening when a xenapi
plugin is modifying the SR at the same time.

To reduce the likely-hood of this error, I have added a mutex around
calls to SR.scan to ensure nova-compute never makes more than one call
at once, because XenServer will join any extra SR.scan requests to
listen to the result of any current SR.scan that may be running.

In addition, when and error occurs during an SR scan, I have added a
retry loop, to attempt to stop these races from impacting users.

Fixes bug 1221293
Change-Id: Ib83e6e0f4f7fab2058fe137e32934606c4fe8f7e