Policy check for forced_host should be before the instance is created

Move the policy enforcment for force_host and force_node into the
_handle_az() method so that any violation of policy is detected before
the DB entry for the instance is created.

Fixes bug: 1220220

Change-Id: Iaf6df4f8df6cca71fcb860244ccf758d7908b8f1
VMware: Fix volume detach failure

The VMwareVCDriver method "detach_volume" is missing the "encryption"
argument and thus does not conform to the driver API. Adding the missing
argument solves TypeError being thrown when detaching a volume.

Change-Id: Ib785c2b2cf4c25deed272f8e3f0f47abe6139d1c
Closes-Bug: #1226826
Fixes the usage of PowerVMFileTransferFailed class

If it fails in the process of "Calculate file size in multiples of 512
bytes", the exception.PowerVMFileTransferFailed message is illegal.
"KeyError: u'file_path'" is occurs when there is no argument.

Change-Id: I15a7467bbc103f683d82949dc0892210cd63c4b6
Closes-Bug: #1223198
Replace OpenStack LLC with OpenStack Foundation

Change-Id: I5203554989bc7244cff2899d52c5764360bba60c
Fixes-Bug: #1214176
Code change for regex filter matching

Remove REGEXP_LIKE operator for oracle db backend since it's not suitable,
Such as:
Oracle: 'select xxx from xxx where REGEXP_LIKE (column, pattern)'
Mysql: 'select xxx from xxx where column REGEXP pattern'

For the simple pattern matching(LIKE operator) used for unsupported backend db,
add '%' wildcard character on both sides of the value.

Change-Id: I4682d38086a06032bb5650a14b253f6b2b859613
Closes-Bug: #1230102
Add tests for pre_live_migration

Add tests to ensure retrying plug_vifs in pre_live_migration works.

Closes-bug: #1228749
Change-Id: I80d4cdc27836511c55b2555556edd7a1964c7158
Fix type is overwritten when UPDATE cell without type specified

If you not specified the 'cell-type' in UPDATE request,
  the value will be overwritten by the default value, 'child'.
That's not correct, you need to reserve the existing cell-type.

The issue is also involved in the new V3-api.

Fixes bug: 1224461

Change-Id: I5c5043b9e5e0c33995422094284c3f19aac78593
Make Instance.refresh() extra careful about recursive loads

This simply makes sure that the instance copy that we fetch to
refresh an instance can not trigger lazy loads itself due to
a bug in get_by_uuid() or below. This hasn't actually been the
cause of any recent bugs, but this makes it much more careful.

It also makes sure that obj_load_attr() refuses to do its work
if self._context is None, which was previously only being
enforced for remoted calls.

Change-Id: I76f433af34620045b079e0a2c30c60b28a4ce525
Don't wrap Glance exceptions in NovaExceptions

Exception objects from glanceclient are not serializable
in Nova and shouldn't be directly wrapped inside NovaExceptions.
The glanceclient portions of Nova that translate from Glance
to Nova should be creating new corresponding exceptions
independent of the Glance counterparts.

Change-Id: Ifbe992796f8c5738ae350617d41a9c312fa8ab94
xenapi: set vcpu cap to ensure weight is applied

There is a bug in XenServer 6.1 and 6.2 that means the vcpu weight
we set is ignored if the cap is not also set.

Setting the cap to its default of 0, meaning no cap, to ensure the
weight value is applied.

Fixes bug 1213960
Change-Id: I367a9afe188734cf5e95c01fc014b9044003d6e3
Check instance exists or not when evacuate

When evacuate instance to another host, similar to instance create,
nova compute must make sure the instance not yet exist on the host.

Currently there is no check for this, the fix was add the checking
of instance exist or not when evacuate.

Fix bug 1216501

Change-Id: I3ccbd5db5174904e17ffa8f9236ca4e1a09f4dbe
plug_vif raise NotImplementedError instead of pass

The Hyper-V, VMware, Docker and PowerVM drivers raise
NotImplementedError from the plug_vifs and unplug_vifs
methods rather than pass.
The compute manager handles the NotImplementedError
case when initializing the host.

Fixed bug 1218593

Change-Id: Id17a1f691f46816eadfe257d9ac0cc46502318ef
Fix stopping instance in sync_power_states

When an instance is marked as stopped in the DB but is actually running,
the sync_power_states periodic task will try to stop it.  It does this
using the compute API.  The compute API checks the vm state before
proceeding.  The STOPPED state is treated as invalid.  This patch allows
the caller to bypass the state check and makes use of that in this
periodic task.

Change-Id: I447b6dbd1da72ed33067659708e1755bf8c933d0
Closes-bug: #1226965
get_all_flavors uses id as key to be unique

Change I1b0c93b3e12714a5c0798aa87c9e3fe9bc14519a fixed returning
deleted flavors by changing the get_all_flavors dictionary key
away from 'name' to 'flavorid'. However flavorid is also not unique.

This patch fixes get_all_flavors to work properly using the 'id' as
the dictionary key. This allows flavors to be created with flavorid's
that were previously deleted and then have them all returned when
get_all_flavor(inactive=True) is used.

Fixes bug 1215295 properly

Change-Id: I357a804fe9f012143c996d66b9719c4f46f60e86
Ensure image property not set to None in build_request_spec()

This is in preparation for having scheduler.util.build_request_spec()
construct the request_spec in the live migration code. Specifically, this
change is needed to avoid reintroducing bug #1199811.

Change-Id: I0cc6b6ec7be7a0afbe184b208f2e16fd5cd4452c
Related-Bug: #1224014
Make LiveMigrateTask use build_request_spec()

Some filters like TrustedFilter makes use of extra_specs. Currently,
when live-migration uses the scheduler to select a host, it constructs
a request_spec that has no extra_specs in it.

By making use of the existing helper method build_request_spec(), the
request to the scheduler will include extra_specs.

Change-Id: I5bc6c6418653c256a42da7b0a343086ec9863da1
Closes-Bug: #1224014
Add notification for pause/unpause instance

When pause/unpause instance, there are no notifications, this
will cause someone cannot know when pause/unpause operation
would be finished.

All other instance operations do have notifications. We should
also add notification support for those two operations.

Change-Id: I5b8cc33a5992f5f028a3220217dcf7a0288d150d
Closes-Bug: #1043861
Skip test if sqlite3 not installed

The test is attempting to run the sqlite3 binary during its test run.
Since there's no guarantee that it's installed it should catch an
execution error and skip.

Change-Id: I9f4ed1872fcfe3b8393644a77692dbfcce8ecab2
Closes-bug: 1227868
Resource limits check sometimes enforced for forced scheduling

Resource limit checks on a compute manager are based on the
contents of filter_properites['limits'].   This in turn is
populated by the scheduler after the host has been selected by
scheduler.utils.populate_filter_properties.   The limits value
is taken from the host_state, which is maintained as a local data
in the host manager. The host_state limits values are populated
by the various filters, for example the ram filter sets
limits['memory_mb'] based on its configuration.

In the case of scheduling to a forced host the filters are
bypassed, so the contents of the host_state['limits'] value
depends on whether the scheduler has already placed a non-forced
instance onto that host (and thus run the filters to populate the
limits).   If it has then the compute manager will impose the
limits, if it hasn't then the limits will be empty and there will
be no check.

As forcing bypasses the filters and is a privileged operation the
limits should always be cleared in the filter_properties in this
case to avoid the limit check on the compute manager.

Fixed bug: 1231963

Change-Id: Ic7ff5d1fc652a8bdf6a0cf8e6ef063d9f2e7ab16
xenapi: reduce impact of errors during SR.scan

There are some race conditions in SR.scan leading to errors similar to:
* The SR scan failed [opterr=['MAP_DUPLICATE_KEY', 'VDI', 'sm_config',
  '$UUID', 'vhd-blocks']]
* The SR scan failed [opterr=['UUID_INVALID', 'VDI', '$UUID']]

This seems to be happening because an SR.scan is happening when a xenapi
plugin is modifying the SR at the same time.

To reduce the likely-hood of this error, I have added a mutex around
calls to SR.scan to ensure nova-compute never makes more than one call
at once, because XenServer will join any extra SR.scan requests to
listen to the result of any current SR.scan that may be running.

In addition, when and error occurs during an SR scan, I have added a
retry loop, to attempt to stop these races from impacting users.

Fixes bug 1221293
Change-Id: Ib83e6e0f4f7fab2058fe137e32934606c4fe8f7e