Accurate exception info in api layer for aggregate

Adjust log information for exceptions in api layer for aggregate
according to
https://wiki.openstack.org/wiki/LoggingStandards#INFO_log_level_is_for_Operators
the LOG.info was removed because nova api already reported error to operator

Change-Id: I89394407495cf2d346e1f532ada236a4364aab24
VMware: deprecate ESX driver from virt configuration

Remove the VMware ESX driver configuration from the virt driver
configuration help options.

The commit d041b95b8977e77a51e6031c38f0011b44d3266a marked the
VMware ESX driver as deprecated. This is the first step in
starting to remove the driver in Juno.

Change-Id: I955ffee336feeca68994d9ea4e244f3c7e10eed2
Fix EC2 not found errors for volumes and snapshots.

According to AWS docs
(http://docs.aws.amazon.com/AWSEC2/latest/APIReference/
api-error-codes.html)

Change-Id: I43af230b975263bb2a14e0cc0bd8718f55bd8a55
Closes-Bug: #1323813
libvirt: virt rescue method now supports objects

Commit a37a96fd13b933123e80ad24e37fc37d73c16059 ensured
that the compute API and manager are now getting objects
for the rescue method.

This patch adds support to the virt driver code.

Change-Id: I2ae89f937b1983a2b0599e2eded343aa8358dbba
libvirt: virt unrescue method now supports objects

Commit 65ad230af5ece020822fa963dd30fd641a54ae91 ensured
that the compute API and manager are now getting objects
for the unrescue method.

This patch adds support to the virt driver code.

Change-Id: I5cca3fa997042b164c21fe2a13c2311f4b18ec06
VMware: spawn refactor - DatastorePath class

Part of spawn refactor work to consolidate a lot of the datastore path
manipulation code. The DatastorePath class is a simpler version of a
class of the same name in earlier versions of
https://review.openstack.org/#/c/87002, and is intentionally limited in
the ways that DatastorePath objects can be constructed. No attempts are
made to ensure that the datastore, directories and files referenced in
these objects are valid on the remote system.

partial blueprint vmware-spawn-refactor

Change-Id: Id65d0308d32835048229e7a3d98b84af2b13583c
VMware: remove dsutil.split_datastore_path

The functionality of split_datatore_path has been subsumed by
the DatastorePath utility class, so remove it and convert
all callers to use the latter.

Change-Id: I07a2cfe15fccaca4aee25bc43157e6c604b91831
VMware: spawn refactor - Datastore class

Introduces the start of a simple Datastore class to encapsulate relevant
data of a datastore.

partial blueprint vmware-spawn-refactor

Change-Id: Ic0ee6cdd47e7cbd66cf456776c61d0b85fb51a0a
Disable rescheduling instance when no retry info

When scheduler_max_attempts=1 means disable rescheduling, compute
won't reschedule instance before. But build_and_run_instance didn't
hornor this behavior. And cause nova reschedule instance infinitely.
This patch fix this problem, make build_and_run_instance won't reschedule
instance without retry info.

Change-Id: I52d6569c1731b5fd5a10f39ff04a40b43d3857a1
Partial-Bug: #1326207
Fix cold-migrate missing retry info after scheduling

If the resize failed, nova-compute will try to rescheduling this resize
task. But the retry info is missed at resize compute task. This patch
add the retry info back.

Change-Id: I1b60704955e2fb218d047394dd20afe7f971c03d
Remove duplicate code in Objects create() function

Remove duplicate code updates.pop('id', None) in create() function.
Because if self.obj_attr_is_set('id'):
            raise exception.ObjectActionError(action='create',
            reason='already created')
already make sure object doesn't have 'id' values, which means that
'id' won't appear in updates.

Change-Id: I582ef8857b32c07ea8ba1eb5fc1e64516110f3ce
Handle service creation race by service workers

This handle a race where a sibling has already created the service
record in the database.  This fix is similar to that of commit
f6c341b4.

Change-Id: I7975fc632eee34390c5d7dc275b52363ad45475e
Closes-bug: 1326901
libvirt: Use VIR_DOMAIN_AFFECT_LIVE for paused instances

When a volume is attached to a paused instance, it does not
appear in the instance's block devices list (i.e. lsblk)
after the instance has resumed. A similar situation
happens when detaching a volume from a paused instance; the
block device continues to exists in the block devices list,
where it should not.  It was found that the volume is only
persisted in the domain's config settings and not its active
settings, since only the VIR_DOMAIN_AFFECT_CONFIG flag was used
on attach and detach volume.  In order to affect the active
settings, the VIR_DOMAIN_AFFECT_LIVE flag has to be added when
attaching and detaching volumes.

Change-Id: I9c90a410a7ecb91f5a4de28acee21fe7da49242c
Closes-Bug: #1242366
Related-Bug: #1299331
Sync oslo lockutils to nova

There is some update for lockutils in oslo, sync from oslo

The commit to be merged:
edd73c9 Merge "Improve help strings"
e446fe5 Merge "Fixed several typos"
fe3389e Improve help strings
f3f14c9 Fixed several typos
0f495ee Emit a log statement when releasing internal lock
f0dd798 Remove rendundant parentheses of cfg help strings
006d9a2 Allow external locks to work with threads
9b73c19 Re-enable file-based locking behavior
edce46c Use Posix IPC in lockutils
ac84a40 Update log translation domains
ee4df7f Merge "Add remove external lock files API in lockutils"
fcf517d Update oslo log messages with translation domains
37a1de8 Move the released file lock to the successful path
3c125e6 Merge "Use threading.ThreadError instead of reraising IOError"
de826f2 Merge "Have the interprocess lock follow lock conventions"
b0d0c33 Add remove external lock files API in lockutils
e71cd1a Merge "Trivial: Make vertical white space after license header consistent"
4f6190a Use threading.ThreadError instead of reraising IOError
195f0b1 Have the interprocess lock follow lock conventions
15cca4b lockutils: move directory creation in lock class
81fe39e lockutils: remove lock_path parameter
14d3669 lockutils: expand add_prefix
dc06d55 lockutils: do not grab the lock in creators
a0948cb lockutils: remove local usage
df8e051 lockutils: split code handling internal/external lock
8b2b0b7 Use hacking import_exceptions for gettextutils._
6d0a6c3 Correct invalid docstrings
6fa29ae Trivial: Make vertical white space after license header consistent
12bcdb7 Remove vim header
79e6bc6 fix lockutils.lock() to make it thread-safe
ace5120 Add main() to lockutils that creates temp dir for locks

Related-bug: #1256306

Change-Id: Iced751f059bd9a893ccefb5f98d63df29f40c622
Catch ProcessExecutionError in revoke_cert

Catch processExecutionError if revoking the
certificate fails.

This change has been abandoned by Chuck Short, so
I am continuing where he left off (mattoliverau).

Continues abandoned change: 17741

Change-Id: I9714ea8cece87256ff5f9a936286c1da3d628af9
Closes-Bug: #883320
Co-Authored-By: chuck.short@canonical.com
Add missing stats info to BaremetalNodeState

This is the first of two patches to prepare baremetal for the removal
of capabilities in Nova HostManager.

BaremetalHostManager used to rely on capabilities info to decide which
host state class to use, VM or baremetal. This has been broken since
host capabilities reporting were removed in Nova, resulting in Nova
HostState class always being used.

BaremetalNodeState lacks stats info needed by some filters during
scheduling. This may result in failures like ComputeCapabilitiesFilter
not returning any hosts. This must be fixed before BaremetalNodeState
can be used again.

Change-Id: I63d992371640b2fd33d35eae72201fd41944f8a4
Partial-Bug: #1260265
Fix BaremetalHostManager node detection logic

This is the second patch to prepare baremetal for the removal of
capabilities in Nova HostManager.

BaremetalHostManager used to rely on capabilities info to decide which
host state class to use, VM or baremetal. This has been broken since
host capabilities reporting were removed in Nova, resulting in Nova
HostState class always being used.

This patch fixes the detection logic in BaremetalHostManager to
correctly choose BaremetalHostState class for use with baremetal nodes.
Tests are added to verify that the correct class is used for both vm
and baremetal nodes.

In reality, the new detection logic will not start to deliver the
correct HostState class until the Nova HostManager change that follows
has landed.

Change-Id: I0c2e6d985fb29f35b823cdf9a83fead2fa45082a
Partial-Bug: #1260265
Remove traces of now unused host capabilities from scheduler

This is take two of this patch. This time after preparing Baremetal
and Ironic code to work with the HostManager changes introduced in
this patch.

Host capabilities updates are no longer used by the scheduler. All host
state is instead fetched from the DB. Some left over fields are still
kept by the host manager but contain no data. This change remove the
last traces of host capabilities from the scheduler.

Baremetal/Ironic host managers used to rely on capabilities info to
decide which host state class to use, VM or baremetal. This has been
broken since host capabilities reporting were removed. This change
works in tandem with prior updates to Baremetal/Ironic node detection
logic to fix this problem.

Important: This must not be approved until the Ironic patch in
https://review.openstack.org/97447 has first landed!

Change-Id: Id5f88c0aa1df9152ddb62904b0544801a8bcfd2a
Closes-Bug: #1260265
XenAPI: disable/enable host will be failed when using XenServer

When disable/enable host with XenServer, the api of set_host_enabled()
in XenServer driver needs to get service info for the target host, but
the rpc call of set_host_enabled() did not transfer target host as
parameter, this will cause the api call failed.

This patch was let the API of set_host_enabled() use CONF.host.

Closes-Bug: #1251943

Change-Id: I2b6427df80ee3188d6f7c4eeb4d2e7be300f0126
Drop support for scheduler 2.x rpc interface

In commit 741661a6695ccc56e5f50c4d85954cd3ed9ebda5, a part of
Icehouse, we added support for the 3.x interface.  Now that Juno
development is open, we can drop support for the old 2.x API.

For more information on this change, see:

    https://wiki.openstack.org/wiki/RpcMajorVersionUpdates

UpgradeImpact

Change-Id: I7bfad51ae46e999b30e003a5a5c7e341461dba02