VMware: add support for VM diagnostics

The drivers now return diagnostics for a requested VM.

Implements blueprint vmware-vm-diagnostics

VMware specific keys will be namespaced.

DocImpact

Change-Id: I08f199f7e3b4edafe60ce73d2fc833f8c5a5574e
handle migration errors

handle errors:
   MigrationPreCheckError, NoValidHost,
   InvalidSharedStorage and InvalidLocalStorage
in _migrate_live so that the 'reason' is passed up

Closes-Bug: 1250523

Change-Id: I3d7e0626223f2606c5e7205e5022e26585506cfb
Use of logging in native thread causes deadlock connecting to libvirtd

When connecting to libvirtd daemon we were calling the _connect
method in a native thread. The use of logging in this method hit this
bug in eventlet https://bitbucket.org/eventlet/eventlet/issue/137/
causing nova-compute to hang.

Closes-Bug: #1252409

Change-Id: I0d22c3ed1b3288aeb1afae130f11a4f38dc6440a
Removes check CONF.dhcp_options_enabled from nova

* Neutron supports extra_dhcp_opts since Havana
* Hence remove boolean check dhcp_options_enabled

Closes-Bug: #1214162
DocImpact
Change-Id: I9af217a87a237e078e70332cbed1c80ad4d5db0a
Fix "in" comparisons with one element tuples

A single value enclosed in parentheses is not a tuple (it needs a comma
after the element) therefore the comparisons like 'if foo in ("bar")'
are wrong.

Closes-Bug: #1248443
Change-Id: I29a2227c9d83efc17ecfacc86d507d75edd4eaca
libvirt: host update disable/enable report HTTP 400

When disable/enable host with host-update command, end user will
get HTTP 400 error even if the host was updated(disabled/enabled)
successfully.

The root cause is that after the host was disabled/enabled, nova
api need depend on host status result to check if the host was
disabled or enabled, but libvirt driver did not return host status
to nova api.

Change-Id: I0688134eecb0628b680c8ed01741f00f80eec53f
Closes-Bug: #1251589
Compact pre-Icehouse database migrations <= 140.

Compacts the pre-Icehouse database migrations <= 140 into a single
migration (140_havana.py).

Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.

Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.

Specific changes include:

1) Add counters to bw_usage_cache table.

2) Add node to instances.

3) Add indexes to instances.

4) Add indexes to migrations.

5) drop server name from instances
 NOTE: for sqlite this change recreated the instances table altogether.

6) add indexes to fixed ips.

7) Drop unused postgresql volume sequences.

Change-Id: I0405c02191a47b96759131bf6b7a1379f4b5711e
Implements: blueprint compact-havana-db-migrations
Add status mapping for shutoff instance when resize

Current code always map instance's status into Shutoff when
getting status of resizing instance with power_state shutoff.
This isn't consistent with active instance, we should add
status mapping for power-off resizing instance.

Closes-Bug: #1247106
Co-Authored-By: chenxiao <chenxiao@cn.ibm.com>
Change-Id: Id4bff71b2c46ec9dccea6d8ac36eba273e48ea5d
Recover from IMAGE-* state on compute manager start-up

If a compute manager is stopped / fails during certain IMAGE-*
operations then the instance will be left stuck in a transitional
task_state

This change handles three possible task states IMAGE_PENDING_UPLOAD,
IMAGE_UPLOADING, and IMAGE_SNAPSHOT.

Both IMAGE_PENDING_UPLOAD and IMAGE_UPLOADING are task states set
after the request has gotten to the compute manager and so we can
clear these safely knowing the operation has ended/failed with the
restart of the compute manager.

With IMAGE-SNAPSHOT it's possible this state was set in the API so
we revert the instance to a task_state of None but the change also
makes provision to set the task_state to IMAGE-SNAPSHOT once the
request comes through to the compute manager so the current
operation is clear once it begins again after the compute manager is
back up and running.

Related to blueprint recover-stuck-state
Change-Id: I665c0ec735968aa0d7553dfc0cfb2373f93451f6
Merge "Use of logging in native thread causes deadlock connecting to libvirtd"
Merge "handle migration errors"
Merge "Recover from IMAGE-* state on compute manager start-up"
Merge "libvirt: host update disable/enable report HTTP 400"
Merge "Add status mapping for shutoff instance when resize"
Merge "Fix "in" comparisons with one element tuples"
network_device_mtu should be IntOpt

Change-Id: Id178cde23779c144c6dfd90c9eb1c4767395f143
Closes-bug: #1256981
Merge "VMware: add support for VM diagnostics"
Merge "Compact pre-Icehouse database migrations <= 140."
Merge "Removes check CONF.dhcp_options_enabled from nova"