VMware: add support for VM diagnostics

The drivers now return diagnostics for a requested VM.

Implements blueprint vmware-vm-diagnostics

VMware specific keys will be namespaced.

DocImpact

Change-Id: I08f199f7e3b4edafe60ce73d2fc833f8c5a5574e
Removes check CONF.dhcp_options_enabled from nova

* Neutron supports extra_dhcp_opts since Havana
* Hence remove boolean check dhcp_options_enabled

Closes-Bug: #1214162
DocImpact
Change-Id: I9af217a87a237e078e70332cbed1c80ad4d5db0a
Compact pre-Icehouse database migrations <= 140.

Compacts the pre-Icehouse database migrations <= 140 into a single
migration (140_havana.py).

Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.

Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.

Specific changes include:

1) Add counters to bw_usage_cache table.

2) Add node to instances.

3) Add indexes to instances.

4) Add indexes to migrations.

5) drop server name from instances
 NOTE: for sqlite this change recreated the instances table altogether.

6) add indexes to fixed ips.

7) Drop unused postgresql volume sequences.

Change-Id: I0405c02191a47b96759131bf6b7a1379f4b5711e
Implements: blueprint compact-havana-db-migrations
network_device_mtu should be IntOpt

Change-Id: Id178cde23779c144c6dfd90c9eb1c4767395f143
Closes-bug: #1256981
Merge "VMware: add support for VM diagnostics"
Merge "Compact pre-Icehouse database migrations <= 140."
Merge "Removes check CONF.dhcp_options_enabled from nova"
XenAPI: Speedup host_ref cannot change - get it once

The host we run on is fixed - getting this each time from XAPI
is expensive

Change-Id: Ifee9a778f3ed855ef52b001de3fc2ebbd0b972cf
XenAPI: get_info was very expensive

Getting the whole VM record is expensive and slowed down Tempest tests
significantly.  Replaced with getting each required item.

When getting power state, only need that specific value, so reduce the
calls to get_info too.

Change-Id: I93bcef5b692a531aac41518dc29d5d4f46cded65
XenAPI: Tidy calls to get_all_ref_and_rec

get_all_records is more efficient than getting them individually in the
old get_all_ref_and_rec implementation.

Replaced some cases of get_all_records with a faster and more
specific API call.

Change-Id: I4725c3f1fdebdf1737941cb7db69b19afb22df9f
XenAPI: Speedup get_vhd_parent_uuid

In many cases the VDI record has already been retrieved.
Use it rather than expensively getting a new VDI record from XAPI.

Change-Id: I469e2a415c1f1924010f47d3927ca18c45a7b8f8
XenAPI: Use get_VALUE in preference to get_record()['VALUE']

Replaces calls to get_record where only a single item is retrieved
from the record with the specific get_VALUE call.

Change-Id: I52e20c4f4c9d6a6ad019694916ee057cc96b18a6
XenAPI: Synchronize on all VBD plug/unplug per VM

Dynamically adding or removing a device needs to be synchronized
to work around a bug in XenServer 6.1 and 6.2 where simultaneous
events can be partially ignored, leading to a failure to plug or
unplug.

Change I041365ed863a1a1c55e88025dff42d01c35494a2 attempted to
work around the issue by retrying - however that does not catch
all failures.  This change should eliminate some of the known
INTERNAL_ERROR failures, but not all, so I041 should not be
reverted.

Change Ie7a64c481a9e3874df9d5924fbb050ac7ab117db does the same
as this, but only for VBD.plug.

Change-Id: If78d4e5f1dd84f1d4a588f25b8f887a7b125bbea
Closes-bug: 1217972