VMware: create datastore utility functions

The patch moves datastore interfaces from vmops and vm_util
to a separate file - ds_util.

This is part of the blueprint vmware-image-cache-management.

In the past the datastore methods would poll the task
to determine that it had completed. This has beem addressed by
making the driver _poll_task return the task information.

The patch set also solves the following bugs by checking if
the FileAlreadyExistsException exception is raised. This could
be due to another process or thread creating the file.

Closes-bug: 1230047
Closes-bug: 1254128

Change-Id: I788e33dbcb3dedc41831b976137607274b1c02ca
Fix development environment docs for redhat-based systems

This adds the new libffi-devel dependency to the devref for new
development environment creation.

Change-Id: I63e0302cffe33b05722579e7d9c7122f2f257843
Move instance delete to new-world BDM objects

Move all the code paths related to instance deletion to new-world
objects. This includes soft- and hard- deletes, instance init on compute
startup, as well as the reclaim_queued_deletes and
_cleanup_running_deleted_instances  periodic tasks.

It introduces an additional database call in the _shutdown_instance
method as there are still paths that call this method but will not pass
it BDM objects. This will be removed as soon as _reschedule_on_error
(basically meaning instance spawn) code paths are moved to BDM objects.

Part of blueprint: icehouse-objects
Part of blueprint: clean-up-legacy-block-device-mapping

Change-Id: Ic63a3afb77362f0797c546a9505fa4bd7a9b986e
Move detach_volume and remove_vol_connection to BDM objects

This patch moves detach volume code paths to use BDM objects. Since both
detach_volume and remove_volume_connection compute manager methods,
since both of them use the _detach_volume method that needed to be
converted as well.

Part of blueprint: icehouse-objects
Part of blueprint: clean-up-legacy-block-device-mapping

Change-Id: I1e9af9be0ea318f568180ddbd6cd0a25e8bb0bfd
Make swap_volume code path use BDM objects

This patch makes the swap_volume compute manager method use new-world
BDM objects internally. Sine this was the last place that used the
helper method _get_instance_volume_bdm, this method is now removed as
it's functionality is completely supported by the new-world objects
interface.

We also add the instance_uuid parameter to the BlockDeviceMapping
get_by_volume_id method, so that we can narrow the possible race window
when swapping volumes call RPC is delayed for a long time and the cloud
administrator re-assigns the volume. In addition, we clean up calls to
this method throughout the codebase to use key word arguments properly.

Part of blueprint: icehouse-objects
Part of blueprint: clean-up-legacy-block-device-mapping

Change-Id: I2e36180876872e756b55171e4903ec3cdfa1d1aa
Move instance_resize code paths to BDM objects

Make both resize_instance and confirm_resize compute manager methods use
BDM objects. This is achieved by moving the helper method
_terminate_volume_connections to use BDM objects.
This patch also avoids unnecessary calls to conductor as now all helper
methods used allow for passing the already dug up BDM list.

Part of blueprint: icehouse-objects
Part of blueprint: clean-up-legacy-block-device-mapping

Change-Id: Idf370609e23aff45e18a67c76768ba80511c2801
Move _poll_volume_usage periodic task to BDM objects

This patch makes the _poll_volume_usage compute service periodic task
use BDM objects internally. It also moves a single instance query to
objects as a side effect. Also adds a missing test for the manager
helper method _get_host_volume_bdms.

Part of blueprint: icehouse-objects

Change-Id: I085fddfbe5872c0b2b044106c0d2507fd4ee32d8
Merge "Fix development environment docs for redhat-based systems"
Merge "VMware: create datastore utility functions"
Merge "Make swap_volume code path use BDM objects"
Merge "Move instance_resize code paths to BDM objects"
Merge "Move _poll_volume_usage periodic task to BDM objects"