Make API part of instance boot use new BDM format

This patch makes the API part of the instance boot process rely on the
new BDM format. It adds additional validations that are enabled due to
using the new format.

Since the new format will allow to specify devices that will be created
as images on the hypervisor (if the hypervisor enables it), in order
to prevent a DOS, a new config option 'max_local_block_devices' was
added. This option allows the operator to set a limit on the number of
destination_type='local' block devices a user can specify per VM.

This patch also refactors and simplifies how block devices are handled
during boot, and makes sure that validation errors are caught and
transformed to expected HTTP errors.

Change-Id: I62e0807fcff0a284ff72555d107cac1c9864e46d
blueprint: improve-block-device-handling
Change RPC to use new BDM format for instance boot

This patch makes rpc calls that are the result of booting an instance
propagate the new block device mapping format.

The way this is done is by adding a 'legacy_bdm' flag to the conductor
(task) API. Since both the scheduler and the compute service propagate
the block device mapping as part of the request_spec field, a flag named
legacy_bdm_in_spec to indicate that was introduced in their respective
APIs.

Currently block_device_mapping is not used by any of the in-tree filters
in the scheduler, so it is assumed that scheduler's boot functionality
has now transitioned to the new format.

This patch also bumps RPC versions of the task, scheduler and compute
manager classes.

Finally this patch propagates the legacy_bdm_in_spec through the
run_instance callback of the compute manager so that, in case of error -
the instance is re-scheduled with the proper flag set.

blueprint: improve-block-device-handling

Change-Id: I5f25ddd4d586dda91061f065c1796be726b0ede3
Refactor how BDMs are handled when booting

This patch refactors how block devices are handled when booting VMs.
Also it introduces new block device mapping format in the run_instance
manager method of the compute service.

The refactoring is done by creating a simple hierarchy of
DriverBlockDevice classes that are meant to:

1) Provide a common place to keep information about the data that is
kept for each block device type

2) Encapsulate the logic of "attaching" the device from the compute
service point of view (driver will still need to do the actual work).

Currently we have special subclasses for ephemeral, swap, volume and
snapshot devices.

These classes also make it possible to easily provide
the drivers that will require old block device mapping format with it by
overriding the legacy() method.

Another advantage that this refactoring brings is that adding additional
block device types, like Image block devices (coming soon in a subsequent
patch) that can have destination type of either a hypervisor image or a
volume will be just a matter of extending the base class and redefining
attach to do the work of creating the volume if needed.

blueprint: improve-block-device-handling

Change-Id: I5b9c3e2d959c602fa22f49db681da918ae0adcea
Virt driver flag for different BDM formats

This patch introduces a new virt driver property
'need_legacy_block_device_info' that will tell the compute manager if
driver needs to be passed legacy block_device_info.

This patch also refactors the compute manager methods that would pass
the block device info to driver methods - to do so according to the
property of the driver. This is done by moving all the code into a single
private method _get_instance_volume_block_device_info. This method existed
before but now uses the new DriverBlockDevice classes for it's
functionality.

As it is sometimes necessary to refresh the connection info for volume
based DriverBlockDevice objects (a private manager method
_refresh_block_device_connection_info was used for this), this patch
also factors this code to a method on the DriverVolumeBlockDeviceClass,
and is now completely used like that when needed, and removes the said
method.

Finally - this patch removes a _get_block_device_info method from the
compute API class. This remained from early pre-conductor attempts to
remove DB access from the compute service, and is irrelevant now that
conductor is used. This is needed so that the source  of block device
data is always conductor and the version of data is always
known.

After this patch - it will be possible to implement changes in drivers
that will allow them to utilise the new block device data format.

blueprint: improve-block-device-handling

Change-Id: If961329061e5413824d891eada645dd8dd9cb6dd
Allow more than one ephemeral device in the DB

This patch changes the database api behaviour when creating/updating a
block device mapping to allow more than one ephemeral device instead of
overriding the old ones.

Part of blueprint: improve-block-device-handling

Change-Id: I71c283cbd5ad3d28a27d06cb01b5ba9433f8ab46
Allow block devices without device_name

This patch removes the check in the API that prevented having block
devices without 'device_name' and makes the compute manager capable of
guessing those names.

Before creating the instance, the manager will check each one of the
block devices that should be attached and verify that all of them have a
device_name assigned. If any of them doesn't, it will try to guess a
valid name using the method `get_next_device_name', which should be
implemented by the drivers, or by calling the generic method from the
utils module `get_device_name_for_instance'.

It also takes care of assigning the correct device name to the root
device (boot_index=0) to make it agree with the field `root_device_name'
kept in the instance.

Part of blueprint: improve-block-device-handling

Change-Id: I84541f8ff6e1b5978734e5def69946d014c66fdf
libvirt driver capable of generating device names

Implement the method from the base driver class to let libvirt
provide device names if the manager requests them.

This allows users to boot an instance without having to care about
device names and delegating that task to the compute driver.

blueprint: improve-block-device-handling

Change-Id: I2b4f2ea6a4d2e4133e0ba12edabaa754744b8fa4