maint: remove unused exceptions

* nova/exception (InstanceSystemMetadataNotFound):
Unreferenced since commit 0d417fa73
* nova/exception (CellError): Never referenced.

Change-Id: I30449748fe1156adf7da03aaace565e63bdac144
Allow user and admin lock of an instance

The existing instance lock capability is currently restricted
to being an admin-only action in the default policy. This
introduces additional functionality that allows the lock to be
used by users (to prevent accidental changes to an instance for
example), but also allows the lock to taken by an administrator
with appropriate privileges such that it overrides any lock held
by the user (for example to block actions on a rouge instance).

When unlocking an instance, an additional check is made to see if
the lock is held by user. If it isn't, an additional check is
made against the unlock_override policy.

Additionally added the current instance lock status to the
v3 extended status extension.

Blueprint mandatory-vm-lock
Flags: DocImpact

Change-Id: I5040276c2d9fd42a71c9d8d4529331992abd7538
Convert camelCase attribute naming to camel_case

Convert cases of camelCase attribute naming to camel_case in the
security groups V3 API to make the v3 API as whole consistent in the naming
of attributes

Partially implements blueprint nova-v3-api

Change-Id: I3def0508d62194545a506ce845cfbfa2b87fd6f9
xenapi: retry when plugin killed by signal

Sometimes the glance upload will get killed because of a signal. We
should retry the upload on these types of errors.

It is currently unclear where the signal comes from, so further
investigation is required to determine why the signal gets raised. It
does not appear that any services are restarting at the time the signal
gets raised.

This change has also increased the coverage around the unit tests for
the glance upload, so we now test the try logic.

Fixes bug 1207746
Change-Id: I116be92e153cf27c353cb20168a67b2c0b3cd595
Merge "Allow user and admin lock of an instance"
Retry failed instance file deletes

This change adds a new column to the instances table named "cleaned".
This column indicates that the instance files on disk have been
deleted for that instance, assuming that the virt driver you are
using supports that feature (which only libvirt does for now).

Then the flag instance_delete_interval specifies how often to
retry deletes. The default is five minutes. A value of zero will
disable retries.

The number of attempts to delete instance files is tracked in
system metadata. We give up after maximum_instance_delete_attempts
attempts at cleaning up the files.

This change is done in this way because cleaning will become a more
generic concept in the next patch.

DocImpact: the following new flags are added...

 - instance_delete_interval: how often to try to clean up instance
   files

 - maximum_instance_delete_attempts: how many attempts to have before
   giving up

Resolves bug 1040110.

Change-Id: I895869e18f92ed57bd2f8c1683cf9a398d88b680
Merge "maint: remove unused exceptions"
Avoid extra glance v2 locations call!

In 7df1c911 we added code to attempt to direct download
images from glance locations. This feature is meant to be off
by default however when it is disabled it adds an unnecessary
glance V2 API call.

This patch removes the extra get_locations call when
CONF.allowed_direct_url_schemes is empty (disabled). This effectively
fixes Nova on Python some 2.6 distros which fail to work with the
recent Glance V2 schema code.

Fixes LP Bug #1208656

Change-Id: I0989ed4f248cd1126d4703ad6498cfd66685ae31
Merge "Convert camelCase attribute naming to camel_case"
Merge "xenapi: retry when plugin killed by signal"
Merge "Avoid extra glance v2 locations call!"
Imported Translations from Transifex

Change-Id: Iee5d22a1cdd1491725d755fe918e36fbdeb89d18
Merge "Retry failed instance file deletes"
Make API part of instance boot use new BDM format

This patch makes the API part of the instance boot process rely on the
new BDM format. It adds additional validations that are enabled due to
using the new format.

Since the new format will allow to specify devices that will be created
as images on the hypervisor (if the hypervisor enables it), in order
to prevent a DOS, a new config option 'max_local_block_devices' was
added. This option allows the operator to set a limit on the number of
destination_type='local' block devices a user can specify per VM.

This patch also refactors and simplifies how block devices are handled
during boot, and makes sure that validation errors are caught and
transformed to expected HTTP errors.

Change-Id: I62e0807fcff0a284ff72555d107cac1c9864e46d
blueprint: improve-block-device-handling
Change RPC to use new BDM format for instance boot

This patch makes rpc calls that are the result of booting an instance
propagate the new block device mapping format.

The way this is done is by adding a 'legacy_bdm' flag to the conductor
(task) API. Since both the scheduler and the compute service propagate
the block device mapping as part of the request_spec field, a flag named
legacy_bdm_in_spec to indicate that was introduced in their respective
APIs.

Currently block_device_mapping is not used by any of the in-tree filters
in the scheduler, so it is assumed that scheduler's boot functionality
has now transitioned to the new format.

This patch also bumps RPC versions of the task, scheduler and compute
manager classes.

Finally this patch propagates the legacy_bdm_in_spec through the
run_instance callback of the compute manager so that, in case of error -
the instance is re-scheduled with the proper flag set.

blueprint: improve-block-device-handling

Change-Id: I5f25ddd4d586dda91061f065c1796be726b0ede3
Refactor how BDMs are handled when booting

This patch refactors how block devices are handled when booting VMs.
Also it introduces new block device mapping format in the run_instance
manager method of the compute service.

The refactoring is done by creating a simple hierarchy of
DriverBlockDevice classes that are meant to:

1) Provide a common place to keep information about the data that is
kept for each block device type

2) Encapsulate the logic of "attaching" the device from the compute
service point of view (driver will still need to do the actual work).

Currently we have special subclasses for ephemeral, swap, volume and
snapshot devices.

These classes also make it possible to easily provide
the drivers that will require old block device mapping format with it by
overriding the legacy() method.

Another advantage that this refactoring brings is that adding additional
block device types, like Image block devices (coming soon in a subsequent
patch) that can have destination type of either a hypervisor image or a
volume will be just a matter of extending the base class and redefining
attach to do the work of creating the volume if needed.

blueprint: improve-block-device-handling

Change-Id: I5b9c3e2d959c602fa22f49db681da918ae0adcea
Virt driver flag for different BDM formats

This patch introduces a new virt driver property
'need_legacy_block_device_info' that will tell the compute manager if
driver needs to be passed legacy block_device_info.

This patch also refactors the compute manager methods that would pass
the block device info to driver methods - to do so according to the
property of the driver. This is done by moving all the code into a single
private method _get_instance_volume_block_device_info. This method existed
before but now uses the new DriverBlockDevice classes for it's
functionality.

As it is sometimes necessary to refresh the connection info for volume
based DriverBlockDevice objects (a private manager method
_refresh_block_device_connection_info was used for this), this patch
also factors this code to a method on the DriverVolumeBlockDeviceClass,
and is now completely used like that when needed, and removes the said
method.

Finally - this patch removes a _get_block_device_info method from the
compute API class. This remained from early pre-conductor attempts to
remove DB access from the compute service, and is irrelevant now that
conductor is used. This is needed so that the source  of block device
data is always conductor and the version of data is always
known.

After this patch - it will be possible to implement changes in drivers
that will allow them to utilise the new block device data format.

blueprint: improve-block-device-handling

Change-Id: If961329061e5413824d891eada645dd8dd9cb6dd
_prep_block_device respect use legacy flag

Change-Id: I5921e876c68f18d6c528c980ab91ddb35e8735a0