Skip none value attributes for ec2 image bdm output.

Fix crash in _format_block_device_mapping when there are None values
for formatting attributes in the mapping.

Change-Id: I0ebdb844d75cdf1580cc88e1ff40fc21aff96f21
Closes-Bug: #1323403
Do not fail cell's instance deletion, if it's missing info_cache

Currently the methods in cell messaging are trying to refresh the
instance. However, in some corner cases info_cache is not being
created for instances in ERROR state. This makes the delete
operation, of such instances, to fail, while it should not.

Handling the InstanceInfoCacheNotFound exception and not
re-raising it, for delete operations.

Closes-Bug: #1316373
Change-Id: I33c33e3ac1180e8293d950d60fb126e325a2c0cf
Make compute api use util.check_string_length

Make some string variables check in compute use util.check_string_length
instead of checking by itself

Change-Id: I9a5d4f647611aa00bed44afdeda5763f8fa9c35b
Prepend '/dev/' to supplied dev names in the API

Make sure that all user input device names end up in the database with
the '/dev/' prefix. This will make things more consistent and avoid
issues when attempting to parse device names.

This includes two main sources - block device mapping that can be passed
as part of a request or as image metadata, and root device name, passed
as image metadata.

Closes-bug: #1337821
Change-Id: Ibda82f511be99f1a68f2f77c72601a1b006be7a0
Don't mask out HostState details in WeighedHost

WeighedHost's __repr__ was poking under the HostState hood to use
.host in its own __repr__ but this hides some important details.
Specifically it hides the hypervisor_hostname, which is important for
Ironic as one 'node' can have thousands of hostnames. The other
details such as available RAM and so on are also useful for ops trying
to debug scheduling issues, so rather than add hypervisor_hostname, I
am just delegating to the underlying __repr__.

Change-Id: I79e55c32b3d0768430132275ebe050f38c63bc87
Security groups: add missing translation

Commit d562012f34eadfe6b68dd5ebe06a2fa565de3b2e added exceptions
that were not translated.

Change-Id: I06dbd32a9bc6d99068e06576e4ea7a8594764db5
Add instance to debug log at compute api

Add instance info to 2 debug logs at compute api layer.

Change-Id: Icbe8e45ace74b8bf0282cf6d6dd1ead792532961
Implement methods to modify volume metadata.

There are four methods in nova/volume/cinder.py which are
NotImplemented. They are as follows:
  1. get_volume_metadata
  2. delete_volume_metadata
  3. update_volume_metadata
  4. get_volume_metadata_value
These methods are required in cases where nova needs to modify
a cinder volume's metadata, e.g. attach and detach time. It also
completes the cinder.API class.

Change-Id: I360076ae28db43e661466f556425947813c9040e
Closes-Bug: #1296164
Remove translation for debug message

Remove translation for debug message in neutron api layer.

Change-Id: I5b3ce1115d8f2c46f6c144f21e735a45f338cb32
Remove rescue/unrescue NotImplementedError handle

There are 2 kinds of RPC call from API layer to compute layer,
one is cast and another is call. For cast, the RPC message will
be posted and the API service will not wait for the message to
be processed. So it won't be able to catch the exception raised
in compute layer and catch and handle the exception is useless
and error leading. This patch removes code in API layer for
rescue and unrescue functions.

Change-Id: Iab8070c25a9dee505c0602efad85c41c21c2aa58
Keep Migration status in automatic confirm-resize

function in compute/manager.py _poll_unconfirmed_resizes
will translate the migrate status from finished to error
whenever finds a problem ,Consider the following case

1) _poll_unconfirmed_resizes runs and founds several migrations to be confirmed
2) user deletes the instance, so its task state will be changed to DELETING
3) _poll_unconfirmed_resizes will find the task state is not None,
it will make the migration error status
4) following code in _delete
if instance.vm_state == vm_states.RESIZED:
self._confirm_resize_on_deleting(context, instance)
will fail because migration status already updated

User is able to do the migration operations when they found error
so we don't need to handle this kind of error for them

Change-Id: Ia79dd56c9c1fa8add7dc138d838cbfbac4523ac3
Closes-Bug: #1301696
Adjust audit logs to avoid negative mem/cpu info

we might have following audit info in the log:
AUDIT nova.compute.resource_tracker [-] Free ram (MB): -1559
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 29
AUDIT nova.compute.resource_tracker [-] Free VCPUS: -3

which is really confusing to operater and useless
This patch adjust the log to print phy info and virtual cpus.

DocImpact

Change-Id: I2bc63e1aae0787bf976660eed8fd8899bfffe593
Closes-Bug: #1316916
Add debug log for availability zone filter

Sometimes operator need information why the host doesn't pass
the check of scheduler, this patch adds information for available
zone filter if the host doesn't belong to required available zone
and modify the logic a little to make it less nested.

Change-Id: Icb45182fa58d69e7c5e3a77ea31eddc492ac46ec
Partial-Bug: #1301830
Handle a flavor create failed better

Catch FlavorCreateFailed exceptions which when flavor creation fails.
This type of flavor creation failure is caused by an internal nova problem.
Return an internal server error (500) to the user so they know
to contact their cloud provider.

Closes-Bug: #1292309

Change-Id: I268c84dae290b0170448d789509af781555df9b0
Correct InvalidAggregateAction reason for Xen

Currently the xenapi will report InvalidAggregateAction
when add aggreate under some conditions. But the reason
didn't use the pre-defined reason list.

Change-Id: I2cd65f26097f154f1a4f794916c74cd1730ae142
servers list API support specify multi-status

Currently the service list API allows the user to specify an optional status
value to use as a filter - for example to limit the list to only servers with
a status of Active.

However often the user wants to filter the list by a set of status values,
for example list servers with a status of Active or Error,
which requires two separate API calls.

Allowing the API to accept a list of status values would reduce this to a
single API call.
Allow to specify status value for many times in a request.
For example::
    GET /v2/{tenant_id}/servers?status=ACTIVE&status=ERROR
    GET /v3/servers?status=ACTIVE&status=ERROR
V2 API extension::
    {
        "alias": "os-server-list-multi-status",
        "description": "Allow to filter the
            servers by a set of status values.",
        "links": [],
        "name": "ServerListMultiStatus",
        "namespace": "http://docs.openstack.org/compute/ext/
            os-server-list-multi-status/api/v2",
        "updated": "2014-05-11T00:00:00Z"
    }

DocImpact: Adds os-server-list-multi-status extension.

blueprint servers-list-support-multi-status

Change-Id: Id0109c56070e2f920be0f95738749aa969258bc1
Rollback quota in os_tenant_network

when we call API with 'os_tenant_networks' to delete an existing network
it might fail due to unexpected reason, we should rollback the
quota we reserved otherwise we will face no quota later.

Change-Id: Ic74a7a8df337070f17b1ea6916809fa93880a287
Closes-Bug: 1296489
handle AutoDiskConfigDisabledByImage at API layer

nova compute api layer might raise AutoDiskConfigDisabledByImage,
so API layer need to catch them and report correct info.
Also, create api handle lots of exceptions which inherit from Invalid,
this patch combined them and catch Invalid only.

Change-Id: Id0c89e96637d2f82b5bd204eba2c6291b9a5dd38
libvirt: VM diagnostics (v3 API only)

There is no formal definition for the VM diagnostics. For the
V2 API the diagnostics will be returned as they are today.

This will support backward compatibility with the existing API's.

Part of the blueprint v3-diagnostics

DocImpact

Change-Id: I4128e76622e73a40d25fb50b10b1c1ac710a5cc4
Fix duplicated images in test_block_device_mapping

In test_block_device_mapping, each test can select to specify
image_ref, block_device_mapping or both. The "both" should be
negative tests because nova API gets image_id from either image_ref
or block_device_mapping.
However, there are some "both" tests which are not their purposes.
This patch fixes these tests for their purposes by passing True as
no_image.

These bugs were found through jsonschema validation framework
development, because the schema could detect "both" cases fastly
and these tests could not get their purposes.

Change-Id: I73f1a15d8e08a3eda3190e660338e274358ddd33