Skip none value attributes for ec2 image bdm output.

Fix crash in _format_block_device_mapping when there are None values
for formatting attributes in the mapping.

Change-Id: I0ebdb844d75cdf1580cc88e1ff40fc21aff96f21
Closes-Bug: #1323403
Do not fail cell's instance deletion, if it's missing info_cache

Currently the methods in cell messaging are trying to refresh the
instance. However, in some corner cases info_cache is not being
created for instances in ERROR state. This makes the delete
operation, of such instances, to fail, while it should not.

Handling the InstanceInfoCacheNotFound exception and not
re-raising it, for delete operations.

Closes-Bug: #1316373
Change-Id: I33c33e3ac1180e8293d950d60fb126e325a2c0cf
Security groups: add missing translation

Commit d562012f34eadfe6b68dd5ebe06a2fa565de3b2e added exceptions
that were not translated.

Change-Id: I06dbd32a9bc6d99068e06576e4ea7a8594764db5
Add instance to debug log at compute api

Add instance info to 2 debug logs at compute api layer.

Change-Id: Icbe8e45ace74b8bf0282cf6d6dd1ead792532961
Implement methods to modify volume metadata.

There are four methods in nova/volume/cinder.py which are
NotImplemented. They are as follows:
  1. get_volume_metadata
  2. delete_volume_metadata
  3. update_volume_metadata
  4. get_volume_metadata_value
These methods are required in cases where nova needs to modify
a cinder volume's metadata, e.g. attach and detach time. It also
completes the cinder.API class.

Change-Id: I360076ae28db43e661466f556425947813c9040e
Closes-Bug: #1296164
Remove rescue/unrescue NotImplementedError handle

There are 2 kinds of RPC call from API layer to compute layer,
one is cast and another is call. For cast, the RPC message will
be posted and the API service will not wait for the message to
be processed. So it won't be able to catch the exception raised
in compute layer and catch and handle the exception is useless
and error leading. This patch removes code in API layer for
rescue and unrescue functions.

Change-Id: Iab8070c25a9dee505c0602efad85c41c21c2aa58
Keep Migration status in automatic confirm-resize

function in compute/manager.py _poll_unconfirmed_resizes
will translate the migrate status from finished to error
whenever finds a problem ,Consider the following case

1) _poll_unconfirmed_resizes runs and founds several migrations to be confirmed
2) user deletes the instance, so its task state will be changed to DELETING
3) _poll_unconfirmed_resizes will find the task state is not None,
it will make the migration error status
4) following code in _delete
if instance.vm_state == vm_states.RESIZED:
self._confirm_resize_on_deleting(context, instance)
will fail because migration status already updated

User is able to do the migration operations when they found error
so we don't need to handle this kind of error for them

Change-Id: Ia79dd56c9c1fa8add7dc138d838cbfbac4523ac3
Closes-Bug: #1301696
Add debug log for availability zone filter

Sometimes operator need information why the host doesn't pass
the check of scheduler, this patch adds information for available
zone filter if the host doesn't belong to required available zone
and modify the logic a little to make it less nested.

Change-Id: Icb45182fa58d69e7c5e3a77ea31eddc492ac46ec
Partial-Bug: #1301830
Handle a flavor create failed better

Catch FlavorCreateFailed exceptions which when flavor creation fails.
This type of flavor creation failure is caused by an internal nova problem.
Return an internal server error (500) to the user so they know
to contact their cloud provider.

Closes-Bug: #1292309

Change-Id: I268c84dae290b0170448d789509af781555df9b0
Rollback quota in os_tenant_network

when we call API with 'os_tenant_networks' to delete an existing network
it might fail due to unexpected reason, we should rollback the
quota we reserved otherwise we will face no quota later.

Change-Id: Ic74a7a8df337070f17b1ea6916809fa93880a287
Closes-Bug: 1296489
handle AutoDiskConfigDisabledByImage at API layer

nova compute api layer might raise AutoDiskConfigDisabledByImage,
so API layer need to catch them and report correct info.
Also, create api handle lots of exceptions which inherit from Invalid,
this patch combined them and catch Invalid only.

Change-Id: Id0c89e96637d2f82b5bd204eba2c6291b9a5dd38
libvirt: VM diagnostics (v3 API only)

There is no formal definition for the VM diagnostics. For the
V2 API the diagnostics will be returned as they are today.

This will support backward compatibility with the existing API's.

Part of the blueprint v3-diagnostics

DocImpact

Change-Id: I4128e76622e73a40d25fb50b10b1c1ac710a5cc4
Move logs of restore state to inner logic

Currently when we rebuild instance, we will restore instance state
if it's STOPPED, but we will log it whenever it's STOPPED or not.
This may confusing operator and for ACTIVE/ERROR state, it's useless.
Also, adds instance into log to provide more info.

Change-Id: I0685484ee0b56c250071c8c9322d2ed053449fd3
xenapi: Attach original local disks during rescue

When rescuing an instance, a new VM is created and only the original
root disk is re-attached. Often when a user is rescuing a VM, they expect
to be able to access all of their original disks so they can potentially
salvage data.

This changes the xenapi driver to attach the original local disks
during rescue so the user can rescue all of their data.

DocImpact

Implements: blueprint rescue-attach-all-disks
Change-Id: Iba5cc85cd03d0a60f1858cf16aa31397e163df50
Partial-bug: 1223396
Add standard constants for CPU architectures

Introduce a nova/compute/arch.py module to serve a similar role
to nova/compute/vmmode.py, by providing constants for known
CPU architectures and validation/canonicalization APIs

Related-bug: #1348624
Change-Id: Idd4907b5488dd0832c15783d87d57d77ae654519
VMWare Driver - Ignore datastore in maintenance mode

A datastore can be part of a datastore cluster. Any
datastore which is part of a cluster can be put in maintenance
mode. Instances cannot reside on a datastore if it is in
maintenance mode.
This fix ignores a datastore in the following scenarios
1. Stats update - Ignore a datastore during stats update.
2. Ignore a datastore during instance spawn.

Change-Id: Id0cb3f5fdb4d0bc4884cf5405fadc433d4a2b6ba
Closes-Bug: #1330065
libvirt: remove use of CONF.libvirt.virt_type in vif.py

The vif.py class imports the CONF.libvirt.virt_type parameter
from the main driver.py file. This sets up a circular dependency
which prevents driver.py importing vif.py in the top of the file,
since the CONF.libvirt.virt_type parameter won't have been
registered yet.

It is generally bad design practice to rely on global variables,
which is effectively what the CONF.* parameters are, so change
the VIF driver get_config() API to accept 'virt_type' as a
parameter instead.

Related-bug: #1302796
Change-Id: I023851df96640eeeb54124add8cc5c725b8d13be
libvirt: remove 'vif_driver' config parameter

The 'vif_driver' config parameter was deprecated in the
previous cycle, so can be deleted now.

The two places in the test file which were overriding
the 'vif_driver' config parameter were redundant as
mocking already ensured the real VIF driver module
was not exercised.

The FakeVIFDriver class is thus no longer required, and
the FakeIptablesFirewallDriver class was already not
required, so both are deleted.

DocImpact
Related-bug: #1302796
Change-Id: I75519266ffc65df92efd20badfeaf88ac0d66f8a
libvirt: remove pointless LibvirtBaseVIFDriver class

There is only one single VIF driver class these days, so it is
pointless having a LibvirtBaseVIFDriver base class to inherit
from.

Related-bug: #1302796
Change-Id: Ie132135f9b6cf5337782ee9d1611c05bd56557fe
Revert "libvirt: add version cap tied to gate CI testing"

This seems like it was really a policy change that should have had
more community discussion.  I think it's appropriate to revert this
now and have this as a full community discussion about options here.

This reverts commit 3bfc25a74e9dfbeb664eecf89d959428713eb178.

Change-Id: Ie19b39e99b223862ca86b9a54c5372b56e1f5d74