Skip none value attributes for ec2 image bdm output.

Fix crash in _format_block_device_mapping when there are None values
for formatting attributes in the mapping.

Change-Id: I0ebdb844d75cdf1580cc88e1ff40fc21aff96f21
Closes-Bug: #1323403
Do not fail cell's instance deletion, if it's missing info_cache

Currently the methods in cell messaging are trying to refresh the
instance. However, in some corner cases info_cache is not being
created for instances in ERROR state. This makes the delete
operation, of such instances, to fail, while it should not.

Handling the InstanceInfoCacheNotFound exception and not
re-raising it, for delete operations.

Closes-Bug: #1316373
Change-Id: I33c33e3ac1180e8293d950d60fb126e325a2c0cf
Make compute api use util.check_string_length

Make some string variables check in compute use util.check_string_length
instead of checking by itself

Change-Id: I9a5d4f647611aa00bed44afdeda5763f8fa9c35b
Prepend '/dev/' to supplied dev names in the API

Make sure that all user input device names end up in the database with
the '/dev/' prefix. This will make things more consistent and avoid
issues when attempting to parse device names.

This includes two main sources - block device mapping that can be passed
as part of a request or as image metadata, and root device name, passed
as image metadata.

Closes-bug: #1337821
Change-Id: Ibda82f511be99f1a68f2f77c72601a1b006be7a0
Security groups: add missing translation

Commit d562012f34eadfe6b68dd5ebe06a2fa565de3b2e added exceptions
that were not translated.

Change-Id: I06dbd32a9bc6d99068e06576e4ea7a8594764db5
Add instance to debug log at compute api

Add instance info to 2 debug logs at compute api layer.

Change-Id: Icbe8e45ace74b8bf0282cf6d6dd1ead792532961
Remove rescue/unrescue NotImplementedError handle

There are 2 kinds of RPC call from API layer to compute layer,
one is cast and another is call. For cast, the RPC message will
be posted and the API service will not wait for the message to
be processed. So it won't be able to catch the exception raised
in compute layer and catch and handle the exception is useless
and error leading. This patch removes code in API layer for
rescue and unrescue functions.

Change-Id: Iab8070c25a9dee505c0602efad85c41c21c2aa58
Adjust audit logs to avoid negative mem/cpu info

we might have following audit info in the log:
AUDIT nova.compute.resource_tracker [-] Free ram (MB): -1559
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 29
AUDIT nova.compute.resource_tracker [-] Free VCPUS: -3

which is really confusing to operater and useless
This patch adjust the log to print phy info and virtual cpus.

DocImpact

Change-Id: I2bc63e1aae0787bf976660eed8fd8899bfffe593
Closes-Bug: #1316916
Add debug log for availability zone filter

Sometimes operator need information why the host doesn't pass
the check of scheduler, this patch adds information for available
zone filter if the host doesn't belong to required available zone
and modify the logic a little to make it less nested.

Change-Id: Icb45182fa58d69e7c5e3a77ea31eddc492ac46ec
Partial-Bug: #1301830
Correct InvalidAggregateAction reason for Xen

Currently the xenapi will report InvalidAggregateAction
when add aggreate under some conditions. But the reason
didn't use the pre-defined reason list.

Change-Id: I2cd65f26097f154f1a4f794916c74cd1730ae142
Rollback quota in os_tenant_network

when we call API with 'os_tenant_networks' to delete an existing network
it might fail due to unexpected reason, we should rollback the
quota we reserved otherwise we will face no quota later.

Change-Id: Ic74a7a8df337070f17b1ea6916809fa93880a287
Closes-Bug: 1296489
handle AutoDiskConfigDisabledByImage at API layer

nova compute api layer might raise AutoDiskConfigDisabledByImage,
so API layer need to catch them and report correct info.
Also, create api handle lots of exceptions which inherit from Invalid,
this patch combined them and catch Invalid only.

Change-Id: Id0c89e96637d2f82b5bd204eba2c6291b9a5dd38
Move logs of restore state to inner logic

Currently when we rebuild instance, we will restore instance state
if it's STOPPED, but we will log it whenever it's STOPPED or not.
This may confusing operator and for ACTIVE/ERROR state, it's useless.
Also, adds instance into log to provide more info.

Change-Id: I0685484ee0b56c250071c8c9322d2ed053449fd3
Use hypervisor hostname for compute trust level

In XenAPI, service hostname and compute node hostname is different
because the Nova compute service may run in a separated VM and is
different with the hostname of the compute node.

The remote attestation service use the compute node's hostname because
it's the compute node that will run the servers.

Closes-Bug: #1223452

Change-Id: I9a7ce74d595531196804615a8947e253b0bd3f1a
Add standard constants for CPU architectures

Introduce a nova/compute/arch.py module to serve a similar role
to nova/compute/vmmode.py, by providing constants for known
CPU architectures and validation/canonicalization APIs

Related-bug: #1348624
Change-Id: Idd4907b5488dd0832c15783d87d57d77ae654519
libvirt: remove use of CONF.libvirt.virt_type in vif.py

The vif.py class imports the CONF.libvirt.virt_type parameter
from the main driver.py file. This sets up a circular dependency
which prevents driver.py importing vif.py in the top of the file,
since the CONF.libvirt.virt_type parameter won't have been
registered yet.

It is generally bad design practice to rely on global variables,
which is effectively what the CONF.* parameters are, so change
the VIF driver get_config() API to accept 'virt_type' as a
parameter instead.

Related-bug: #1302796
Change-Id: I023851df96640eeeb54124add8cc5c725b8d13be
libvirt: remove 'vif_driver' config parameter

The 'vif_driver' config parameter was deprecated in the
previous cycle, so can be deleted now.

The two places in the test file which were overriding
the 'vif_driver' config parameter were redundant as
mocking already ensured the real VIF driver module
was not exercised.

The FakeVIFDriver class is thus no longer required, and
the FakeIptablesFirewallDriver class was already not
required, so both are deleted.

DocImpact
Related-bug: #1302796
Change-Id: I75519266ffc65df92efd20badfeaf88ac0d66f8a
libvirt: remove pointless LibvirtBaseVIFDriver class

There is only one single VIF driver class these days, so it is
pointless having a LibvirtBaseVIFDriver base class to inherit
from.

Related-bug: #1302796
Change-Id: Ie132135f9b6cf5337782ee9d1611c05bd56557fe
Fix attaching config drive issue on Hyper-V when migrate instances

After instance resized or migrated on Hyper-V hypervisor.
The configdrive iso or vhd is copied to resized or migrated
instance, but is not attached to instance.

Because there are configurations for config drive like
config_drive_cdrom, config_drive_format, and the configurations
on different Hyper-V compute node may be different. it will need
to convert configdrive format after resized or migrated.
It is easy to convert from iso9660 or vfat to vhd, but it seems
impossible to convert from vhd to iso9660 or vfat.
So this commit just ignore the target Hyper-V compute node's
config drive configurations, leave the original config drive format.

Change-Id: I349e3b2221fff0ae217a71a91895afd21ff7d18d
Closes-Bug: #1321640
Revert "libvirt: add version cap tied to gate CI testing"

This seems like it was really a policy change that should have had
more community discussion.  I think it's appropriate to revert this
now and have this as a full community discussion about options here.

This reverts commit 3bfc25a74e9dfbeb664eecf89d959428713eb178.

Change-Id: Ie19b39e99b223862ca86b9a54c5372b56e1f5d74