Downgrade log when attach interface can't find resources

Downgrade the log when a resource (e.g. instance is deleted concurrently)
can't be found, no need to LOG.exception because an exception
is already raised and no need to log again.

Change-Id: I260e89c8deab11bc9707abbbbc7bb9a4f88640e3
Closes-Bug: #1322410
Remove traces of now unused host capabilities from scheduler

Host capabilities updates are no longer used by the scheduler. All host
state is instead fetched from the DB. Some left over fields are still kept
by the host manager but contain no data. This change remove the last
traces of host capabilities from the scheduler.

The baremetal host manager used to rely on capabilities info to
decide which host state class to use, VM or baremetal. This has been
broken since host capabilities were removed (bug 1260265). The logic to
decide on host state class to use is updated to use DB data instead and
a test is added to verify the same.

Change-Id: I350109492c2addf5fdfaadee7cec4784f244e102
Closes-Bug: #1260265
Mask block_device_info auth_password in virt driver debug logs

The block_device_info object can have an auth_password key which is
getting logged at debug level in several virt drivers so we need to
sanitize the message getting logged.

Adds tests to ensure the logged messages are properly sanitized.

Note that bug 1321785 was opened to track the long-term design issues
with storing the password in the block_device_info dict since this can
crop up elsewhere if it's logged.  The immediate fix here is to mask
what's already exposed.

Closes-Bug: #1319943

Change-Id: I0eae07ce3f0f39861eb97ec3dec44895386c7d04
Align internal image API with volume and network

First part of standardizing the nova internal image API with the API
objects for volume and network. This patch creates a nova.image.api.API
object that matches the signature of nova.network.api.API and
nova.volume.cinder.API for CRUD operations on image records. The
existing unit tests were enough to fully cover the new code paths, since
much of this patch simply aligned the internal image API calls and API
object with that of the other submodules.

The next in this patch series will work on cleaning up the remainder of
places in nova that directly call on nova.image.glance methods, as well
as the untouched download/upload handling.

Change-Id: I67e7c9d841e5bad82cdca8f7048eaa57f0cfbb2f
Partially-implements: blueprint standardize-nova-image
Catch missing Glance image attrs with None

Glance is known to drop values from its response if they don't resolve to true.
 Previously in I67b7dd16a94fe60d873c012f6bd246ab24500d5a the None catch was
 removed as this WILL cause a call to glance for missing value and
 optimizations where made to attempt to pre-cache values that can be missing
 in expected ways. However there are still situations that can't be optimized.

Notably this can occur with disk_format, container_format, name, deleted,
 and checksum.

Test adapted from I85c42f8351763da201021a22f5ff0ebd62c6b2db by Mike Perez

Closes-bug #1308058
Co-Author: Mike Perez <thingee@gmail.com>

Change-Id: Icddead6f746d4d3ba652858eaae425acce8c177d
Fix nova image-show with queued image

Currently, the nova image-show command cannot be used with queued images,
because they don't have certain attributes set yet. Nova will raise
AttributeError, which causes a 500 to be returned from the API.

Change-Id: I7d73b891545492e3e247eebdf02507cb630eb250
Closes-bug: #1321186
Inject expected results for IBM Power when testing bus

There's no support for ide in IBM Power, and this test is
failing when it's being running in Power.
This patch inject the expected value in
test_xml_disk_bus_ide_and_virtio and
test_xml_disk_bus_ide when it's tested in Power
platform, so, when architecture is Power, the injected value
replace the ide by scsi.

Change-Id: I2db1ade0e049c3471989b121b9497bc5124ea567
[EC2]Correct the return status of attaching volume

The EC2 API return volume's status as 'detached' in response when
attaching a volume.
It should be 'attaching' according to EC2's API docs, so we should convert
the volume_status or volume attach_status to ec2 volume attachment status.
The situation is similar in detaching volume.

Change-Id: Ifacf0ffcab49394def948d428c4c4c13d08152f0
Closes-bug: #1280572
Raise HTTPInternalServerError when boot_from_volume with cinder down

This patch changes exception type.
Boot_from_volume api raises BlockDeviceMappingIsInvalid exception
when cinder-api is down.
Also it returns response code 400.
It is not exact. Because request is valid.
Exception was changed to raise HTTPInternalServerError and
return a 500 error.

Closes-Bug: #1239952

Change-Id: Iec8839d46e6186fe544b28b0aafc8aa6faad53b7
Ensure live-migrate reverts if server not running

When the check to ensure a VM is running before attempting the
live migrate fails, it raises InstanceNotRunning. It should really be
treated in the same way as MigrationPreCheckError.

Along with fixing this bug, I have ensured the expected exceptions in
migrate_server are in sync with the except clause in _live_migrate.

This goes through to the API, ensuring the errors are not treated as
internal server errors, but are given a slightly more appropriate error
return value.

Currently the API returns 400 for all these kinds of errors. While other
error codes may be more appropriate, this change is concentrating purely
on ensuring we treat all the exceptions in the same way.

Change-Id: Iafe5026ef5a095d76e9864b489571e7bcfeb55f0
Closes-Bug: #1312132
Add helpers to create per-aggregate filters

This patchset introduce a new file of utility methods
to deal with aggregate metadata.
As well it updates related filters to use this new logic.
* AggregateRamFilter
* AggregateCoreFilter,
* AggregateTypeAffinityFilter

Closes-Bug: #1301340
Change-Id: I5e4aebe9e61cfbdc7b9e6b5de0788fc6670eb936
XenAPI: Tolerate multiple coalesces

VHD coalescing might coalesce more than one VDI while waiting
and might coalesce the 'grandparent' before the parent.  Wait
for the parent to be any of the original tree instead of just
the original parent.

Closes bug: 1317792

Change-Id: I82c4db0e8a36c41a86adf4bd32304a4bfdabebbf
Add helper methods to convert disk

Add a _disk_raw_to_qcow2 and _disk_qcow2_raw methods which
convert disk.
Update finish_migration to use these helpers.

Change-Id: If98f7fc04668916fda788afb65116a89463b0020
Related-Bug: #1298975
xenapi: fix live-migrate with volume attached

We need to attach the volume SR on the destination host during
pre-live-migrate, so XenAPI spots the volume is present, and does't try
to block migrate the volume to the destination host, instead it just
connects to the remote volume before resuming the VM on the destination
host.

The fix is done by calling connect_volume instead of attach_volume. The
None passed into instance was causing the lookup of the vm_ref to
(correctly) fail, with:
Failure: ['FIELD_TYPE_ERROR', 'label']

Change-Id: I7eaf90582508b42a202da79072c891c216d137d1
Closes-Bug: #1317208
versions API: ignore request with a body

Update the OS API versions controller so it ignores requests
with a body. Previously an incoming (unauthenticated)
request to the versions API would log a TypeError trace
in the nova-api.log file with the following message:

TypeError: index() got an unexpected keyword argument 'body'

Change-Id: Icc3ccfc537b13627b8d5ba43ae3ba91e34e08c9e
Closes-bug: #1306727
Refactor test to ensure file is closed

Python requires parentheses to call a method. Without parentheses it
will return the method object which doesn't do anything useful without
an lvalue.

Since we're in here, switch it to be a context manager to be shorter
while making sure the file gets closed.

Change-Id: Ib8b15e286c8bbca1f59c530c9200cb7a781c3d6b
Fix for multiple misspelled words

In Nova code, few misspelled words have been found.
The misspellings python library has been used to scan
all nova code and find misspelled words.
This commit fixes all misspelled words found with this library.

Change-Id: I5919dddbbe812d8d32ecd8b249c1312249cd2a66
Closes-Bug: 1323730
Imported Translations from Transifex

Change-Id: I13f0ee4bc4861041e2fcefc748deb05d58a4577d
Merge "Align internal image API with volume and network"
Merge "Catch missing Glance image attrs with None"