_translate_from_glance() can cause an unnecessary HTTP request

After returning from a get() call to python-glanceclient, nova runs a
translation function on the returned Image to get the data it wants. Part of
this process is checking for an expected set of attributes, one of which is
the deletion time ('deleted_at'). However, if the image has not been deleted,
deleted_at key will not exist. This forces another call to glance to occur for
the same image. A similar problem exists for the checksum attribute, which does
not exist before an image is active. The fix here is to only consider
deleted_at and checksum if they are expected to be present.

Change-Id: I67b7dd16a94fe60d873c012f6bd246ab24500d5a
Closes-Bug: #1275173
(cherry picked from commit 8fcefef5c6db3bb7f182df62de9c5bf986f97303)
Fix nova image-show with queued image

Currently, the nova image-show command cannot be used with queued images,
because they don't have certain attributes set yet. Nova will raise
AttributeError, which causes a 500 to be returned from the API.

Change-Id: I7d73b891545492e3e247eebdf02507cb630eb250
Closes-bug: #1321186
(cherry picked from commit 4c5dea6a5103a141b0ca13dcd57b32b3261180d1)
libvirt: return the correct instance path while cleanup_resize

If we resized a pre-grizzly instance with grizzly or later nova
to another host, while the resize confirmation process,
_cleanup_resize will find the instance resize backup dir and
delete it, but a wrong xxx_resize dir like ${uuid}_resize,
instead of the correct ${name}_resize will be found.
This is because the instance is a resized one which original
instance dir exists on another host(the dest host),
get_instance_path method could not find the original instance
dir on the source host, so the path with uuid will be returned,
and the `target` existing check in _cleanup_resize is failed,
then the ${name}_resize dir will never be deleted.

Closes-bug: #1290294
Change-Id: I904b6751dec740e001f5ec29f637ef456528746f
(cherry picked from b4964eb6a570e290545f95d45411dc8441985cd5)
VM in rescue state must have a restricted set of actions

Right now it is possible to pause, suspend and stop a VM in state RESCUED,
so after the state is changed, it's not possible to trigger unrescue anymore
since the original state is lost.

This patch remove vm_states.RESCUED as valid state from stop,
pause and suspend actions.

The vm_states devref is also updated to reflect this change including the
current reboot flow.( vm_states.RESCUED cannot be rebooted as per
today code)

This is a backport of 8ff170dc95bf3101fe38a2624e941bfa3b7c1138 .

DocImpact
Closes-Bug: #1319182
Co-Authored-By: Cyril Roelandt <cyril.roelandt@enovance.com>
Change-Id: I531dea5a5499bf93c24bea37850d562134dee281
Fixes Hyper-V boot from volume root device issue

Checks that instances are booting from volume by checking
the correct root device obtained from the block device info.

Change-Id: I94941dfe96c1596c8e9b91df3d3d19d33ae7fe92
Co-Authored-By: Zsolt Dudas <zdudas@cloudbasesolutions.com>
Closes-Bug: #1357972
(cherry picked from commit ea19fb10c5e09ff5df383607654ab9dc2791ec21)
libvirt: Save device_path in connection_info when booting from volume

If you boot an instance from a volume and later terminate it, the
libvirt volume driver disconnect_volume method does not have the
'device_path' key in connection_info['data'].  However, if you
attach a volume to an existing instance and then detach it,
the disconnect_volume method would have the 'device_path' key in
connection_info['data'].  Having the 'device_path' key would be
useful for some volume drivers to determine the device path of the
volume.  This patch saves the 'device_path' in connection_info['data']
when _create_domain_and_network is called, so it could be later used.

This commit also adjusts unit test to pass the check test.

Change-Id: I8ebb5f3c2e7a81b11d776f8c0a15f3491ed273be
Closes-Bug: #1291007
(cherry picked from commit d19c75c19d2de8b20e82e6de9413ba53671ad7fb)
Fix live-migration failure in FC multipath case

Currently, /dev/dm-<NUM> instead of /dev/mapper/<multipath_id> is
used to access multipath FC volumes by Compute Node and
multipath_id in connection_info is not maintained properly and
may be lost during connection refreshing.

This implementation will make source Compute Node and destination
Compute Node fail to disconnect/connect to volumes properly and
result in live-migration failure.

To fix it, /dev/mapper<multipath_id> will be used instead of
/dev/dm-<NUM> to access multipath devices, just like iSCSI multipath
implementation, and logic to preserve the unique (across Compute
Nodes) multipath_id is also added.

Closes-Bug: #1327497
(cherry picked from commit 3ea14e8a70a946dbb162ecafa848e4f2fa29772a)

Conflicts:
	nova/storage/linuxscsi.py
	nova/tests/virt/libvirt/test_libvirt_volume.py
	nova/virt/block_device.py
	nova/virt/libvirt/volume.py

This backport commit adjust oslo.i18n usage to oslo-incubator common code,
due to we didn't have oslo.i18n in icehouse.
And remove unused  variable value dev_str in test_libvirt_volume.py,
it should be deleted but not worth a specific commit in stable/icehouse.

Change-Id: I17f15852c098af88afd270084c62eb87693c60d4
Fixes Hyper-V agent force_hyperv_utils_v1 flag issue

WMI root\virtualization namespace v1 (in Hyper-V) has been removed
from Windows Server / Hyper-V Server 2012 R2.

Hyper-V compute agent now creates instances which uses
root\virtualization\v2 namespace if the agent's OS is
Windows Server / Hyper-V Server 2012 R2 or newer.

Closes-Bug: #1344036
(cherry picked from 52de9395e5fe4f328f6dab0b35d660a700787c76)

Change-Id: I874ade4456b92a63959a765c7851bcd001befa32
Sync process and str utils from oslo

This patch backports the necessary changes to fix both issues:
* Make execute method clean password in exception
* Make sure mask_password works properly

------------------------------------------------
The sync pulls in the following changes (newest to oldest):

63c99a0f - Mask passwords in exceptions and error messages
66142c34 - Make strutils.mask_password more secure
d6b55fb2 - Remove `processutils` dependency on `log`
cb5a804b - Move `mask_password` to strutils

-----------------------------------------------

Backport in oslo-incubator: https://review.openstack.org/121365

Closes-Bug: 1343604
Closes-Bug: 1345233
SecurityImpact

Change-Id: I3b49b1d667f6ade9ae3f6765d735440a3e838917
Add _wrap_db_error() support to Session.commit()

This patch adds _wrap_db_error() to session.commit(),
which has been observed to be a common point of failure for
deadlock exceptions.   In order to achieve this, the
_wrap_db_error() decorator itself also needed to propagate an
existing DBError, as it is the case that SQLAlchemy's
session.commit() calls into the session.flush() method.
Tests are added to exercise both the nesting of _wrap_db_error()
when a flush() inside commit() raises an exception, as well
as when commit() alone raises an exception that the error
is wrapped as expected.

Tests are omitted here as we are relying upon the tests
that were added to the corresponding oslo-incubator code.

Closes-bug: #1370191
Change-Id: I91510a2b864f0c1b73cfae18f271e94334714dce
Adds list_instance_uuids to the Hyper-V driver

In case of large number of servers, the _destroy_evacuated_instances
fails randomly. Implementing list_instance_uuids solves the issue.

Co-Authored-By: Robert Tingirica <rtingirica@cloudbasesolutions.com>
Co-Authored-By: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
Closes-bug: #1291364
(cherry picked from commit 73da55e4ef626283ae58a97c7ad89854ec77daa3)

Conflicts:
	nova/tests/virt/hyperv/test_vmops.py
	nova/tests/virt/hyperv/test_vmutils.py
	nova/tests/virt/hyperv/test_vmutilsv2.py
	nova/virt/hyperv/vmutils.py

Change-Id: I9c58168c012b342bb5dfa0c62a7c39a327f442b3
Fixes a Hyper-V list_instances localization issue

The Hyper-V WMI MSVM_ComputerSystem class Caption property can
have different values on various locales. This patch uses a
different query to avoid any localization issue.

Co-Authored-By: Robert Tingirica <rtingirica@cloudbasesolutions.com>
Closes-Bug: #1296478
(cherry picked from commit a5405fa3532d9dd3d33e848f36cc6530e74e6bb7)

Change-Id: I15103b8edb3ff8934c835826ed00e08593cdeacc
Merge "libvirt: return the correct instance path while cleanup_resize" into stable/icehouse
Merge "Add _wrap_db_error() support to Session.commit()" into stable/icehouse
Merge "Adds list_instance_uuids to the Hyper-V driver" into stable/icehouse
Merge "Fixes a Hyper-V list_instances localization issue" into stable/icehouse
Merge "Sync process and str utils from oslo" into stable/icehouse
Merge "_translate_from_glance() can cause an unnecessary HTTP request" into stable/icehouse
Merge "Fix nova image-show with queued image" into stable/icehouse
Merge "Fixes Hyper-V agent force_hyperv_utils_v1 flag issue" into stable/icehouse