Loosen import_exceptions to cover all of gettextutils

Now that we are starting to move over to multiple log domains we want to
import multiple items on the same line for gettextutils.

Change-Id: Icd88774a0f1087270234a6849bf96a93c0927534
(cherry picked from commit 2d75fafb88f7e871ae63304f88eb34e48edac2e1)
_translate_from_glance() can cause an unnecessary HTTP request

After returning from a get() call to python-glanceclient, nova runs a
translation function on the returned Image to get the data it wants. Part of
this process is checking for an expected set of attributes, one of which is
the deletion time ('deleted_at'). However, if the image has not been deleted,
deleted_at key will not exist. This forces another call to glance to occur for
the same image. A similar problem exists for the checksum attribute, which does
not exist before an image is active. The fix here is to only consider
deleted_at and checksum if they are expected to be present.

Change-Id: I67b7dd16a94fe60d873c012f6bd246ab24500d5a
Closes-Bug: #1275173
(cherry picked from commit 8fcefef5c6db3bb7f182df62de9c5bf986f97303)
Fix nova image-show with queued image

Currently, the nova image-show command cannot be used with queued images,
because they don't have certain attributes set yet. Nova will raise
AttributeError, which causes a 500 to be returned from the API.

Change-Id: I7d73b891545492e3e247eebdf02507cb630eb250
Closes-bug: #1321186
(cherry picked from commit 4c5dea6a5103a141b0ca13dcd57b32b3261180d1)
libvirt: return the correct instance path while cleanup_resize

If we resized a pre-grizzly instance with grizzly or later nova
to another host, while the resize confirmation process,
_cleanup_resize will find the instance resize backup dir and
delete it, but a wrong xxx_resize dir like ${uuid}_resize,
instead of the correct ${name}_resize will be found.
This is because the instance is a resized one which original
instance dir exists on another host(the dest host),
get_instance_path method could not find the original instance
dir on the source host, so the path with uuid will be returned,
and the `target` existing check in _cleanup_resize is failed,
then the ${name}_resize dir will never be deleted.

Closes-bug: #1290294
Change-Id: I904b6751dec740e001f5ec29f637ef456528746f
(cherry picked from b4964eb6a570e290545f95d45411dc8441985cd5)
VM in rescue state must have a restricted set of actions

Right now it is possible to pause, suspend and stop a VM in state RESCUED,
so after the state is changed, it's not possible to trigger unrescue anymore
since the original state is lost.

This patch remove vm_states.RESCUED as valid state from stop,
pause and suspend actions.

The vm_states devref is also updated to reflect this change including the
current reboot flow.( vm_states.RESCUED cannot be rebooted as per
today code)

This is a backport of 8ff170dc95bf3101fe38a2624e941bfa3b7c1138 .

DocImpact
Closes-Bug: #1319182
Co-Authored-By: Cyril Roelandt <cyril.roelandt@enovance.com>
Change-Id: I531dea5a5499bf93c24bea37850d562134dee281
Delete image when backup operation failed on snapshot step

when we snapshot an instance, we will use @delete_image_on_error to
delete any failed snapshot, however, the image will not be removed
by backup code flow, it will be an issue if too many backup failed
at last ,all useful image will be removed and we have only 'error'
image left in host

Change-Id: Ib527ade86aef52a8445b57fa03fed434578c4204
Closes-Bug: #1288574
(cherry picked from commit 3824051b1e5618388a17c88867a3037397bc96b7)
Avoid traceback logs from simple tenant usage extension

Avoid generating traceback logs when invalid formatted datetime
parameters are passed to the simple tenant usage
extension. Exceptions have not been correctly handled by the
os-simple-tenant usage extension which results in tracebacks
being output to the log files. This patch correctly handles
exceptions as a result of badly formatted datetime strings.

Conflicts:
        nova/tests/api/openstack/compute/contrib/test_simple_tenant_usage.py

Change-Id: Iecd42239357b96450b6530c0486b27c68995c37f
Partial-Bug: 1300972
(cherry picked from commit f60c72fac04a97b321a7f2bd0865c1ae81250882)
Fix _parse_datetime in simple tenant usage extension

_parse_datetime in os-simple-tenant-usage incorrectly attempts
to parse the datetime string even when it is None. As a result
a 400 BadRequest is returned. This has the overall effect of making
the start and end datetime parameters compulsory when they are meant
to be optional.

Note that this restores the behavior where start/end are optional which is
in contradiction to the API docs, but is the long standing API behaviour.
This was an API change which was accidentally applied in commit
I8e0e870727d687da165c809ffb7a4456bff81122 as part of some nova internal
changes but not picked up by the unitests

Change-Id: I00427379dc7aa39770f9a16ff026addb6e311735
Closes-Bug: 1300972
(cherry picked from commit 99e6ec9131c2e3eeb5fe3e1a4dc5a26e6302ebd2)
VMware: do not cache image when root_gb is 0

Save space on datastore when the instance size has root_gb of
0. There is no need to cache the disk as it already exists on
the datastore.

TrivialFix

Closes-bug: #1304593
(cherry picked from commit 9b14ae62a6396a2599c8a2ada1a50e3f69a2eddc)

Conflicts:
	nova/tests/virt/vmwareapi/test_driver_api.py
	nova/virt/vmwareapi/vmops.py

Change-Id: Iac75d97f40a1a91b8fa2df7fb4318fc2965fd93a
Include next link when default limit is reached

The /servers and /servers/detail APIs support pagination and a
"next" link should be included when more data is available, see
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html. In the documentation it states
that 'collections are required to contain atom "next" links'.

When the default "osapi_max" limit is reached then the "next" link
is not included in the API reply. In this case, the caller cannot
determine if there are any more servers and has no marker value
such that they can retrieve the rest of the servers.

For example, assume that there are 1050 VMs and the osapi_max value
is the default of 1000. The /servers API will return the first 1000
VMs but it will not include the "next" link; the caller has no way
to determine if there are more then 1000 VMs and no way to retrieve
the other 50 VMs.

The fix for this is to include the "next" link when the number of
servers being returned is the maximum limit, even if the "limit"
parameter is not supplied. Note: the caller could workaround this
by specifying the exact "osapi_max" limit value (ie, limit=1000)
on the API call but this requires that the caller knows the
configured limit.

Change-Id: I9b3da852b0ab11dc980524e2317ee876a3ec28b8
Closes-bug: 1288466
(cherry picked from commit 968c01afe7ff43346f5817e2a3adf97fd03106b1)
Fixes Hyper-V vm state issue

The method which gets VM related information can fail if the
vm is in an intermediary state such as "Shutting down".
The reason is that some of the Hyper-V specific vm states
are not defined as possible states.

This patch assures that a valid state is always returned by
this method.

Change-Id: I64e95f6b537e83a30dd64474f63f140e0e8e7373
Closes-Bug: #1352428
(cherry picked from commit 86a4bd7d9b34057f04ebbe2d2131f033d33d082b)
Adds get_instance_disk_info to compute drivers

Solves a live migration regression issue by adding the
get_instance_disk_info to the compute drivers that do not
have it already implemented.

The method is called by the compute manager.

Change-Id: If5cd9a6f5948c08ef5f1cba2eda31f1175709137
Closes-Bug: #1358719
Co-Authored-By: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
(cherry picked from commit 9c9e44e111abafde1e05a5e38f7e1645ec8ba6df)
Update block_device_info to contain swap and ephemeral disks

An ephemeral or swap disk is attached to an instance on boot
as follows: nova boot --flavor FLAVOR --image IMAGE_ID
--swap 512 --ephemeral size=2 INSTANCE.  When a hard reboot is
performed on the instance, nova fails to recreate the
appropriate libvirt XML definition, containing the ephemeral
disk.  This is because the correct block_device_info dict that
is passed to the compute manager's reboot_instance method does
not contain swap or ephemeral disk key values that are necessary
to recreate those disks.  In addition to nova boot, the correct
block_device_info dict is also needed by nova rebuild, reboot,
resize, and migrate to recreate those disks.  This patch updates
_get_instance_volume_block_device_info (renamed
_get_instance_block_device_info) to return the swap and ephemeral
disk key values, in addition to the block_device_mapping it
already returns today.

Change-Id: Iec329d1c12a48ea90ba9d57decd0996fde6544f0
Closes-Bug: #1305423
(cherry picked from commit f087a6f77ef1338bb8d10943d2a18712220c3c44)
Catch missing Glance image attrs with None

Glance is known to drop values from its response if they don't resolve to true.
 Previously in I67b7dd16a94fe60d873c012f6bd246ab24500d5a the None catch was
 removed as this WILL cause a call to glance for missing value and
 optimizations where made to attempt to pre-cache values that can be missing
 in expected ways. However there are still situations that can't be optimized.

Notably this can occur with disk_format, container_format, name, deleted,
 and checksum.

Test adapted from I85c42f8351763da201021a22f5ff0ebd62c6b2db by Mike Perez

Closes-bug #1308058
Co-Author: Mike Perez <thingee@gmail.com>

Change-Id: Icddead6f746d4d3ba652858eaae425acce8c177d
(cherry picked from commit ef3f158a5aa1988be8ab2f295252373f3ee85809)
libvirt: Use VIR_DOMAIN_AFFECT_LIVE for paused instances

When a volume is attached to a paused instance, it does not
appear in the instance's block devices list (i.e. lsblk)
after the instance has resumed. A similar situation
happens when detaching a volume from a paused instance; the
block device continues to exists in the block devices list,
where it should not.  It was found that the volume is only
persisted in the domain's config settings and not its active
settings, since only the VIR_DOMAIN_AFFECT_CONFIG flag was used
on attach and detach volume.  In order to affect the active
settings, the VIR_DOMAIN_AFFECT_LIVE flag has to be added when
attaching and detaching volumes.

This change adjusts little about the unit test:
add two imports.

Change-Id: I9c90a410a7ecb91f5a4de28acee21fe7da49242c
Closes-Bug: #1242366
Related-Bug: #1299331
(cherry picked from commit ead9cfca93c9b5ca55e3ba269213c98d5e6e1d38)
Fix instance boot when Ceph is used for ephemeral storage

is_block_dev attribute was mistakenly set to True for RBD images
(we don't actually map RBD images to block devices in the *host*
system, but only in *guests*, so as far as the host system is
concerned an RBD image must not be treated like a system block
device). This led to a situation when mkfs was called for a non
existing file and failed when trying to create an ephemeral fs.

Closes-Bug: #1362221

Change-Id: I54c0d117a53bb61f278b2e137bd29a595548f5a1
(cherry picked from commit e9eb1d69bf75c22b0ad4b50e4892a5644c78cf68)
Fix rootwrap for non openstack.org iqn's

The encryption methods implemented for attached volumes
require a symbolic link created to the /dev/disk-by* iqn.

The current implementation works fine for LVM, however the rootwrap
is restricted to only allow iqns of the form openstack.org, for
vendors that use their own target and iqn this won't work and will
result in the attach failing for unauthorized command.

This just makes the regex for the rootwrap filter a bit more
permissive, only looking for iscsi-iqn.*

Change-Id: I023ad24867c045a88f72c5ac7ac4e4da097a3643
Closes-Bug: 1362854
(cherry picked from commit 00808f2072c3ee8958ad16eabad7994730bb8b86)
db: Add @_retry_on_deadlock to service_update()

Add the _retry_on_deadlock decorator to the service_update() method of
nova's sqlalchemy API.  This decorator is scattered throughout the
sqlalchemy API for other methods that may encounter this error.  The
referenced bug report shows a trace from where it occurred on this
method.

Change-Id: I93b370d6457d2e85493be01a62a76404d228a6fa
Closes-bug: #1370191
(cherry picked from commit 61d54f2652424279af81c4084e87651044fac4bd)
Fixes Hyper-V resize down exception

The Hyper-V driver does not support resize down and is currently
raising an exception if the user attempts to do that, causing the
instance to go in ERROR state.

The driver should use the recently introduced instance faults
"exception.InstanceFaultRollback" instead, which will leave the
instance in ACTIVE state as expected.

Closes-Bug: #1354448

(cherry picked from commit 3a5919fd4af6c3b772397a5e7d90eebdf9b371af)

Conflicts:
	nova/virt/hyperv/migrationops.py

Change-Id: Ibaf8482562094cd2b3165dc62a907fa9e0e56e19
Fixes Hyper-V boot from volume root device issue

Checks that instances are booting from volume by checking
the correct root device obtained from the block device info.

Change-Id: I94941dfe96c1596c8e9b91df3d3d19d33ae7fe92
Co-Authored-By: Zsolt Dudas <zdudas@cloudbasesolutions.com>
Closes-Bug: #1357972
(cherry picked from commit ea19fb10c5e09ff5df383607654ab9dc2791ec21)