Use block_device_info at post_live_migration_at_destination

post_live_migration_at_destination make libvirt.xml to destination
host. when execute (live) block migration for vm made by bootable
volume, method post_live_migration_at_destination write a
libvirt.xml in destination host.
but it missed block disk information so moved libvirt.xml always
has a wrong disk information.
I add block_device_info in blockinfo.get_disk_info at
post_live_migration_at_destination.
This change will make libvirt.xml with valid block disk information.

Closes-Bug: 1271780
(cherry picked from commit 0486647e1c85947bc48cbdd279b3f49824a6e692)

Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py
	nova/virt/libvirt/driver.py

Change-Id: I31b3f2ef5f03f9ef61524bdb7ae089b18e4cd254
Take a vm out of SNAPSHOTTING after Glance error

If a snapshot is attempted to be taken when Glance can't find the
underlying image (usually because it's been deleted), then ImageNotFound
is raised to nova.compute.manager.ComputeManager._snapshot_instance. The
error path for this leaves the instance in a snapshotting state; after
which "operator intervention" is required.

This simply reverts the instance state under that specific circumstance.

DocImpact: previously a snapshot that failed for this reason would leave
the instance in a spurious snapshotting task state indefinitely. Now,
the task state is reverted. At the moment that revert is 'silent'
(although the failure to snapshot will be evident to the user).

Fixes-bug: #1267097
(cherry picked from commit 96dba50737819c24b4ddd9cb1078da305638abc9)

Conflicts:

	nova/compute/manager.py

Change-Id: I9623cfaed31e8337ab65bc1eab85ad3b51b1a5ca
Further harden boto version checking in EC2 tests

Use pkg_resources to determine correct boto interface
to use in API tests.  stable/havana version.

Fixes Bug #1239220

Rebased-By: Dirk Mueller <dirk@dmllr.de>

Change-Id: I455ce5f2aabfad7178be361844c6b9b7b0d9c946
Explicity teardown on error in libguestfs setup()

Previously, in the setup method of the libguestfs driver
for Nova's VFS, on an error the handle object was simply
dereferenced (set to None), and then an new error was thrown.
This relied on an implicit close() on the handle being called
by GC.  However, in some cases the setup progresses far enough
that the implicit close is not enough, and leaves an "orphaned"
libguestfs VM.

Now, the teardown() method is called explicitly, which ensures that
no "orphaned" VMs are left around in case of an error.

Change-Id: I7fbe470046ce6c76bc13d77d8df360351a3ef715
Fixes: bug #1270304
(cherry picked from commit fa62cfe15fd6dddecf657966d166a2d3a2bdd317)
neutron_metadata_proxy_shared_secret should not be written to log file

This patch sets secret=True for the following config option
neutron_metadata_proxy_shared_secret

Conflicts:

	nova/api/metadata/handler.py

Change-Id: I381ea9e89f76beac11b03def6e3ccfee2a55e48f
Closes-bug: #1288463
(cherry picked from commit fa73a5d037471dbec11893ad83bd93276968b9af)
VMware: fix the VNC port allocation

There is small chance for VNC port collisions with the current
implementation which choose the port number based on the MoRef id
of the VM.
This patch fixes this by running a query for all allocated ports
and then selects one which is not taken.

Closes-Bug: #1255609
(cherry picked from commit 2f49ed4b5dbb5c954fc7a9b42ee7b170c38c775c)

Conflicts:

	nova/exception.py
	nova/tests/virt/vmwareapi/test_vmwareapi.py
	nova/tests/virt/vmwareapi/test_vmwareapi_vm_util.py
	nova/virt/vmwareapi/vm_util.py
	nova/virt/vmwareapi/vmops.py

Change-Id: If7c3b14dd49ed05c5fde819c5a36d5608650cbbc
Merge "neutron_metadata_proxy_shared_secret should not be written to log file" into stable/havana
Merge "VMware: fix the VNC port allocation" into stable/havana
Merge "Explicity teardown on error in libguestfs setup()" into stable/havana
Merge "Take a vm out of SNAPSHOTTING after Glance error" into stable/havana
Grizzly to Havana Upgrade wipes out Nova quota_usages

Migration script 216 will wipe out all data in quota_usages table
when user_id is null. After the upgrade from Grizzly to Havana is
completed, the migration script 203 adds a new column user_id, so
user_id is null for all data in quota_usages. So the correct
behaviour should be sync quota_usages table first and then delete
data if user_id is still null.

Fixing the existing migrate script is because we can not find
the quota_usages data after migration script 216.

Closes-bug:1245746

Change-Id: I4db62884785a4eb6e1db2c5b114b9c6e1b4882ad
VMware: enable driver to work with postgres database

The hypervisor version is expected to be an integer and not
a string.  The nova/utils.py change is part of the following
patch but is not being backported as it adds new feature
functionality and is therefore not acceptable for backport

https://review.openstack.org/#/c/36073/

Closes-Bug: #1195139
(cherry picked from commit e2efa57ab2fb03b92f80abbf612a257c28475140)

Conflicts:

	nova/tests/virt/vmwareapi/test_vmwareapi.py
	nova/virt/vmwareapi/host.py

Change-Id: Ifac9ca3480191413e3a02f0dae94e452707ae82f
Merge "VMware: enable driver to work with postgres database" into stable/havana
handle empty network info in instance cache

The nw_info value [] was being incorrectly equated with None. This caused
infinite recursion when api._get_instance_nw_info returned [] and was wrapped
with @refresh_cache (e.g., _get_instance_nw_info in
nova.network.neutronv2.api.API).

Fixes bug #1251792.

Change-Id: I3034fbb20e790f2d6f22c50b74a9f9dcdf9081aa
(cherry picked from commit 8bc974be50bea6ff6beacda890d9a33fa90059b3)
Do not hide exception in update_instance_cache_with_nw_info

From time to time an exception is raised in this method causing
the nw_info cache not to be saved. If this occurs we should raise
as this error will cause later errors to occur. For example, one
won't be able to associate a floatingip with the instance as there
is no nw_info found in this table. In addition, the fixed_ips on
the instance won't be returned via the api.

This patch also stubs out update_instance_cache_with_nw_info in a
few tests where an exception was being raised previously but went
unnoticed as it was not reraised but now is.

Related-Bug: #1252849
Related-Bug: #1249065

Change-Id: Ic860f72210ba736e11c10df21c4cb7625e9c0928
(cherry picked from commit 4c03383f21bc13caf3fed4db5aa1317d37758d5c)
Persist image format to a file, to prevent attacks based on changing it

The attack is based on creating a raw image that looks like a qcow2
image, and taking advantage of the code that used 'qemu-img info' to
autodetect the image format.

Now we store the image format to a 'disk.info' file, for Qcow2 and Raw
images, and only autodetect for images that have never been written to
that file.

SecurityImpact

Conflicts:
	nova/virt/libvirt/imagebackend.py

Manual tweaks to some mocking in test_imagebackend.py

Change-Id: I2016efdb3f49a44ec4d677ac596eacc97871f30a
Co-authored-by: Nikola Dipanov <ndipanov@redhat.com>
Closes-bug: #1221190
Merge "Use block_device_info at post_live_migration_at_destination" into stable/havana
Merge "Further harden boto version checking in EC2 tests" into stable/havana
Change gitreview to point to nectar

Change-Id: Ib31fa3b8522cf43adf70ebd76be83d6b82a3aa65
Use DB for cells tests