Make SecurityGroupAPI use Object instead of instance_get_by_uuid()

This patch makes the SecurityGroupAPI use the SecurityGroup object
for querying the list of security group names instead of pulling
the instance by id, and then following the ['groups'] link.

This is part of a set aimed at deprecating the instance_get_by_uuid()
method on conductor.

Related to blueprint compute-manager-objects-juno

Change-Id: Id69c2019e934cef72c9b06caa503ea74f28720ce
Treat instance objects like objects in set_admin_password flow

The set_admin_password API path uses instance objects but does a direct
database update in nova.compute.api and accesses fields on the object
like a dict.

This patch changes the code to update the database via the
Instance.save() method and accesses fields on the object using dot
notation.

Part of blueprint compute-manager-objects-juno

Change-Id: Iff82b19ba4ea007930d20d26a613687484392861
Remove unnecessary else block in compute manager set_admin_password

The else block is useless since the conditional block before it raises
an exception, so remove the else.

Change-Id: I1cd859a3f20b1ab9ab2502967526d306ebca9961
Don't translate debug level logs in nova.virt

Our translation policy
(https://wiki.openstack.org/wiki/LoggingStandards#Log_Translation) calls
for not translating debug level logs. This is to help prioritize log
translation. Furthermore translation has a performance overhead, even if
the log isn't used (since nova doesn't support lazy translation yet).

Change-Id: I524b48f530d8afd59a067074332e3964426e4d70
Remove variability from object change detection unit test

Insure hashes of object representations are consistent by normalizing
data using sorted lists (as compared to dictionaries whose keys and
values are listed in an arbitrary, variable, dependent order:
https://docs.python.org/2/library/stdtypes.html#dict.items).

Closes-Bug: #1325246

Change-Id: I1344a820263683a024818d181ec6ee08e5e18984
Merge "Don't translate debug level logs in nova.virt"
Merge "Make SecurityGroupAPI use Object instead of instance_get_by_uuid()"
Merge "Remove variability from object change detection unit test"
Merge "Treat instance objects like objects in set_admin_password flow"
Merge "Remove unnecessary else block in compute manager set_admin_password"
Improve shared storage checks for live migration

Due to an assumption that libvirt live migrations work only when both instance
path and disk data is shared between source and destination hosts (e.g. libvirt
instances directory is on NFS), instance disks are removed from shared storage
when instance path is not shared (e.g. Ceph RBD backend is enabled).

Distinguish cases that require shared instance drive and shared libvirt
instance directory. Reflect the fact that RBD backed instances have shared
instance drive (and no shared libvirt instance directory) in the relevant
conditionals. Disallow live migrations involving a compute host running an
older version of Nova.

Closes-bug: 1250751
Closes-bug: 1314526
Co-authored-by: Ryan Moe <rmoe@mirantis.com>
Co-authored-by: Yaguang Tang <yaguang.tang@canonical.com>
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
Change-Id: I2755c59b4db736151000dae351fd776d3c15ca39
Move libvirt RBD utilities to a new file

This will make it easier to share rbd-related code with cinder and glance.
Port the applicable unit tests over from cinder.

Most of the code is lifted as-is from imagebackend to rbd_utils, resize() is
fixed to take bytes instead of KiB.

blueprint rbd-clone-image-handler
Co-Authored-By: Dmitry Borodaenko <dborodaenko@mirantis.com>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
Change-Id: I614b8e2bf0ea9e71a73772cbabcf812b39fa8a34
Use library instead of CLI to cleanup RBD volumes

'rbd list' CLI returns error code when there are no rbd volumes, which
causes problems during live migration of VMs with RBD backed ephemeral
volumes. It's safer to use the library that only raises an exception in
case of a real problem.

The only case where rbd CLI is still justified is import, which is
needed to correctly import sparse image files.

All code related to cleanup of RBD volumes is moved to rbd_utils.py,
this fixes a yo-yo problem with single-use methods scattered across 3
different files, and minimizes impact of this fix on imports in
imagebackend and utils.

blueprint rbd-clone-image-handler
Change-Id: I92cd6b16fbd93b377fe47b15d22efbbf68d02513
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
enable cloning for rbd-backed ephemeral disks

Currently when using rbd as an image backend, nova downloads the
glance image to local disk and then copies it again into rbd. This
can be very slow for large images, and wastes bandwidth as well as
disk space.

When the glance image is stored in the same ceph cluster, the data is
being pulled out and pushed back in unnecessarily. Instead, create a
copy-on-write clone of the image. This is fast, and does not depend
on the size of the image. Instead of taking minutes, booting takes
seconds, and is not limited by the disk copy.

Add some rbd utility functions from cinder to support cloning and
let the rbd imagebackend rely on librbd instead of the rbd
command line tool for checking image existence.

Add a direct_fetch() method to the image backend, so backends like rbd
can make optimizations like this. Try to use direct_fetch() for the root
disk when it comes from an image, but fall back to fetch_to_raw() if
direct_fetch() fails.

Instead of calling disk.get_disk_size() directly from
verify_base_size(), which assumes the disk is stored locally, add a new
method that is overridden by the Rbd subclass to get the disk size.

Implements: blueprint rbd-clone-image-handler
Closes-Bug: 1226351
Co-Authored-By: Josh Durgin <josh.durgin@inktank.com>
Signed-Off-By: Josh Durgin <josh.durgin@inktank.com>
Signed-Off-By: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Signed-Off-By: Dmitry Borodaenko <dborodaenko@mirantis.com>
Change-Id: I0f50659b54a92fc21086990be8925ea15008569a