Move libvirt RBD utilities to a new file

This will make it easier to share rbd-related code with cinder and glance.
Port the applicable unit tests over from cinder.

blueprint rbd-clone-image-handler
Co-Authored-By: Dmitry Borodaenko <dborodaenko@mirantis.com>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
Change-Id: I614b8e2bf0ea9e71a73772cbabcf812b39fa8a34
Use library instead of CLI to cleanup RBD volumes

'rbd list' CLI returns error code when there are no rbd volumes, which
causes problems during live migration of VMs with RBD backed ephemeral
volumes. It's safer to use the library that only raises an exception in
case of a real problem.

The only case where rbd CLI is still justified is import, which is
needed to correctly import sparse image files.

All code related to cleanup of RBD volumes is moved to ceph.py, this
fixes a yo-yo problem with single-use methods scattered across 3
different files, and minimizes impact of this fix on imports in
imagebackend and utils.

blueprint rbd-clone-image-handler
Change-Id: I92cd6b16fbd93b377fe47b15d22efbbf68d02513
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
Add include_locations kwarg to nova.image.API.get()

To avoid feature envy from image backends that need image location URIs,
functionality of nova.image.glance._get_locations() is moved into
nova.image.glance.show() and exposed via a new include_locations kwarg
to the nova.image.API.get() method, which, when True, simply includes a
collection of location links when the image has them.

blueprint rbd-clone-image-handler
Co-Authored-By: Jay Pipes <jaypipes@gmail.com>

Change-Id: I5118013aa24f68520dee806c24265f2808a90588
Enable cloning for rbd-backed ephemeral disks

Currently when using rbd as an image backend, nova downloads the glance
image to local disk and then copies it again into rbd. This can be very
slow for large images, and wastes bandwidth as well as disk space.

When the glance image is stored in the same ceph cluster, the data is
being pulled out and pushed back in unnecessarily. Instead, create a
copy-on-write clone of the image. This is fast, and does not depend on
the size of the image. Instead of taking minutes, booting takes seconds,
and is not limited by the disk copy.

Add some rbd utility functions from cinder to support cloning and let
the rbd imagebackend rely on librbd instead of the rbd command line tool
for checking image existence.

Adds a new clone() method to the image backend, so backends like rbd can
make optimizations like this. Try to use clone() for the root disk when
it comes from an image, but fall back to fetch_to_raw() if clone()
fails.

Instead of calling disk.get_disk_size() directly from
verify_base_size(), which assumes the disk is stored locally, add a new
method that is overridden by the Rbd subclass to get the disk size.

Implements: blueprint rbd-clone-image-handler
Closes-Bug: 1226351
Co-Authored-By: Josh Durgin <josh.durgin@inktank.com>
Signed-Off-By: Josh Durgin <josh.durgin@inktank.com>
Signed-Off-By: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Signed-Off-By: Dmitry Borodaenko <dborodaenko@mirantis.com>
Change-Id: I0f50659b54a92fc21086990be8925ea15008569a
Use Ceph cluster stats to report disk info on RBD

Local disk statistics on compute nodes are irrelevant when ephemeral
disks are stored in RBD. With RBD, local disk space is not consumed when
instances are started on a compute node, yet it is possible for
scheduler to refuse to schedule an instance when combined disk usage of
instances already running on the node exceeds total disk capacity
reported by the hypervisor driver.

Change-Id: I9718c727db205b6f2191f8435583391584e96e6e
Closes-bug: #1332660
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
Live migration is broken for NFS shared storage

One of the new checks, introduced in
I2755c59b4db736151000dae351fd776d3c15ca39
is missing a check against filebased shared storage,
leading to live migration being broken.

Closes-Bug: #1346385
Change-Id: I096ecd6eadc3b13ef918ca1e6c98acdd01556c0d