remove redundant copy of test_cache_base_dir_exists

Second copy of RbdTestCase.test_cache_base_dir_exists was accidentally
introduced in https://review.openstack.org/82840.

Change-Id: I16b2e472b2499f01379fdca9c860da0b95e291d8
Closes-bug: #1302211
Fix incorrect root partition size and compatible volume name

When using rbd backend, it needs to resize volume size if size presents. Else
we will get wrong image size.

Rbd volume always uses uuid as the main pattern of volume name, now change to
it is necessary.

Fix bug 1219658

Change-Id: I3dc8d639b066eac35d190b8d4687c41c9f4d5824
(cherry picked from commit 1b66a47b5f4cbcdf1a9a1ec38532474588a3fee2)
make rbd usable for ephemeral storage

Currently when using rbd as an image backend, nova downloads the
glance image to local disk and then copies it again into rbd.  This
can be very slow for large images, and wastes bandwidth as well as
disk space.

When the glance image is stored in the same ceph cluster, the data is
being pulled out and pushed back in unnecessarily. Instead, create a
copy-on-write clone of the image. This is a fast, and does not depend
on the size of the image. Instead of taking minutes, booting takes
seconds, and is not limited by the disk copy.

Add a direct_fetch() method to the image backend, so backends
like rbd can make optimizations like this. Try to use direct_fetch()
for the root disk when it comes from an image, but fall back to
fetch_to_raw() if direct_fetch() fails.

The majority of this code was taken from the cinder rbd driver.
The only new parts are the direct_fetch() method for the image
backend, and the small backwards-compatible interfaces changes
needed to make use of it.

fixes bug 1226351

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Change-Id: Ic9b9fbcb6e133cc9315581354f1b70ed7c9909ad
Make rbd.libvirt_info parent class compatible

Rbd.libvirt_info function definition misses hypervisor_version argument added in change:
https://review.openstack.org/32379

Closes-Bug: #1233188
Change-Id: Ib68d743e783af0f6d82d2ba180869ee642e86050
(cherry picked from commit 7a34be0ec0cd0cb9555fe64ff6c486faae1ae91d)
imagebackend: make verify_base_size() behave for rbd

verify_base_size() calls disk.get_disk_size(), which assumes the disk
is stored locally.

Instead of calling disk.get_disk_size() directly from
verify_base_size(), add a new method that is overridden by the Rbd
subclass to get the disk size.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Add max_size to direct_fetch
Improve shared storage checks for live migration

Distinguish cases that require shared instance drive and shared libvirt
instance directory. Reflect the fact that RBD backed instances have
shared instance drive (and no shared libvirt instance directory) in the
relevant conditionals.
Fix dictionary update and key issues, update tests
pass rbd_user id as part of RBD URI for qemu-img

Unlike rados.py, qemu-img does not use id from CEPH_ARGS and has to be
given a Cephx user id explicitly in RBD URI.
Use Ceph cluster stats to report disk info on RBD

Local disk statistics on compute nodes are irrelevant when ephemeral
disks are stored in RBD. With RBD, local disk space is not consumed when
instances are started on a compute node, yet it is possible for
scheduler to refuse to schedule an instance when combined disk usage of
instances already running on the node exceeds total disk capacity
reported by the hypervisor driver.

Change-Id: Ifa1a2c4c12c9c9da2e96e36da8cd55165ea67b42
Live migration is broken for NFS shared storage

One of the new checks, introduced in
I2755c59b4db736151000dae351fd776d3c15ca39
is missing a check against filebased shared storage,
leading to live migration being broken.

Closes-Bug: #1346385
Change-Id: I096ecd6eadc3b13ef918ca1e6c98acdd01556c0d
test_direct_fetch_fail fails if the rbd module is installed

The prepare_mocks method is not called in the test and so the rbd
module is not mocked. Because of that, when the rbd module is not
installed, it is defined as None in the imagebackend module. This
is masquerading a couple of issues in the test.

* The _is_cloneable method takes one argument and so should do the
  lambda function that mocks it
* The tested method(direct_fetch) and most of the methods it calls,
  raise the same type of exception(ImageUnacceptable) with different
  messages. Asserting that the exception is raised without checking
  the message may lead to a false positive result

This commit fixies all the issues and split the test in several
tests to make different failures easier to identify.

Change-Id: I72f4c17d4ba3e284ba0c73eb4d4a8f8ea8fcbe2d
Signed-off-by: Rosario Di Somma <rosario.disomma@dreamhost.com>
rbd: use rbd snapshots and cloning for ephemeral disk snapshots

When rbd is used for ephemeral disks and image storage, we can avoid
local disk by doing the snapshot with ceph, and sending the location
of it to glance.

At this point glance is effectively the owner of the snapshot, and
nova cannot delete the original disk, so a corresponding change to
glance is needed to handle this cleanup later, when the snapshot is
deleted.

To accomplish this, create an rbd clone and store the clone's location
in glance, and when nova deletes an rbd disk with snapshots, rename
the disk to include a marker that glance looks for to know they can be
deleted when glance-scrubber gets to them.

This requires changing the typical permissions for the glance ceph
user to allow writing to the pool where vm images are stored.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Merge branch 'rbd-ephemeral-snapshot-havana' of https://github.com/jdurgin/nova into merge-rbd-cloning

Conflicts:
	nova/tests/virt/libvirt/test_imagebackend.py
	nova/virt/libvirt/driver.py
	nova/virt/libvirt/imagebackend.py
	nova/virt/libvirt/utils.py