remove redundant copy of test_cache_base_dir_exists

Second copy of RbdTestCase.test_cache_base_dir_exists was accidentally
introduced in https://review.openstack.org/82840.

Change-Id: I16b2e472b2499f01379fdca9c860da0b95e291d8
Closes-bug: #1302211
Not-in-upstream: true
Closes-rally-bug: DE1186
Implements: rally-task TA8264
Make rbd.libvirt_info parent class compatible

Rbd.libvirt_info function definition misses hypervisor_version argument added in change:
https://review.openstack.org/32379

Closes-Bug: #1233188
Change-Id: Ib68d743e783af0f6d82d2ba180869ee642e86050
(cherry picked from commit 7a34be0ec0cd0cb9555fe64ff6c486faae1ae91d)
Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
imagebackend: make verify_base_size() behave for rbd

verify_base_size() calls disk.get_disk_size(), which assumes the disk
is stored locally.

Instead of calling disk.get_disk_size() directly from
verify_base_size(), add a new method that is overridden by the Rbd
subclass to get the disk size.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
Add max_size to direct_fetch

Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
Fix dictionary update and key issues, update tests

Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
pass rbd_user id as part of RBD URI for qemu-img

Unlike rados.py, qemu-img does not use id from CEPH_ARGS and has to be
given a Cephx user id explicitly in RBD URI.

Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
Use Ceph cluster stats to report disk info on RBD

Local disk statistics on compute nodes are irrelevant when ephemeral
disks are stored in RBD. With RBD, local disk space is not consumed when
instances are started on a compute node, yet it is possible for
scheduler to refuse to schedule an instance when combined disk usage of
instances already running on the node exceeds total disk capacity
reported by the hypervisor driver.

Change-Id: Ifa1a2c4c12c9c9da2e96e36da8cd55165ea67b42
Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
Live migration is broken for NFS shared storage

One of the new checks, introduced in
I2755c59b4db736151000dae351fd776d3c15ca39
is missing a check against filebased shared storage,
leading to live migration being broken.

Closes-Bug: #1346385
Change-Id: I096ecd6eadc3b13ef918ca1e6c98acdd01556c0d
Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
test_direct_fetch_fail fails if the rbd module is installed

The prepare_mocks method is not called in the test and so the rbd
module is not mocked. Because of that, when the rbd module is not
installed, it is defined as None in the imagebackend module. This
is masquerading a couple of issues in the test.

* The _is_cloneable method takes one argument and so should do the
  lambda function that mocks it
* The tested method(direct_fetch) and most of the methods it calls,
  raise the same type of exception(ImageUnacceptable) with different
  messages. Asserting that the exception is raised without checking
  the message may lead to a false positive result

This commit fixies all the issues and split the test in several
tests to make different failures easier to identify.

Change-Id: I72f4c17d4ba3e284ba0c73eb4d4a8f8ea8fcbe2d
Signed-off-by: Rosario Di Somma <rosario.disomma@dreamhost.com>
Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true
rbd: use rbd snapshots and cloning for ephemeral disk snapshots

When rbd is used for ephemeral disks and image storage, we can avoid
local disk by doing the snapshot with ceph, and sending the location
of it to glance.

At this point glance is effectively the owner of the snapshot, and
nova cannot delete the original disk, so a corresponding change to
glance is needed to handle this cleanup later, when the snapshot is
deleted.

To accomplish this, create an rbd clone and store the clone's location
in glance, and when nova deletes an rbd disk with snapshots, rename
the disk to include a marker that glance looks for to know they can be
deleted when glance-scrubber gets to them.

This requires changing the typical permissions for the glance ceph
user to allow writing to the pool where vm images are stored.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Closes-rally-bug: DE1186
Implements: rally-task TA8264
Not-in-upstream: true