ceph-disk: reimplement is_partition

Previously we were assuming any device that ended in a digit was a
partition, but this is not at all correct (e.g., /dev/sr0, /dev/rbd1).
Instead, look in /dev/disk/by-id and see if there is a symlink that ends in
-partNN that links to our device.

There is probably still a better way...

Signed-off-by: Sage Weil <sage@inktank.com>
ceph-disk: reimplement list_all_partitions

Use /dev/disk/by-id to list disks and their partitions.  This is more
accurate and correct than the previous (as-yet unused) implementation.

Signed-off-by: Sage Weil <sage@inktank.com>
librbd: flush cache when set_snap() is called

If there are writes pending, they should be sent while the image
is still writeable. If the image becomes read-only, flushing the
cache will just mark everything dirty again due to -EROFS.

Fixes: #4525
Backport: bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Merge pull request #163 from ceph/wip-rbd-snapset-cache

librbd: flush cache when set_snap() is called

Reviewed-by: Sage Weil <sage@inktank.com>
ObjectCacher: optionally make writex always non-blocking

Add a callback argument to writex, and a finisher to run the
callbacks. Move the check for dirty+tx > max_dirty into a helper that
can be called from a wrapper around the callbacks from writex, or from
the current place in _wait_for_write().

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: make aio_writes to the cache always non-blocking by default

When the ObjectCacher's writex blocks, it affects the thread requesting
the aio, which can cause starvation for other I/O when used by QEMU.

Preserve the old behavior via a config option in case this has any
bad side-effects, like too much memory usage under heavy write loads.

Fixes: #4091
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: always complete flush_set() callback

This removes the last remnants of
b5e9995f59d363ba00d9cac413d9b754ee44e370. If there's nothing to flush,
immediately call the callback instead of deleting it. Callers were
assuming they were responsible for completing the callback whenever
flush_set() returned true, and always called complete(0) in this
case. Simplify the interface and just do this in flush_set(), so that
it always calls the callback.

Since C_GatherBuilder deletes its finisher if there are no subs,
only set its finisher when subs are present. This way we can still
call ->complete() for the callback.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: remove NULL checks in flush_set()

Callers will always pass a callback, so assert this and remove the
checks for it being NULL.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: remove unneeded var from flush_set()

The gather will only have subs if there is something to flush. Remove
the safe variable, which indicates the same thing, and convert the
conditionals that used it to an else branch. Movinig gather.activate()
inside the has_subs() check has no effect since activate() does
nothing when there are no subs.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librados: increase version number to indicated new functions

Listing snaps and listing watchers were added to the C++ api.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librados: add async flush interface

Sometimes you don't want flush to block, and can't modify
already scheduled aio_writes. This will be useful for a
librbd async flush interface.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librados: add versions of a couple functions taking explicit snap args

Usually the snapid to read from or the snapcontext to send with a write
are determined implicitly by the IoCtx the operations are done on.

This makes it difficult to have multiple ops in flight to the same
IoCtx using different snapcontexts or reading from different snapshots,
particularly when more than one operation may be needed past the initial
scheduling.

Add versions of aio_read, aio_sparse_read, and aio_operate
that don't depend on the snap id or snapcontext stored in the IoCtx,
but get them from the caller. Specifying this information for each
operation can be a more useful interface in general, but for now just
add it for the methods used by librbd.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: add an is_complete() method to AioCompletions

Mainly this is useful for testing, like flushing and checking that
all pending writes are complete after the flush finishes.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: use the same IoCtx for each request

Before we were duplicating the IoCtx for each new request since they
could have a different snapshot context or read from a different
snapshot id. Since librados now supports setting these explicitly
for a given request, do that instead.

Since librados tracks outstanding requests on a per-IoCtx basis, this
also fixes a bug that causes flush() without caching to ignore
all the outstanding requests, since they were to separate,
duplicate IoCtxs.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: add an async flush

At this point it's a simple wrapper around the ObjectCacher or
librados.

This is needed for QEMU so that its main thread can continue while a
flush is occurring. Since this will be backported, don't update the
librbd version yet, just add a #define that QEMU and others can use to
detect the presence of aio_flush().

Refs: #3737
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: increment version for new functions is_complete() and aio_flush()

This done in a separate commit since the increased version number
should not be backported.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
PG: use int64_t for pool id in PGPool

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: update PGPool::name in PGPool::update

Fixes: #4471
Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge pull request #168 from athanatos/wip_4471

Wip 4471

Reviewed-by: Sage Weil <sage@inktank.com>
ceph: propagate do_command()'s return value to user space

We were returning '1' regardless of what do_command() returned in case
of error.  This would make building tools relying on command error codes
short of useless, and forced them to rely instead on error messages.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit e91405d540ce11b9996e4977212553bd33afb3ed)