osd/ReplicatedPG: move r<0 handling into finish_promote()

Let logic in header, and will let us handle ENOENT with a whiteout.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: use simple_repop_{create,submit} for finish_promote

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados_api_tier: rename tests

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados_api_tier: add simple promote-on-read test

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: create whiteout on promote ENOENT

If we try to fetch an object from the base tier and it is not present, we
can create a whiteout object.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: ENOENT when deleting a whiteout

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados_api_tier: verify delete creates whiteouts

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: set whiteout in cache pool on delete

If we delete an object in the cache pool, set the whiteout flag instead of
removing the on-disk object.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: clear whiteout when writing into cache tier

If we have a whiteout object and then write over it, clear the whiteout
flag.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: handle is_whiteout in do_osd_ops()

Most of the time we handle whiteouts by returning ENOENT before we even
get this far. However, for a mixed read/write transaction (e.g., a guard)
or certain ops (like create exclusive) we need to deal with the
exists == true and whiteout flag set case explicitly.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: set object_info and snapset xattrs on promote

For the normal write path, prepare_transaction() handles this for us.  In
this case, we need to do it explicitly.

Signed-off-by: Sage Weil <sage@inktank.com>
librados: add cache_flush(), cache_evict() methods

Not yet implemented by the OSD.

Signed-off-by: Sage Weil <sage@inktank.com>
librados, osd: add IGNORE_OVERLAY flag

Add a flag that will make the OSD bypass the cache overlay logic.  This is
needed in order to handle operations like CACHE_EVICT and CACHE_FLUSH.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/osd_types: introduce helper for osd op flags -> string conversion

Signed-off-by: Sage Weil <sage@inktank.com>

Conflicts:

	src/osd/osd_types.h
librados: add an aio_operate that takes a write and flags

Until now you could only pass flags to read operations.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: implement cache_evict

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: skip promote for DELETE

If an op starts with DELETE there is no need to promote the old content
from the base tier.  Note that this only works if the FAILOK flag is
set.  Otherwise, we need to know whether the object existed or not to
return either 0 or -ENOENT.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplcatedPG: maybe_handle_cache style

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: allow osds to issue writes to osds

We asserted that the client was not an OSD years ago when we separated out
the client and cluster networks.  Now, we are about to allow an OSD to
trigger a copy_from on another pool (for cache flush) and the assert can
go away.  We've long since verified that the messages are going out on
the correct interfaces.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: drop unnecessary temp vars in execute_ctx()

Both of these are pulled out of ctx->obs, which is not updated until the
very end; use that instead!

Signed-off-by: Sage Weil <sage@inktank.com>