osd: add rados CACHE mode (different from RD and WR)

It is useful to distinguish cache operations from read and modify
operations.  Specifically, we will allow cache ops to be sent for
snaps and also allow those ops to result in a write.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: allow cache-evict on snaps

We do three things here:

 - make cache-evict a CACHE instead of WR op, allowing us to submit it
   on snaps (not just head)
 - allow eviction of a snap
 - verify that all snaps are missing before evicting a head

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: make cache-[try-]flush CACHE instead of WR ops

This will allow us to send a flush op on a snap.

Signed-off-by: Sage Weil <sage@inktank.com>
vstart.sh: allow MDS=0

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: refuse to flush when older dirty clones are present

If the next oldest clone is dirty, we cannot flush.  That is, we must
always flush starting with the oldest dirty clone.

Note that we can never have a sequence like dirty -> clean -> dirty,
because clones are only dirty on creation, are created in order, and cannot
be flushed (cleaned) out of order.  Thus checking the previous clone is
sufficient (and thankfully cheap).

Signed-off-by: Sage Weil <sage@inktank.com>
osd: add pg_log_entry_t event type CLEAN

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: construct appropriate snapc for flush/writeback

Construct a snap context that will trigger the appropriate cloning (if any)
on the base pool.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados_api_tier: test flush on snaps/clones

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados: don't update any state on successful cache-evict

- we didn't touch the user_version
- we didn't change the clean/dirty state

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados: test flush/evict on snaps

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados: IsDirty on random snaps

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados: make flush clean correct snap in model
ceph_test_rados: assert on read error

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados: is_dirty on non-flushing objects only

This makes its results reliable.  Otherwise, we can't mix the is_dirty
test with flush, which eliminates much of its value.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: do not promote nonexistent clones

Do not promote a clone for a snap that we know doesn't exist.  If
find_object_context() didn't give us a missing_oid, there is nothing to
promote.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: always set obc->ssc SnapSetContext for clones

This can be useful!

Signed-off-by: Sage Weil <sage@inktank.com>
osd: include snap_seq in copy-get results

This is needed by the cache layer when reading a logical snap from a head
object on the backend in order to correctly recreate the clone in the
cache layer.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: infer snaps from head when promoting oldest clean clone

Consider:

 - base and cache have same object foo; marked clean in cache pool
 - modify + clone foo in cache pool.  foo clone is clean.
 - foo clone is evicted
 - foo clone is read, and promoted
 - we read foo@something from base pool, and get the head's content

copy-get does not provide us with a snaps list.  Instead, we use the
snap_seq from the head to infer what the snaps vector was in the cache
pool and will be in the base pool when we flush the updates to the object.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_test_rados: improve read debug output

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: preserve clean/dirty state on clone

If we have a clean object and clone it in make_writeable(), the clone
should also be clean (it does not need to be written back to the base
pool).  If the object was dirty, the clone should be dirty.

Signed-off-by: Sage Weil <sage@inktank.com>