ObjectCacher: deduplicate final part of flush_set()

Both versions of flush_set() did the same thing. Move it into a
helper called from both.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
WritebackHandler: make read return nothing

The tid returned by reads is ignored, and would make tracking writes
internally more difficult by using the same id-space as them. Make read
void and update all implementations.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
LibrbdWriteback: use a tid_t for tids

An int could be much smaller, leading to overflow and bad behavior.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
LibrbdWriteback: removed unused and undefined method

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
LibrbdWriteback:  complete writes strictly in order

RADOS returns writes to the same object in the same order. The
ObjectCacher relies on this assumption to make sure previous writes
are complete and maintain consistency. Reads, however, may be
reordered with respect to each other. When writing to an rbd clone,
reads to the parent must be performed when the object does not exist
in the child yet. These reads may be reordered, resulting in the
original writes being reordered. This breaks the assmuptions of the
ObjectCacher, causing an assert to fail.

To fix this, keep a per-object queue of outstanding writes to an
object in the LibrbdWriteback handler, and finish them in the order in
which they were sent.

Fixes: #4531
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Merge pull request #214 from ceph/wip-objectcacher-handler-ordered

keep write responses to clones in order

Reviewed-by: Sage Weil <sage@inktank.com>