ReplicatedPG::cancel_copy: clear cop->obc

Otherwise, an objecter callback might still be hanging
onto this reference until after the flush.

Fixes: #8894
Introduced: 589b639af7c8834a1e6293d58d77a9c440107bc3
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::maybe_handle_cache: do not forward RWORDERED reads

Even with READFORWARD, we can't forward RWORDERED reads.

Fixes: #9119
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::start_flush: remove superfluous loop

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG:start_flush do not use ORDERSNAP

Suppose we start with the following in the cache pool:

30:[29,21,20,15,10,4]:[22(21), 15(15,10), 4(4)]+head

The object doesn't exist at 29 or 20.

First, we flush 4 leaving the backing pool with:

3:[]+head

Then, we begin to flush 15 with a delete with snapc 4:[4] leaving the
backing pool with:

4:[4]:[4(4)]

Then, we finish flushing 15 with snapc 9:[4] with leaving the backing
pool with:

9:[4]:[4(4)]+head

Next, snaps 10 and 15 are removed causing clone 10 to be removed leaving
the cache with:

30:[29,21,20,4]:[22(21),4(4)]+head

We next begin to flush 22 by sending a delete with snapc 4(4) since
prev_snapc is 4 <---------- here is the bug

The backing pool ignores this request since 4 < 9 (ORDERSNAP) leaving it
with:

9:[4]:[4(4)]

Then, we complete flushing 22 with snapc 19:[4] leaving the backing pool
with:

19:[4]:[4(4)]+head

Then, we begin to flush head by deleting with snapc 22:[21,20,4] leaving
the backing pool with:

22[21,20,4]:[22(21,20), 4(4)]

Finally, we flush head leaving the backing pool with:

30:[29,21,20,4]:[22(21*,20*),4(4)]+head

When we go to flush clone 22, all we know is that 22 is dirty, has snaps
[21], and 4 is clean. As part of flushing 22, we need to do two things:
1) Ensure that the current head is cloned as cloneid 4 with snaps [4] by
sending a delete at snapc 4:[4].
2) Flush the data at snap sequence < 21 by sending a copyfrom with snapc
20:[20,4].

Unfortunately, it is possible that 1, 1&2, or 1 and part of the flush
process for some other now non-existent clone have already been
performed. Because we don't know whether 1 has already happened, we
currently use ORDERSNAP to prevent us from deleting a second time after
2) has been sent. I think we can get away without it:

If the snap_seq of this object on the backing pool is > 4, we know that
at some point, a delete was first sent with snapc 4:[4], so clone 4 has
been correctly created. In that case, because clone 22 is still dirty,
the newest snap seq which could have been sent to the backing pool is
20.  Thus, because there are no snaps left between 20 and 4 (there
actually can't be since there would have to be another clone there),
performing a second delete at 4:[4] is harmless since it will be
promoted to some snap_seq seq where 4 <= seq < 21, which is fine
since we will be sending a copyfrom at 20.

Fixes: #9054
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
DNM: working