PG::init: clear rollback info for backfill as well

Otherwise, we won't remove the old rollback objects from a resurrected pg.  In
rare cases, this can cause us to get an EEXIST if we happen to reuse the same
rename id on the same object in a subsequent interval.

Fixes: #9293
Related to: 8346e10755027e982f26bab4642334fd91cc31aa
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/osd_types: add num_bytes_hit_set_archive to stats

The tiering agent's life is easier/better if it knows how many bytes these
objects consume (instead of making a conservative guess).

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: account for hit_set_archive bytes

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: scrub and repair hit_set_archive bytes

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: calculate num_user_bytes

This is the bytes stored by the user, excluding hit_sets.

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: adjust avg_size calculation in agent_choose_mode

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
messages/MWatchNotify: include an error code in the message

Document the fields, while we are here.

Signed-off-by: Sage Weil <sage@redhat.com>
osd: include ETIMEDOUT in notify reply on timeout

If a notify operation times out (all watchers to not ACK in time), include
an ETIMEDOUT in the final error message back to the client, so that they
know about it.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: rename watch/notify callback register functions

Make it clear these are for watch OR notify, not just watch.  I was
confused.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: prefix msg handler with handle_

Signed-off-by: Sage Weil <sage@redhat.com>
librados: refactor watch/notify; return notify error code

Get rid of a level of intermediate classes with confusing names and put
the notify and notify finish logic in a single place so that it is easier
to follow and understand.

Pass the return value from the notify completion message to the caller.

Fixes: #9193
Signed-off-by: Sage Weil <sage@redhat.com>
Merge remote-tracking branches 'origin/wip-9179', 'upstream/wip-hitset-bytes', 'upstream/wip-9193' and 'origin/wip-9293' into wip-sam-testing