osd: track per-pg epochs, min

Add some simple tracking so that we can quickly determine what the min
pg osdmap epoch is.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: prevent pgs from getting too far ahead of the min pg epoch

Bound the range of PG epochs between the slowest and fastest pg
(epoch-wise) with 'osd map max advance'.  This value should be set to
something less than 'osd map cache size' so that the maps we are
processing will be in memory as many PGs advance forward in time in
loose synchrony.

This is part of the solution to #7576.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: ignore MarkMeDown message if we aren't in PREPARING_TO_STOP state

If we aren't waiting for this, ignore it.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: fix 'ack' to be 'request_ack' in MOSDMarkMeDown

This field was passed along but always set to false.  It did not seem to
indicate whether this was/wasn't an ack (the reply message kept the same
value) or whether an ack was requested (it was always false but we always
reply).

Change this by bumping the message encoding, renaming the field, and
making old messages that are decoded force request_ack to true.  Reply
messages will always have request_ack false (although that part doesn't
matter much since the fact that it is a reply is clear from context).

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: do not reply to MOSDMarkMeDown if ack is not requested

If a reply isn't requested, do not bother to send one.  Note that old
clients did not request an ack, but we will infer that they do from the
message encoding (see previous commit).

Signed-off-by: Sage Weil <sage@inktank.com>
osd: mark ourselves down if we are slow processing maps

If we get are not able to get the pgs to keep up with new osdmaps, ask the
mon to mark us down.  This will let us avoid all of the other work so that
we can catch up.  Also, once we are down we will drain peering_wq on each
map message.

Introduce a new tunable to control how far behind we have to be before this
kicks in.

Fixes: #7576
Signed-off-by: Sage Weil <sage@inktank.com>
Merge remote-tracking branch 'gh/wip-7576' into wip-sam-testing