hobject_t: make max private

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: maintain osd local last_epoch_started for find_best_info

In order to proceed with peering, we need an osd with a log including
the last commit sent to a client.  This translates to the oldest
last_update from the infos of the most recent acting set to go active.
history.last_epoch_started gives us a lower bound on the last time the
entire acting set persisted authoratative logs/infos.  However, it
doesn't indicate anything about the info/log on the osd which sent it.
Thus, we will maintain an osd local info.last_epoch_started to determine
which osds were actually active (and thus have the required log
entries).  The max info.last_epoch_started in the prior set gives us an
upper bound on the last interval during which writes occurred.  The min
last_update among the infos with that last_epoch_started must therefore
be an upper bound on the oldest operation which clients consider
committed.  Any osd with an info.last_updated past that version must be
sufficient.

The observed bug was there was an empty pg info with a
last_epoch_started at the most recent interval which pushed
min_last_update_acceptable to eversion_t().  There were two down osds,
but peering proceeded since the backfill peer did survive.  However,
its info was later disregarded due to incomplete.  An empty osd was
then chosen as the best_info since it's last_update was equal to
min_last_update_acceptable.  This caused the contents of the pg to be
lost.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: history.last_epoch_started should start at 0

history.last_epoch_started marks a lower bound on the last epoch at
which the pg went active.  As with info.last_epoch_started, it should be
0 prior to the first activation.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: get_or_create_pg return null if pool is gone

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: remove from snap_collections even without objects to trim

Also, make sure to write_info after updating snap_collections.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: scrubber.end should be exactly a boundary

Let scrubber.end be (foo, HEAD, 10) where the oid is foo , HEAD is the
snap, and 10 is the hash and scrubber.begin similarly be (bar, 5, 1).

After choosing to scan [(bar, 5, 1), (foo, HEAD, 10)), we block writes
on that interval.

1) A write might then come in for foo (which isn't blocked) which
creates a new snap (foo, 400, 10) which happens to fall in the interval.
This will result in a crash in _scrub() when it attempts to compare
clones since it will get (foo, 400, 10) but not the head object
(foo, HEAD, 10).

2) Alternately, the write from 1) has already happened.  When we scan
the log, we find 34'10 and 34'11 are the clone operation creating
(foo, 400, 10) and the modify on (foo, HEAD, 10) respectively.  Both
primary and replica will wait for last_update_applied to be 34'10
before scanning, but last_update_applied will in fact skip to 34'11
since 34'10 and 34'11 happened in the same transaction.  This can
result in IO hanging on the scrubber interval.

Instead, we ensure that scrubber.end is exactly a hash boundary
(min hobject_t a with the specified hash).  No such object can
exist since we don't create objects with empty oids, so no writes
can occur on that object.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge remote-tracking branch 'gh/wip_next_bugs' into next