PG: maintain osd local last_epoch_started for find_best_info

In order to proceed with peering, we need an osd with a log including
the last commit sent to a client.  This translates to the oldest
last_update from the infos of the most recent acting set to go active.
We can determine the last_epoch_started of that interval from the max of
history.last_epoch_started over all infos (since we know we have at
least one from each acting set back to last_clean).  However, for any
particular osd, info.history.last_epoch_started does not allow us to
conclude that the osd actually has logs from that interval.  Thus, we
will maintain an osd local last_epoch_started to determine which osds
were actually active (and thus have the required log entries).

The observed bug was there was an empty pg info with a
last_epoch_started at the most recent interval which pushed
min_last_update_acceptable to eversion_t().  There were two down osds,
but peering proceeded since the backfill peer did survive.  However,
its info was later disregarded due to incomplete.  An empty osd was
then chosen as the best_info since it's last_update was equal to
min_last_update_acceptable.  This caused the contents of the pg to be
lost.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: history.last_epoch_started should start at 0

history.last_epoch_started defines a minimum permissible
info.last_epoch_started for the chosen log during calc_acting.
Thus, at creation time, info.last_epoch_started must be 0 since
no osd has gone active in that pg.  history.last_epoch_started
thus must also be 0.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: get_or_create_pg return null if pool is gone

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: remove from snap_collections even without objects to trim

Also, make sure to write_info after updating snap_collections.

Signed-off-by: Samuel Just <sam.just@inktank.com>
os/: add filestore collection_split

Signed-off-by: Samuel Just <sam.just@inktank.com>
test/store_test: add simple tests for collection_split

Signed-off-by: Samuel Just <sam.just@inktank.com>
os/: Add failure CollectionIndex failure injection

Several pieces of HashIndex involve multi-step operations
which are sensitive to OSD crashes.  This patch introduces
failure injection to force retries from various points in
the LFNIndex helper methods to be used with store_test.cc.

Signed-off-by: Samuel Just <sam.just@inktank.com>
os/: Add CollectionIndex::prep_delete

If an unlink is interupted between removing the file
and updating the subdir attribute, the attribute will
overestimate the number of files in the directory.  This
is by design, at worst we will merge the collection later
than intended, but closing the gap would require a second
subdir xattr update.  However, this can in extreme cases
result in a collection with subdirectories but no objects.
FileStore::_destry_collection would therefore see an
erroneous -ENOTEMPTY.

prep_delete allows the CollectionIndex implementation to
clean up state prior to removal.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: don't wait for superblock writes in handle_osd_map

Instead, pass the pinned maps into a Context and clear the
cache after the transaction is applied.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: dispatch_context only discard transaction if contexts empty

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGMonitor,OSD: don't send creates on split

Splits will be handled when the map update effecting the split is
processed for the splitting pg on each OSD.  This will mesh
with the pg history which will place the new pg at the current
positions of the splitting pg.

Signed-off-by: Samuel Just <sam.just@inktank.com>
mon/OSDMonitor: enable split in Monitor

Signed-off-by: Samuel Just <sam.just@inktank.com>
PrioritizedQueue: allow caller to get items removed by removed_by_filter

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: splitting a pg now triggers a new interval

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: add split_into to populate child members

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: add initial split support

PGs are split after updating to the map on which they split.
OSD::activate_map populates the set of currently "splitting"
pgs.  Messages for those pgs are delayed until the split
is complete.  We add the newly split children to pg_map
once the transaction populating their on-disk state completes.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: split ops for child objects into child

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: mark info.stats as invalid after split, fix in scrub

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: dirty info and log on child during split

Otherwise, the log may not get written out.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: do _remove_pg in add_newly_split_pg is pool if gone

Signed-off-by: Samuel Just <sam.just@inktank.com>