PG: CephPeeringEvt

CephPeeringEvt is now the supertype for all peering state machine
events.  This will allow us to generalize checking for stale peering
events and delaying events for future maps.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: Move handle_* methods to PG

PG now calls handle_event in RecoveryState.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: Remove handle_pg_missing, MOSDPGMissing no longer used

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
OSD,PG: push message checking to pg

old_peering_evt now checks CephPeeringEvts generically in
PG::handle_peering_event().

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: move history update from handle_pg_query into pg

Previously, replica history was updated in OSD::handle_pg_query.
Updating the history is now handled in the pg state machine.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: use intrusive_ptr in CephPeeringEvt

Properly disposing of the event_base member of CephPeeringEvt
requires use of intrusive_ptr.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: process peering events in a queue

Peering events are now queued via queue_peering_event in the
peering_queue.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD,PG: Move Op,SubOp queueing into PG

PG now handles delaying/discarding messages since pg map epoch may not
be the same as the OSD map.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD,PG: handle pg map advance in process_peering_event

The pg map will now be advanced in process_peering_event (in advance_pg)
to allow handle_osd_map to not grab pg locks in-line.  handle_osd_map
queues NullEvts to ensure that each pg is updated in a timely fashion.

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: Make pg osdmap be independent of osd, other pg maps

This will allow handle_osd_map to not stop other work queues.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: include info and query by value in peering events

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: initialize pgs in get_or_create_pg via handle_create

Previously, pgs were initialized via Info/Log/etc.  Since the event
which triggered the pg creation may now be queued, map update events may
occur before the event is processed.  Thus, get_or_create_pg now handles
the initialization prior to queuing the event.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: add debugging to handle_peering_event

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: CephPeeringEvents can now be descriptively printed

Rather than inheriting from boost::statechart::event<> directly,
events now inherit from CephEvent<>, which includes a print method.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: compound messages must carry epoch_sent for each part

Query and Notify messages include logical messages from multiple
pgs.  Each logical message (pg_query_t and pg_notify_t) now
contains an epoch_sent.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: use osd->requeue_ops for ops, pg->queue_for_peering to requeue pg

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: check for deleting in process_peering_event

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,OSD: add OSD::queue_for_op, use in PG::queue_op

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,OSD: fix op checking in pg, take_waiters during ActMap

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: flush ops by the end of peering without osr.flush

Rather than explicitely flushing the filestore, send a noop through the
filestore at the beginning of peering and, at the end, wait for it to
finish by adding an extra state.

Signed-off-by: Samuel Just <sam.just@inktank.com>