OSD,PG: Move Op,SubOp queueing into PG

PG now handles delaying/discarding messages since pg map epoch may not
be the same as the OSD map.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD,PG: handle pg map advance in process_peering_event

The pg map will now be advanced in process_peering_event (in advance_pg)
to allow handle_osd_map to not grab pg locks in-line.  handle_osd_map
queues NullEvts to ensure that each pg is updated in a timely fashion.

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: Make pg osdmap be independent of osd, other pg maps

This will allow handle_osd_map to not stop other work queues.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: include info and query by value in peering events

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: initialize pgs in get_or_create_pg via handle_create

Previously, pgs were initialized via Info/Log/etc.  Since the event
which triggered the pg creation may now be queued, map update events may
occur before the event is processed.  Thus, get_or_create_pg now handles
the initialization prior to queuing the event.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: add debugging to handle_peering_event

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: CephPeeringEvents can now be descriptively printed

Rather than inheriting from boost::statechart::event<> directly,
events now inherit from CephEvent<>, which includes a print method.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: compound messages must carry epoch_sent for each part

Query and Notify messages include logical messages from multiple
pgs.  Each logical message (pg_query_t and pg_notify_t) now
contains an epoch_sent.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: use osd->requeue_ops for ops, pg->queue_for_peering to requeue pg

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: check for deleting in process_peering_event

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,OSD: add OSD::queue_for_op, use in PG::queue_op

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: flush ops by the end of peering without osr.flush

Rather than explicitely flushing the filestore, send a noop through the
filestore at the beginning of peering and, at the end, wait for it to
finish by adding an extra state.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: remove superfluous pg get/put around enqueue_op

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: do not eval_repop if aborted

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: change ReplicatedPG debug output to match PG

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: don't assume pending pg removals have flushed

_create_lock_pg might encounter a preexisting pg collection simply
because the removal transaction had not yet completed.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: do not lock osd during dequeue_op

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: push_waiters is no longer used

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: get map read lock during queue_want_up_thru

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: map_in_progress no longer needed, osd_lock not dropped

Signed-off-by: Samuel Just <sam.just@inktank.com>