filestore: sync object_map object in lfn_remove when nlink > 1

In the following sequence:

1) create (a, 1)
2) setattr (a, 1)
3) link (a, 1), (b, 1)
4) remove (a, 1)

If we play 1-4 and then replay 1-4 again, we will end up removing
(b, 1)'s attributes since nlink for (a, 1) the second time through
is 1.  We fix this by marking spos on the object_map header for
(a, 1) when we remove (a, 1) but not eh attributes.

Signed-off-by: Samuel Just <sam.just@inktank.com>
DBObjectMap: clones must inherit spos from parent

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: CephPeeringEvt

CephPeeringEvt is now the supertype for all peering state machine
events.  This will allow us to generalize checking for stale peering
events and delaying events for future maps.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: Move handle_* methods to PG

PG now calls handle_event in RecoveryState.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: Remove handle_pg_missing, MOSDPGMissing no longer used

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
OSD,PG: push message checking to pg

old_peering_evt now checks CephPeeringEvts generically in
PG::handle_peering_event().

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: move history update from handle_pg_query into pg

Previously, replica history was updated in OSD::handle_pg_query.
Updating the history is now handled in the pg state machine.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: use intrusive_ptr in CephPeeringEvt

Properly disposing of the event_base member of CephPeeringEvt
requires use of intrusive_ptr.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: process peering events in a queue

Peering events are now queued via queue_peering_event in the
peering_queue.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD,PG: Move Op,SubOp queueing into PG

PG now handles delaying/discarding messages since pg map epoch may not
be the same as the OSD map.

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: Make pg osdmap be independent of osd, other pg maps

This will allow handle_osd_map to not stop other work queues.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD,PG: handle pg map advance in process_peering_event

The pg map will now be advanced in process_peering_event (in advance_pg)
to allow handle_osd_map to not grab pg locks in-line.  handle_osd_map
queues NullEvts to ensure that each pg is updated in a timely fashion.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: include info and query by value in peering events

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: initialize pgs in get_or_create_pg via handle_create

Previously, pgs were initialized via Info/Log/etc.  Since the event
which triggered the pg creation may now be queued, map update events may
occur before the event is processed.  Thus, get_or_create_pg now handles
the initialization prior to queuing the event.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: CephPeeringEvents can now be descriptively printed

The CephPeeringEvt constructor is now templated to allow
storing a description string for debugging.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: compound messages must carry epoch_sent for each part

Query and Notify messages include logical messages from multiple
pgs.  Each logical message (pg_query_t and pg_notify_t) now
contains an epoch_sent.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: use osd->requeue_ops for ops, pg->queue_for_peering to requeue pg

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: check for deleting in process_peering_event

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,OSD: add OSD::queue_for_op, use in PG::queue_op

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,OSD: fix op checking in pg, take_waiters during ActMap

Signed-off-by: Samuel Just <sam.just@inktank.com>